We extract photonics components, optomechanics, pricing, and stock availability from Thorlabs. Delivered as clean JSON, CSV, or Parquet to S3 or PostgreSQL on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Optomechanics objects from thorlabs.com. All fields typed and schema-versioned.
"item_number": "TR50/M", "description": "Optical Post, SS, M4 Tapped Hole, 12.7 mm Diameter, 50 mm Length", "price": 6.34, "stock_status": "In Stock", "lead_time_days": 0, "imperial_metric": "Metric", "thread_type": "M4", "material": "416 Stainless Steel"
| # | item_number | description | price | stock_status | lead_time_days | rohs_status |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Optical Specs objects from thorlabs.com. All fields typed and schema-versioned.
"item_number": "LA1131-B", "optical_type": "Plano-Convex Lens", "wavelength_range_nm": "650 - 1050", "ar_coating": "B (650-1050 nm)", "focal_length_mm": 50.0, "diameter_mm": 25.4, "surface_quality": "40-20 Scratch-Dig", "substrate": "N-BK7"
| # | item_number | optical_type | wavelength_range_nm | ar_coating | focal_length_mm | diameter_mm |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Pricing & Stock objects from thorlabs.com. All fields typed and schema-versioned.
"item_number": "TR50/M", "base_price": 6.34, "currency": "USD", "volume_pricing_tiers": "1-9: 6.34, 10-24: 5.71, 25+: 5.39", "in_stock_qty": 1450, "backorder_date": "None", "lead_time_days": 0
| # | item_number | base_price | currency | volume_pricing_tiers | in_stock_qty | backorder_date |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for CAD & Documents objects from thorlabs.com. All fields typed and schema-versioned.
"item_number": "TR50/M", "solidworks_model_url": "https://www.thorlabs.com/cad/TR50_M.sldprt", "step_model_url": "https://www.thorlabs.com/cad/TR50_M.step", "dxf_model_url": "https://www.thorlabs.com/cad/TR50_M.dxf", "drawing_pdf_url": "https://www.thorlabs.com/drawings/TR50_M.pdf", "ce_certification_url": "None", "manual_url": "None"
| # | item_number | manual_url | spec_sheet_url | solidworks_model_url | step_model_url | dxf_model_url |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Hierarchy objects from thorlabs.com. All fields typed and schema-versioned.
"item_number": "TR50/M", "primary_category": "Optomechanics", "sub_category_1": "Optical Posts", "sub_category_2": "Ø1/2 inch Stainless Steel Posts", "breadcrumbs": "Home > Optomechanics > Optical Posts > Ø1/2 inch Stainless Steel Posts", "product_family_url": "https://www.thorlabs.com/newgrouppage9.cfm?objectgroup_id=1295"
| # | item_number | primary_category | sub_category_1 | sub_category_2 | breadcrumbs | related_items |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Thorlabs structures its catalogue with deeply nested tables, complex product families, and dynamic pricing tabs. We normalise this into flat, queryable records.
Extract focal lengths, damage thresholds, and coating curves from complex HTML tables. We map variable column headers to a standardised schema.
Capture base price and volume break tiers. We track currency variants across regional Thorlabs domains.
Monitor exact in-stock quantities, backorder availability dates, and standard lead times for manufacturing planning.
Extract direct URLs for STEP files, SolidWorks models, DXF drawings, and PDF spec sheets associated with every item number.
Capture compatible accessories, recommended mounts, and alternative part numbers listed on product family pages.
Scrape thorlabs.com, thorlabs.de, thorlabs.co.jp, and other regional variants to capture local pricing and stock availability.
Extract RoHS compliance status, CE certification links, and export control classification numbers (ECCN) for procurement audits.
Navigate Thorlabs' multi-tabbed product group pages to extract every individual item number and its specific attributes.
Run daily or weekly diffs. We emit records only when price, stock status, or technical specifications change.
Brief in. Clean data out.
Provide categories, product family URLs, or specific item numbers. We design the extraction schema for your engineering requirements.
We configure Scrapy crawlers to navigate nested product tabs, parse spec tables, and handle Thorlabs' pagination.
Schema validation, null-rate checks on critical specs, and unit normalisation (e.g., metric vs imperial) before launch.
JSON, CSV, or Parquet pushed to your S3 bucket, PostgreSQL database, or via Webhook on your required cadence.
Industrial catalogues present unique scraping challenges: deeply nested tables, inconsistent spec formatting, and dynamic tab rendering.
Thorlabs uses hundreds of different table layouts for optical and mechanical specs. We deploy heuristic parsers that identify key-value pairs regardless of DOM structure, normalising units like 'mm' and 'nm' into strict numeric fields.
Many item numbers are hidden inside JavaScript-rendered tabs on product family pages. We use Playwright to execute click events and expose all nested variants, ensuring zero catalogue coverage gaps.
Stock levels dictate manufacturing timelines. We isolate the specific XHR requests Thorlabs uses to fetch real-time inventory and backorder dates, bypassing heavy page loads for high-frequency stock monitoring.
Optomechanics often mix metric (M4, M6) and imperial (8-32, 1/4-20) threading. We extract and tag these attributes explicitly, preventing mismatched parts in your Bill of Materials.
Thorlabs maintains pages built over the last two decades with varying HTML standards. Our fallback chains handle legacy markup, ensuring data flows even when page templates diverge.
Optical manufacturers track Thorlabs' pricing and volume discounts to optimise their own component pricing strategies.
Engineering teams ingest current pricing directly into their PLM systems to calculate accurate Bill of Materials costs.
R&D departments populate internal component libraries with accurate specs, CAD links, and RoHS statuses.
Procurement teams monitor lead times and backorder dates for critical optical components to prevent production delays.
Research institutions automate the cross-referencing of required laser and optomechanical setups against available budgets.
Industrial distributors map their inventory against Thorlabs part numbers to offer direct drop-in replacements.
"Thorlabs holds the definitive catalogue for photonics and optomechanics, but extracting nested optical specifications requires precision parsing."
Industrial distribution sites rely heavily on complex, nested tables and dynamic tab structures to display technical specifications. DataFlirt parses these variable schemas into normalised records, handling volume pricing and CAD metadata so your procurement systems stay updated.
Everything supported by our thorlabs.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy manages crawl queues and deduplication. Playwright handles JavaScript execution for dynamic product tabs and inventory XHR requests.
Custom Python modules using lxml and BeautifulSoup4 to identify, clean, and normalise complex engineering specification tables.
Airflow schedules daily diff runs. AWS Lambda scales concurrent extraction across thousands of product family pages simultaneously.
Data delivered to where your team already works — no new tooling required.
About thorlabs.com scraping, legality, and pipeline operations.
Ask us directly →Yes. Thorlabs groups multiple item numbers under a single product family URL, often using JavaScript tabs. We use Playwright to navigate these structures and extract every individual item number, its specific price, and its unique technical specifications.
We extract the direct URLs pointing to the STEP, SolidWorks, DXF, and PDF files. We deliver these URLs in the dataset so your internal systems can download the files directly, but we do not host the binary files ourselves.
Thorlabs tables vary wildly between categories. We build custom parsers for major categories (optics, mechanics, lasers) that map variable column headers to a strict schema, ensuring units like 'mm' and 'nm' are separated from numeric values.
Yes. We extract exact 'In Stock' quantities and specific backorder dates. For out-of-stock items, we capture the standard manufacturing lead time in days.
Yes. We can configure pipelines for thorlabs.de, thorlabs.co.jp, and other regional domains to capture local currency pricing and regional warehouse availability.
We support daily or weekly pipeline runs. For high-priority components, we can configure hourly webhook alerts triggered by stock status changes.
We extract the base price and all standard volume discount tiers listed publicly on the item page. We cannot extract customer-specific negotiated pricing that requires an authenticated login.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a complete catalogue extraction for your PLM system or daily stock monitoring for critical components. Tell us your requirements.
