We extract designer lighting specifications, finish variants, pricing, and dimensional data from Shades of Light. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Product Specs objects from shadesoflight.com. All fields typed and schema-versioned.
"sku": "CH19042", "title": "Modern Brass Sputnik Chandelier", "category": "Ceiling Lights > Chandeliers", "style": "Mid-Century Modern", "width_inches": 32.5, "height_inches": 24.0, "bulb_type": "E12 Candelabra", "max_wattage": 60
| # | sku | title | designer | category | style | material |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Pricing & Stock objects from shadesoflight.com. All fields typed and schema-versioned.
"sku": "CH19042-BRS", "variant_price": 845.0, "currency": "USD", "stock_status": "In Stock", "lead_time_days": 3, "trade_discount_eligible": true, "shipping_surcharge": 45.0, "price_timestamp": "2026-05-12T09:14:00Z"
| # | sku | base_price | variant_price | currency | stock_status | lead_time_days |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Categorisation objects from shadesoflight.com. All fields typed and schema-versioned.
"sku": "CH19042", "primary_category": "Ceiling Lights", "sub_category": "Chandeliers", "room_type": "['Dining Room', 'Living Room']", "style_tag": "Mid-Century Modern", "collection_name": "Sputnik Series", "url": "https://www.shadesoflight.com/products/modern-brass-sputnik-chandelier"
| # | sku | primary_category | sub_category | room_type | style_tag | collection_name |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Reviews objects from shadesoflight.com. All fields typed and schema-versioned.
"review_id": "REV-88421", "sku": "CH19042", "rating": 5, "author": "Sarah J.", "date": "2026-03-14", "title": "Stunning focal point", "body": "Perfect scale for our 10-foot ceilings. The brass finish is high quality.", "verified_buyer": true
| # | review_id | sku | rating | author | date | title |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Visual Assets objects from shadesoflight.com. All fields typed and schema-versioned.
"sku": "CH19042", "primary_image_url": "https://cdn.shadesoflight.com/images/CH19042_main.jpg", "variant_image_urls": "['https://cdn.shadesoflight.com/images/CH19042_alt1.jpg']", "finish_swatch_urls": "['https://cdn.shadesoflight.com/images/swatch_brass.jpg']", "assembly_instructions_url": "https://cdn.shadesoflight.com/pdf/CH19042_install.pdf", "video_url": "None"
| # | sku | primary_image_url | variant_image_urls | lifestyle_image_urls | finish_swatch_urls | manual_pdf_url |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Our Shades of Light scraper handles the complexities of high-end decor e-commerce: dynamic finish switchers, complex dimensional specifications, and nested category structures.
Extract and normalise complex measurements including canopy size, chain length, minimum/maximum hanging height, and fixture dimensions.
Capture all available finish options (e.g., Antique Brass, Polished Nickel) mapped to their specific SKUs, pricing, and variant images.
Extract wattage limits, bulb base types (E12, E26, LED integrated), voltage requirements, and dimmer compatibility.
Retrieve URLs for uncompressed product imagery, lifestyle room shots, finish swatches, and PDF installation manuals.
Track base price, variant-specific pricing, sale discounts, and oversized item shipping surcharges.
Monitor inventory status, backorder dates, and estimated shipping lead times per variant.
Preserve the full site taxonomy, extracting room recommendations, style tags, and designer collection groupings.
Paginate through customer reviews to extract star ratings, text bodies, and verified buyer status.
Run scheduled pipelines that only output records when prices, stock status, or specifications change.
Brief in. Clean data out.
Specify target categories, collections, or specific SKUs. We configure the extraction schema to match your data model.
We deploy Scrapy / Playwright crawlers, configuring proxy rotation and JavaScript rendering for dynamic variant selectors.
We run schema validation, check dimension normalisation logic, and ensure variant matrices are complete.
Data is pushed as JSON, CSV, or Parquet to your designated S3 bucket or data warehouse on schedule.
Extracting structured data from Shades of Light requires specific technical approaches to handle dynamic content and non-standard specification tables.
Pricing, SKUs, and images change dynamically when a user selects a finish option. We use Playwright to interact with the DOM, triggering these state changes to extract the full matrix of variant data.
Specification tables often contain mixed units and free-text descriptions. Our pipeline applies regular expressions to parse strings like 'H: 24in W: 32.5in' into distinct numerical fields for database ingestion.
High-resolution product images and lifestyle shots are lazy-loaded. We simulate scrolling and intercept network requests to capture the source URLs of the highest quality assets available.
E-commerce platforms deploy WAFs to block aggressive crawling. We distribute requests across US-based residential IP pools with automated retry logic to maintain continuous extraction without blocks.
We compute a hash for each SKU's data state. Subsequent runs only emit records where the hash has changed, providing an efficient feed of price updates and stock changes.
Lighting retailers track pricing across identical or similar designer fixtures to adjust their own pricing strategies.
Merchandising teams analyse category depth, finish popularity, and price architecture to inform inventory decisions.
Platforms ingest product specs and imagery to populate unified catalogues for professional interior designers.
Computer vision and spatial AI models use dimensional data and high-res imagery to train 3D rendering systems.
Lighting manufacturers audit retail prices to ensure compliance with Minimum Advertised Price agreements.
Analysts track the introduction of new styles, materials, and finishes over time to identify emerging design trends.
"Designer lighting catalogues contain hundreds of dimensional and finish variables per SKU — extracting this requires precise schema mapping, not just basic web crawling."
Most teams underestimate the complexity of extracting structured dimensional specs and variant matrices from premium decor sites. DataFlirt handles the JavaScript rendering for dynamic finish selectors and standardises messy specification tables so your engineers can focus on analysis.
Everything supported by our shadesoflight.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, cookie sessions, and interaction flows. Combined via scrapy-playwright middleware.
We maintain pools of residential ISP proxies across IN/US/UK/DE regions. Rotation happens per-request with sticky sessions where required. IP score monitoring prevents blacklisted pool contamination.
Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.
Data delivered to where your team already works — no new tooling required.
About shadesoflight.com scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available catalogue and pricing data is generally permissible. DataFlirt extracts only public, non-authenticated product information. We do not bypass login walls to access Trade Program pricing or user data. Clients should review the target site's ToS and consult legal counsel for their specific use cases.
We use Playwright to render the JavaScript on the product page, programmatically clicking each finish option to trigger the DOM updates. This allows us to capture the specific SKU, price, and image URL for every variant combination.
Yes. We apply regex patterns to the unstructured specification text to output clean, numerical fields for width, height, weight, and chain length, standardising the units across the catalogue.
We can configure pipelines to run daily, hourly, or continuously. For inventory monitoring, we recommend a daily full-catalogue run with high-frequency checks on a subset of critical SKUs.
Yes. We intercept the network requests made by the image viewer to locate the source URLs for the highest quality uncompressed images, rather than scraping the low-resolution thumbnails.
Our minimum engagement typically starts at a defined category extraction (e.g., all Chandeliers) delivered weekly. Contact us to scope a custom pipeline based on your exact data volume requirements.
Yes. We provide a sample extraction of up to 100 SKUs during the scoping phase so your engineering team can review the schema, dimensional parsing, and variant mapping before committing.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off catalogue export or a continuous feed of pricing and stock updates — we scope, build, and operate the pipeline. Tell us what you need.
