SYSTEM all green source hudsonvalleylighting.com queue 12,841 pages p99 latency 184ms dataflirt.com · scraper/hudsonvalleylighting-com
RUN · 17 active pipelines · hudsonvalleylighting.com live

Hudson Valley Lighting data,
at warehouse scale.

We extract product catalogues, technical specifications, finishes, pricing, and collection metadata from Hudson Valley Lighting. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Fixtures extracted
45.2K /run
Spec sheets parsed
38.9K /run
Image assets
184K /run
Active pipelines
17
Uptime
99.98%
Data Dictionary

Every field we extract from hudsonvalleylighting.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Product Specifications objects from hudsonvalleylighting.com. All fields typed and schema-versioned.

skuproduct_namebrandcollectioncategorystylefinishmaterialwidthheightdepthweightlampingvoltageul_ratingdescriptionurl
product_specifications
● 200 OK
"sku": "1234-AGB",
"product_name": "Astor Chandelier",
"brand": "Hudson Valley Lighting",
"collection": "Astor",
"category": "Chandeliers",
"finish": "Aged Brass",
"width": "24.00",
"height": "18.50"
# skuproduct_namebrandcollectioncategorystyle
1
2
3

Complete list of extractable fields for Pricing & Availability objects from hudsonvalleylighting.com. All fields typed and schema-versioned.

skuretail_pricetrade_pricecurrencyin_stockstock_statuslead_timemap_pricediscount_eligibleprice_timestamp
pricing_& availability
● 200 OK
"sku": "1234-AGB",
"retail_price": 850.0,
"currency": "USD",
"in_stock": true,
"stock_status": "In Stock",
"lead_time": "1-2 Weeks",
"price_timestamp": "2026-05-12T09:14:00Z"
# skuretail_pricetrade_pricecurrencyin_stockstock_status
1
2
3

Complete list of extractable fields for Assets & Documents objects from hudsonvalleylighting.com. All fields typed and schema-versioned.

skuprimary_image_urlgallery_image_urlsspec_sheet_urlinstall_guide_urlcad_file_urlies_file_urlvideo_urllifestyle_image_urls
assets_& documents
● 200 OK
"sku": "1234-AGB",
"primary_image_url": "https://example.com/images/1234-AGB_main.jpg",
"spec_sheet_url": "https://example.com/docs/1234-AGB_spec.pdf",
"install_guide_url": "https://example.com/docs/1234-AGB_install.pdf",
"gallery_image_urls": "['https://example.com/images/1234-AGB_alt1.jpg', 'https://example.com/images/1234-AGB_alt2.jpg']",
"cad_file_url": "None"
# skuprimary_image_urlgallery_image_urlsspec_sheet_urlinstall_guide_urlcad_file_url
1
2
3

Complete list of extractable fields for Lamping & Electrical objects from hudsonvalleylighting.com. All fields typed and schema-versioned.

skubulb_typebulb_basewattagevoltagecolor_temperaturecrilumensdimmabledark_sky_compliantada_compliant
lamping_& electrical
● 200 OK
"sku": "1234-AGB",
"bulb_type": "LED",
"bulb_base": "E26 Medium",
"wattage": "60W",
"voltage": "120V",
"dimmable": true,
"ada_compliant": false
# skubulb_typebulb_basewattagevoltagecolor_temperature
1
2
3

Complete list of extractable fields for Collection Metadata objects from hudsonvalleylighting.com. All fields typed and schema-versioned.

collection_idcollection_namebranddesignerdescriptionhero_image_urltotal_productscategories_includedfinishes_availableurl
collection_metadata
● 200 OK
"collection_name": "Astor",
"brand": "Hudson Valley Lighting",
"designer": "In-House",
"total_products": 12,
"categories_included": "['Chandeliers', 'Sconces', 'Pendants']",
"finishes_available": "['Aged Brass', 'Polished Nickel']"
# collection_idcollection_namebranddesignerdescriptionhero_image_url
1
2
3

Capabilities

Extracting the complete Hudson Valley catalogue

Our scraper navigates the complex taxonomy of Hudson Valley Lighting, capturing detailed technical specifications, finish variants, and high-resolution assets while handling dynamic product configurations.

Technical Spec Extraction

Capture exact dimensions, weight, UL ratings, and material specifications for every fixture, normalised across the entire catalogue.

Finish & Variant Mapping

Extract all available finishes for a base SKU, mapping parent-child relationships and capturing finish-specific imagery.

Lamping Details

Parse bulb requirements, wattage limits, voltage, and dimmability specifications critical for electrical planning.

Document Asset Links

Scrape direct URLs for PDF spec sheets, installation guides, and IES files associated with each product.

High-Res Imagery

Capture primary product shots, secondary angles, and lifestyle imagery at the highest available resolution.

Brand & Collection Taxonomy

Maintain the hierarchical structure of brands (Mitzi, Troy, Corbett) and collections for accurate categorisation.

Pricing Signals

Extract retail pricing and track changes over time. (Note: Trade pricing requires authenticated access).

Inventory Indicators

Capture stock status and lead time estimates displayed on product pages.

Change Detection

Run continuous pipelines with hash-based diffing to detect new product launches, discontinued items, or spec updates.

// engagement pipeline

From target catalogue to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Select specific brands, categories, or collections. We design the extraction schema together.

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers, handle pagination, and manage dynamic content rendering.

Validation & QA
d 4–6

Schema validation, null-rate checks on critical specs, and variant mapping verification before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

Navigating complex B2B catalogues

Lighting catalogues present unique data challenges: complex variant structures, technical specs buried in PDFs, and dynamic rendering. Here is how we handle it.

pipeline-monitor · hudsonvalleylighting.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Variant Explosion
Mapping base SKUs to finish options

Lighting fixtures often have a base SKU with multiple finish options (e.g., Aged Brass, Polished Nickel). Our pipeline explodes these into discrete records, ensuring every finish variant has its own row with accurate pricing and imagery.

Spec Parsing
Standardising technical data

Technical specifications are often presented in unstructured formats or tabs. We use targeted selectors and regex patterns to extract and normalise dimensions (width, height, depth), lamping details, and electrical ratings into structured fields.

Asset Management
Capturing document and image URLs

We extract direct links to high-resolution images, spec sheet PDFs, and installation guides. This ensures your downstream systems have access to the necessary assets without needing to host them immediately.

Brand Taxonomy
Maintaining catalog hierarchy

Hudson Valley Lighting Group encompasses multiple brands (Hudson Valley, Mitzi, Troy, Corbett). Our crawler respects this taxonomy, ensuring products are correctly attributed to their respective brand and collection.

Dynamic Rendering
Handling JavaScript-heavy pages

Product configurators and inventory checks often rely on JavaScript. We utilise Playwright to fully render pages, interact with finish selectors, and capture the resulting data that headless HTTP clients miss.

Applications

Who uses lighting catalogue data

Teams across industries use hudsonvalleylighting.com data to build competitive products and smarter operations.

01
B2B Distributors & Showrooms

Automate catalogue ingestion, ensuring accurate specs, finishes, and pricing are reflected in internal ERP and e-commerce systems.

02
Interior Design Platforms

Populate design software and specification tools with accurate product dimensions, lamping details, and 3D asset links.

03
Competitor Analysis

Lighting manufacturers track pricing strategies, new collection launches, and material trends across competing brands.

04
PIM Synchronisation

Keep Product Information Management (PIM) systems updated with the latest manufacturer specifications and discontinued SKU alerts.

05
Market Research

Analyse category saturation, popular finishes, and pricing tiers within the decorative lighting market.

06
Procurement & Sourcing

Commercial builders and procurement teams aggregate specifications to verify UL ratings and ADA compliance for large projects.

Why DataFlirt

"Accurate technical specifications are the lifeblood of the lighting industry. Without structured data, distributors and specifiers are left manually copying PDFs."

Extracting data from B2B lighting catalogues requires more than simple web scraping. It demands an understanding of variant structures, technical terminology, and complex taxonomies. DataFlirt builds pipelines that translate unstructured product pages into the rigorous schemas required by modern ERP and PIM systems.

Technical Spec

Hudson Valley Lighting scraper — technical capabilities

Everything supported by our hudsonvalleylighting.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Playwright sessions for dynamic finish selection and inventory checks
Supported
Variant mapping
Explodes base SKUs into discrete finish variants
Supported
PDF link extraction
Captures direct URLs for spec sheets and install guides
Supported
Dimension normalisation
Parses unstructured dimension strings into structured width/height/depth fields
Supported
High-res image capture
Extracts highest available resolution URLs for primary and gallery images
Supported
Brand taxonomy
Maintains correct attribution across Mitzi, Troy, Corbett, and HVL
Supported
Change detection
Hash-based diffing to identify new launches and discontinued SKUs
Supported
Trade pricing
Requires authenticated dealer portal access
Partial
Real-time inventory levels
Exact stock counts are often gated behind dealer logins
Partial
Infrastructure

Infrastructure powering the pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration and deduplication. Playwright handles JavaScript rendering for dynamic product configurators.

Custom Parsers

Domain-specific parsers normalise complex lighting specifications (lamping, dimensions, UL ratings) into structured formats.

Cloud-Native Orchestration

Pipelines run on AWS Lambda and ECS. Airflow handles scheduling, dependency management, and SLA alerting.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested — schema versioned per run
CSV
Flat file with typed columns — Excel/Sheets compatible
Parquet
Columnar format for BigQuery, Snowflake, Athena
S3
Direct bucket delivery — compatible with any data lake
Webhook
HTTP POST per record for real-time downstream processing
BigQuery
Streamed directly into your dataset
Postgres
Upsert into your existing schema
Snowflake
Stage + COPY INTO workflow
// faq

Common questions.

About hudsonvalleylighting.com scraping, legality, and pipeline operations.

Ask us directly →
Can you extract all finish options for a fixture?

Yes. Our pipeline interacts with the product configurator to select each available finish, capturing the specific SKU, imagery, and pricing associated with that variant.

Do you download the PDF spec sheets?

By default, we extract the direct URLs to the PDF spec sheets and installation guides. If you require the actual files downloaded and delivered to your S3 bucket, this can be configured.

How do you handle the different brands under Hudson Valley Lighting Group?

The scraper navigates the entire hudsonvalleylighting.com domain, correctly attributing products to their respective brands (Hudson Valley Lighting, Mitzi, Troy Lighting, Corbett Lighting) based on the site's taxonomy.

Can you get trade pricing?

Trade pricing and exact inventory counts are typically gated behind a dealer login portal. We do not circumvent authentication walls. We extract the publicly available retail pricing and stock status indicators.

How often can the data be refreshed?

For full catalogue extractions, we typically recommend weekly or monthly cadences to capture new releases and discontinued items. More frequent runs can be configured for specific categories if required.

Are the dimensions normalised?

Yes. We parse the unstructured dimension strings on the product pages and map them to structured width, height, and depth fields to facilitate easy ingestion into your PIM or ERP.

$ dataflirt scope --new-project --source=hudsonvalleylighting.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Stop manually copying specifications. We scope, build, and operate the pipeline to deliver structured Hudson Valley Lighting data directly to your systems.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in lighting

Services

Data Extraction for Every Industry

View All Services →