We extract used equipment listings, CNC machinery specs, clearance pricing, and warehouse location data from HGR Industrial Surplus. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Machinery Listings objects from hgrinc.com. All fields typed and schema-versioned.
"item_id": "0621045", "title": "Haas VF-2 CNC Vertical Machining Center", "manufacturer": "Haas", "model": "VF-2", "price": 24500.0, "condition": "Used", "warehouse_location": "Aisle 14, Bin B"
| # | item_id | title | manufacturer | model | category | sub_category |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Pricing & Clearance objects from hgrinc.com. All fields typed and schema-versioned.
"item_id": "0621045", "current_price": 24500.0, "original_price": 28000.0, "markdown_pct": 12.5, "clearance_flag": false, "make_offer_eligible": true, "price_timestamp": "2023-10-24T08:00:00Z"
| # | item_id | current_price | original_price | markdown_pct | clearance_flag | make_offer_eligible |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Specifications objects from hgrinc.com. All fields typed and schema-versioned.
"item_id": "0621045", "voltage": "220V", "phase": "3-Phase", "dimensions_lwh": "102 x 98 x 105 in", "weight_lbs": 7800.0, "motor_hp": "20", "serial_number": "45892"
| # | item_id | voltage | phase | dimensions_lwh | weight_lbs | motor_hp |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for MRO & Parts objects from hgrinc.com. All fields typed and schema-versioned.
"item_id": "099812", "part_number": "6204-2RS", "brand": "SKF", "quantity_available": 45, "lot_size": 10, "package_type": "Box", "material": "Steel"
| # | item_id | part_number | brand | quantity_available | lot_size | package_type |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Category Taxonomy objects from hgrinc.com. All fields typed and schema-versioned.
"category_id": "cat-cnc", "category_name": "CNC Machinery", "parent_category": "Machinery", "url_slug": "/machinery/cnc", "item_count": 452, "new_arrivals_count": 12, "clearance_count": 34
| # | category_id | category_name | parent_category | url_slug | item_count | new_arrivals_count |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Our HGR scraper handles the complexities of industrial surplus: high turnover rates, unstructured description text, and massive image galleries. We normalise the data into clean schemas.
Title, manufacturer, model, dimensions, weight, and images scraped at the item level across all active categories.
Track price drops, clearance events, and original vs current pricing to identify arbitrage opportunities.
Detect new items added to the floor daily before they are sold out or moved.
Extract structured specifications like voltage, phase, and horsepower from unstructured description blocks using NLP.
Capture aisle and bin locations for logistics planning and warehouse walkthroughs.
Download and link high-resolution equipment photos directly to your cloud storage.
Reconstruct HGR's category tree to track inventory density across different machinery types.
Identify items eligible for negotiation to optimise your purchasing strategy.
Only emit records when price, location, or availability status changes to reduce data bloat.
Identify when items are sold or removed from the floor to keep your database accurate.
Brief in. Clean data out.
Provide category URLs, manufacturer lists, or keyword sets. We design the extraction schema together.
We configure Scrapy crawlers, proxy rotation, and session management for hgrinc.com.
Schema validation, null-rate checks, and price-outlier detection before full launch.
JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.
Used machinery marketplaces present unique scraping challenges. Here is how we maintain data integrity.
We use rotating residential IPs to avoid IP bans and rate limiting while scanning thousands of inventory pages.
HGR descriptions are often unstructured text blocks. We use regex and NLP models to extract structured fields like voltage, phase, and dimensions.
Used machinery moves fast. Our change detection logic identifies items that appear and disappear within hours, ensuring you never miss a deal.
Industrial equipment listings contain dozens of high-res images. We efficiently scrape and route these assets directly to your S3 bucket.
We maintain a state file of all active inventory. Subsequent runs only push updates when prices drop or items change location.
Identify underpriced equipment and clearance markdowns for profitable resale.
Benchmark used equipment values across categories to inform purchasing decisions.
Automate alerts for specific machinery models or hard-to-find MRO parts.
Calculate potential scrap value based on extracted weight and material specifications.
Monitor HGR's inventory turnover rates and pricing strategies across different asset classes.
Train computer vision models on thousands of classified industrial equipment images.
"HGR Industrial Surplus moves thousands of items daily. Tracking this high-velocity inventory requires continuous extraction, not manual browsing."
Most teams underestimate the complexity of scraping used machinery marketplaces. Unstructured descriptions, rapid inventory turnover, and rate limiting make internal scripts brittle. DataFlirt builds resilient pipelines that normalise messy industrial data into clean, queryable warehouse records so your engineers can focus on analysis.
Everything supported by our hgrinc.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering and interaction flows.
We maintain pools of residential ISP proxies to avoid IP bans while scanning thousands of inventory pages.
Pipelines run on AWS Lambda and ECS. Airflow handles scheduling, dependency management, and alerting.
Data delivered to where your team already works — no new tooling required.
About hgrinc.com scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available inventory data from hgrinc.com is generally permissible. DataFlirt targets only public, non-authenticated product and pricing data.
We use regex and NLP parsing models to extract structured specifications like voltage, phase, horsepower, and dimensions from messy text blocks.
We configure pipelines based on your needs. Daily runs capture all new arrivals and markdowns, while hourly runs can monitor specific high-value categories.
Yes. Every pipeline run produces timestamped snapshots. We track current price against original price to calculate markdown percentages.
Yes. We can provide direct image URLs in the dataset or download the assets and push them directly to your S3 bucket.
Our packages start at defined category lists with weekly delivery. For full catalogue extraction, we price based on volume and frequency.
Yes. We provide a sample run of up to 500 items to validate schema fit, specification parsing accuracy, and data quality before signing a contract.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off machinery export or a continuous inventory feed: we scope, build, and operate the pipeline. Tell us what you need.
