SYSTEM all green source hmart.com queue 12,403 pages p99 latency 185ms dataflirt.com · scraper/hmart-com
RUN · 18 active pipelines · hmart.com live

Hmart grocery data,
delivered at scale.

We extract product listings, regional pricing, stock depth, and ingredient lists from Hmart. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Products extracted
84.2K /run
Price updates
112K /24h
Out-of-stock flags
4.1K /run
Active pipelines
18
Uptime
99.98%
Data Dictionary

Every field we extract from hmart.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Product Listings objects from hmart.com. All fields typed and schema-versioned.

skutitle_entitle_kobrandcategorysub_categorypricesale_pricestock_statusweightimage_urlsurl
product_listings
● 200 OK
"sku": "8801043014748",
"title_en": "Nongshim Shin Ramyun Noodle Soup",
"title_ko": "농심 신라면",
"brand": "Nongshim",
"price": 5.99,
"sale_price": 4.49,
"stock_status": "IN_STOCK",
"weight": "4.2 oz x 4 packs"
# skutitle_entitle_kobrandcategorysub_category
1
2
3

Complete list of extractable fields for Nutritional Data objects from hmart.com. All fields typed and schema-versioned.

skuserving_sizecaloriestotal_fat_gsodium_mgtotal_carbohydrate_gprotein_gingredientsallergensdietary_flags
nutritional_data
● 200 OK
"sku": "8801043014748",
"calories": 500,
"total_fat_g": 16,
"sodium_mg": 1790,
"total_carbohydrate_g": 79,
"protein_g": 10,
"allergens": "['Wheat', 'Soy', 'Fish']"
# skuserving_sizecaloriestotal_fat_gsodium_mgtotal_carbohydrate_g
1
2
3

Complete list of extractable fields for Pricing & Promos objects from hmart.com. All fields typed and schema-versioned.

skubase_pricecurrent_pricediscount_pctpromo_badgeweekly_salevalid_untilstore_idzip_codescraped_at
pricing_& promos
● 200 OK
"sku": "8801043014748",
"base_price": 5.99,
"current_price": 4.49,
"discount_pct": 25,
"promo_badge": "Weekly Sale",
"weekly_sale": true,
"zip_code": "10001",
"scraped_at": "2023-10-24T08:15:00Z"
# skubase_pricecurrent_pricediscount_pctpromo_badgeweekly_sale
1
2
3

Complete list of extractable fields for Category Hierarchy objects from hmart.com. All fields typed and schema-versioned.

category_idnameparent_categorylevelurltotal_productstop_brandstrending_itemsscraped_at
category_hierarchy
● 200 OK
"category_id": "c_noodles",
"name": "Noodles & Ramen",
"parent_category": "Pantry",
"level": 2,
"total_products": 412,
"top_brands": "['Nongshim', 'Samyang', 'Paldo']",
"scraped_at": "2023-10-24T08:10:00Z"
# category_idnameparent_categorylevelurltotal_products
1
2
3

Complete list of extractable fields for Search Results objects from hmart.com. All fields typed and schema-versioned.

keywordpositionskutitlepricein_stockpromo_flagthumbnail_urlzip_codescraped_at
search_results
● 200 OK
"keyword": "kimchi",
"position": 1,
"sku": "846034000181",
"title": "Jongga Sliced Napa Cabbage Kimchi",
"price": 8.99,
"in_stock": true,
"promo_flag": false,
"zip_code": "10001"
# keywordpositionskutitlepricein_stock
1
2
3

Capabilities

Extract the complete Asian grocery catalogue

Our Hmart scraper navigates store-specific inventories, bilingual product data, and weekly promotions, delivering structured FMCG datasets without the maintenance overhead.

Full Product Catalogue Extraction

Extract SKUs, titles, brands, weights, categories, and images across all grocery, meat, seafood, and homeware departments.

Location-Based Pricing

Hmart pricing and availability change by zip code. We manage session states to scrape localised data for your target delivery zones.

Weekly Sales & Promotions

Track weekly circulars, discount percentages, and promotional badges to monitor competitor markdowns and pricing strategies.

Bilingual Text Parsing

Capture English and native language (Korean, Chinese, Japanese) product titles and descriptions, maintaining correct UTF-8 encoding.

Nutritional & Ingredient Data

Extract structured ingredient lists, allergen warnings, and macro-nutritional tables from product detail pages.

Stock Availability Tracking

Monitor out-of-stock flags and inventory depth indicators across different regional distribution centres.

Search Rank Tracking

Monitor organic search positions for high-volume keywords like 'ramen' or 'kimchi' to understand product visibility.

Brand Intelligence

Filter and aggregate data by specific FMCG brands to track their pricing and placement within the Hmart ecosystem.

Incremental Updates

Run daily diff pipelines that only deliver records for products with changed prices or stock statuses, reducing compute costs.

// engagement pipeline

From target zip codes to structured feeds

Brief in. Clean data out.

Define Scope
d 0

Provide target zip codes, categories, or specific brand lists. We design the extraction schema together.

Pipeline Build
d 2–4

We configure Scrapy crawlers, handle location-based cookies, and manage proxy rotation for hmart.com.

Validation & QA
d 4–6

Schema validation, UTF-8 encoding checks, and price-outlier detection before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

Overcoming grocery extraction challenges

Grocery platforms present unique scraping hurdles due to regional variation and rapid inventory turnover. Here is how we maintain data integrity.

pipeline-monitor · hmart.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Session Management
Handling zip-code dependent inventory

Hmart requires a user to set a delivery zip code to view accurate pricing and stock. Our crawlers inject and maintain specific location cookies throughout the session, allowing parallel extraction across multiple US regions simultaneously.

Data Normalisation
Bilingual string processing

Asian grocery catalogues feature mixed-character strings. We enforce strict UTF-8 encoding rules and separate English titles from Korean/Chinese text into distinct schema fields, preventing downstream database corruption.

Anti-bot layer
Bypassing edge protection

eCommerce sites employ edge protection to block automated traffic. We route requests through US-based residential proxies with legitimate browser fingerprints, distributing request volumes to blend with normal consumer traffic.

Schema stability
Resilient DOM parsing

Promotional events often change the layout of product cards. We implement multi-layered fallback selectors targeting embedded JSON data layers and CSS structures, ensuring your pipeline does not break during a weekly sale update.

Change detection
Efficient delta updates

Grocery pricing changes constantly. We hash product records and only emit rows where pricing, promo status, or stock availability has shifted since the last run, keeping your data warehouse lean.

Applications

Who uses Hmart data

Teams across industries use hmart.com data to build competitive products and smarter operations.

01
Competitor Price Intelligence

Rival Asian grocers and mainstream supermarkets track Hmart pricing to optimise their own markups on imported FMCG goods.

02
FMCG Brand Monitoring

Food manufacturers monitor their product placement, out-of-stock rates, and retail pricing across different Hmart regional zones.

03
Grocery Delivery Aggregators

Third-party delivery platforms sync Hmart's catalogue and stock status to keep their own apps accurate for end users.

04
Inflation & CPI Tracking

Economic analysts scrape basket costs over time to measure food inflation specific to imported and specialty Asian goods.

05
Supply Chain Forecasting

Logistics firms analyse regional out-of-stock patterns to predict demand surges for specific imported commodities.

06
Market Entry Analysis

New food brands analyse category depth and competitor pricing before pitching their products to buyers.

Why DataFlirt

"Hmart holds the definitive catalogue for Asian FMCG pricing in North America, but capturing store-level availability requires precise session management."

Grocery scraping is notoriously difficult due to zip-code-dependent pricing and rapid inventory turnover. DataFlirt handles the localized session state, proxy rotation, and schema maintenance so your engineering team receives normalised, warehouse-ready product feeds without the operational headache.

Technical Spec

Hmart scraper — technical capabilities

Everything supported by our hmart.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

Location-based pricing
Session injection to extract prices and stock by specific zip code
Supported
Weekly sales tracking
Capture promotional badges, sale prices, and discount percentages
Supported
Bilingual product attributes
Accurate extraction of mixed English and Asian language strings
Supported
Nutritional table parsing
Structured extraction of macros and ingredient lists
Supported
JavaScript rendering
Playwright execution for dynamically loaded inventory statuses
Supported
Residential proxy rotation
US-based ISP IPs to bypass edge protection
Supported
Change detection (diffs)
Emit records only when price or stock status changes
Supported
Webhook delivery
HTTP POST per batch for real-time inventory syncing
Supported
Smart Savings Card pricing
Member-exclusive discounts requiring authenticated login
Partial
User order history
Historical purchase data locked behind user accounts
Partial
Infrastructure

Infrastructure powering the grocery pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy orchestrates the crawl while Playwright handles JavaScript execution and local storage injection for zip-code management.

Targeted Proxy Pools

We route requests through US-based residential proxies, maintaining sticky sessions to keep location data consistent during extraction.

Automated Orchestration

Airflow schedules daily or hourly runs on Kubernetes clusters, executing diff computations before pushing to your warehouse.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Nested structures ideal for handling complex nutritional arrays
CSV
Flat files for immediate analyst use in Excel or Tableau
Parquet
Columnar format optimised for BigQuery and Snowflake
S3
Direct delivery to your AWS environment
BigQuery
Native streaming into GCP datasets
Webhook
HTTP POST delivery for immediate application updates
Postgres
Direct database inserts with conflict resolution
API
RESTful endpoints to query extracted datasets on demand
// faq

Common questions.

About hmart.com scraping, legality, and pipeline operations.

Ask us directly →
Can you scrape pricing for different store locations?

Yes. We manage session cookies to simulate a user located in a specific zip code. You can provide a list of target zip codes, and we will extract the corresponding regional pricing and inventory data for each.

How do you handle non-English characters in product titles?

Our pipelines enforce strict UTF-8 encoding. We separate English text from Korean, Chinese, or Japanese characters into distinct schema fields, ensuring the data remains clean and queryable in your database.

How frequently can you update the catalogue?

We support daily or weekly full-catalogue refreshes. For specific high-priority categories or top-selling SKUs, we can configure hourly pipelines to track rapid inventory shifts.

Is it legal to scrape grocery data from Hmart?

Scraping public pricing, nutritional facts, and availability data is generally permissible. We do not bypass authentication walls or extract personal user data. Clients should consult their legal counsel regarding their specific commercial use cases.

Do you extract nutritional and ingredient information?

Yes. We parse the nutritional tables and ingredient text blocks present on product detail pages, delivering them as structured arrays or nested JSON objects.

How do you handle out-of-stock items?

We capture the stock status flag exactly as it appears on the site. If an item goes out of stock in a specific region, that status is recorded and timestamped in the delivery payload.

Can I get a sample dataset?

Yes. We provide sample exports covering specific categories or zip codes during the scoping phase, allowing your team to validate the schema before committing to a production pipeline.

$ dataflirt scope --new-project --source=hmart.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Stop dealing with broken selectors and blocked IPs. Tell us your target categories and zip codes, and we will build the pipeline.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in food drink and kitchen

Services

Data Extraction for Every Industry

View All Services →