SYSTEM all green source woolworths.com.au queue 18,294 pages p99 latency 147ms dataflirt.com · scraper/woolworths-com.au
RUN · 94 active pipelines · woolworths.com.au live

Woolworths data,
at warehouse scale.

We extract grocery product listings, weekly specials, Everyday Rewards pricing, unit price intelligence, nutritional data, and availability signals from Woolworths. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Products extracted
620K /day
Price updates
2.8M /24h
Promo records
180K /run
Active pipelines
94
Uptime
99.95%
Data Dictionary

Every field we extract from woolworths.com.au

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Product Listings objects from woolworths.com.au. All fields typed and schema-versioned.

product_idbarcodetitlebrandcategorysub_categoryaislepricewas_pricecurrencyunit_priceunit_of_measurepack_sizepack_quantityis_on_specialspecial_typeeveryday_rewards_pricein_stockcountry_of_originis_own_brandimage_urlsproduct_url
product_listings
● 200 OK
"product_id": "WOW_134827",
"title": "Mainland Tasty Cheese Block 500g",
"brand": "Mainland",
"price": 7.50,
"was_price": 9.00,
"currency": "AUD",
"unit_price": 1.50,
"unit_of_measure": "per 100g",
"is_on_special": true,
"special_type": "HALF_PRICE",
"country_of_origin": "New Zealand"
# product_idbarcodetitlebrandcategorysub_category
1
2
3

Complete list of extractable fields for Promotions & Specials objects from woolworths.com.au. All fields typed and schema-versioned.

product_idspecial_typepromo_pricewas_pricediscount_pctdiscount_absspecial_start_datespecial_end_dateeveryday_rewards_pricemember_discountmulti_buy_qtymulti_buy_pricecatalogue_pageprice_timestamp
promotions_& specials
● 200 OK
"product_id": "WOW_134827",
"special_type": "HALF_PRICE",
"promo_price": 7.50,
"was_price": 9.00,
"discount_pct": 50,
"special_start_date": "2026-05-07",
"special_end_date": "2026-05-13",
"everyday_rewards_price": 7.00
# product_idspecial_typepromo_pricewas_pricediscount_pctdiscount_abs
1
2
3

Complete list of extractable fields for Nutritional Data objects from woolworths.com.au. All fields typed and schema-versioned.

product_idserving_sizeservings_per_packenergy_kjenergy_kcalprotein_gfat_total_gfat_saturated_gcarbohydrate_gsugars_gdietary_fibre_gsodium_mgingredients_textallergenshealth_star_ratingcertificationsdietary_claims
nutritional_data
● 200 OK
"product_id": "WOW_134827",
"serving_size": "25g",
"health_star_rating": 4.5,
"energy_kj": 408,
"protein_g": 6.3,
"fat_total_g": 8.5,
"sodium_mg": 162,
"allergens": "milk",
"dietary_claims": ["Gluten Free"]
# product_idserving_sizeservings_per_packenergy_kjenergy_kcalprotein_g
1
2
3

Complete list of extractable fields for Search Results objects from woolworths.com.au. All fields typed and schema-versioned.

querypositionproduct_idtitlebrandpriceunit_priceis_on_specialspecial_typeis_own_brandin_stockthumbnail_urlscraped_at
search_results
● 200 OK
"query": "cheese block",
"position": 1,
"product_id": "WOW_134827",
"is_on_special": true,
"special_type": "HALF_PRICE",
"is_own_brand": false,
"unit_price": 1.50,
"scraped_at": "2026-05-12T07:00:00Z"
# querypositionproduct_idtitlebrandprice
1
2
3

Capabilities

Everything you need from Woolworths — nothing you don't

Our Woolworths scraper is purpose-built for the Australian grocery market: weekly specials cycles, unit price intelligence, Everyday Rewards member pricing, full nutritional panels, and own-brand vs national brand signals.

Full Grocery Product Extraction

Title, brand, barcode, pack size, country of origin, and every metadata field Woolworths surfaces — including aisle-level category taxonomy.

Weekly Specials & Promotion Tracking

Capture half-price, multi-buy, and Everyday Rewards specials with start and end dates — building a full promotions calendar for every product.

Unit Price Intelligence

Every product's price per 100g, per 100ml, or per unit — the true comparison metric for grocery benchmarking, basket analysis, and inflation tracking.

Full Nutritional Panel Extraction

Serving size, energy, protein, fat, carbohydrates, sugars, sodium, fibre, health star rating, allergens, and dietary claims — per product.

Everyday Rewards Pricing

Capture member-exclusive Everyday Rewards prices alongside regular shelf prices — quantifying the loyalty discount layer for each promotion.

Own-Brand vs National Brand Flags

Flag Woolworths Macro, Woolworths Gold, and other private label products — a critical dimension for category mix and margin analysis.

Country of Origin Data

Country of origin for food products — particularly valuable for supply chain analysts, food provenance researchers, and import tracking.

Stock Availability Monitoring

Track in-stock and out-of-stock status across the online catalogue — useful for supply chain signal monitoring and shelf gap analysis.

Scheduled + Streaming Modes

Weekly specials sync on Wednesdays, daily price monitoring, or real-time stock availability pipelines — all with change-detection diffing.

// engagement pipeline

From product barcode to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Provide product IDs, barcodes, category URLs, or keyword sets. We design the extraction schema and promotions cadence together.

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers, Australian residential proxies, session management, and Woolworths-specific rate pacing.

Validation & QA
d 4–6

Unit price completeness checks, nutritional panel null-rate audits, promotion date validation, and sample records before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

How our Woolworths pipeline handles the hard parts

Grocery data has its own rhythms — weekly specials cycles, Wednesday resets, and nutritional tables that vary in structure across categories. Here's how we handle them.

pipeline-monitor · woolworths.com.au · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Weekly specials cadence
Wednesday reset capture and promotion lifecycle tracking

Woolworths resets its weekly specials on Wednesdays. Our pipeline schedules a full promotion sweep within hours of the Wednesday reset, capturing the new special type, promo price, was price, and end date for every affected product — building a complete promotions history from day one.

Unit price parsing
Normalised per-unit pricing across pack formats

Woolworths expresses unit prices in different formats across categories — per 100g, per 100ml, per litre, per unit, per sheet. Our pipeline normalises unit prices into a consistent format per category, enabling true price-per-unit comparisons across pack sizes and brands.

Nutritional table extraction
Structured nutritional panels across 40K+ SKUs

Nutritional panel formats vary across product categories and supplier submissions. Our parser handles tabular, prose, and mixed-format nutritional presentations — extracting clean, queryable fields from each with consistent unit normalisation.

Australian residential proxies
AU IP pool for geo-consistent availability data

Woolworths serves availability and pricing data based on location. Our pipeline uses Australian residential ISP proxies to ensure product availability, Everyday Rewards pricing, and delivery option data matches what Australian consumers see.

Monitoring & alerting
24/7 pipeline health with anomaly detection

Every run emits structured logs to our observability stack. We alert on null-rate spikes, price outliers, nutrition completeness drops, and coverage gaps — and respond before you notice. SLA uptime is contractual, not aspirational.

Applications

Who uses Woolworths data — and how

Teams across industries use woolworths.com.au data to build competitive products and smarter operations.

01
Grocery Price Intelligence & FMCG Strategy

FMCG brands and category managers track Woolworths shelf prices, unit prices, and promotional intensity to benchmark against Coles and IGA — and to calibrate promotional investment.

02
Inflation & Food Price Research

Economists, government bodies, and consumer advocacy groups use Woolworths pricing data as a high-frequency signal for food inflation tracking across categories.

03
Nutritional & Health Research

Public health researchers, dietitians, and food policy teams use Woolworths nutritional data at category scale — health star ratings, sodium, sugar, and fibre across the entire supermarket range.

04
Private Label & Brand Share Analysis

FMCG teams and retail analysts track the growing shelf presence and pricing of Woolworths own-brand products versus national brands across categories.

05
Supply Chain & Country of Origin Tracking

Procurement teams and food provenance researchers use country-of-origin data to map Australian vs imported product ratios across supermarket categories.

06
Promotional Effectiveness Modelling

Trade marketing teams model the relationship between Woolworths promotion type (half-price, multi-buy, Rewards), promotion frequency, and baseline price recovery across brands.

Why DataFlirt

"Woolworths and Coles together account for over 65% of Australian grocery retail — and their pricing, promotion, and nutritional data is among the most consequential FMCG dataset in the country."

Grocery scraping is operationally distinct from general e-commerce: weekly promotion cycles, unit price normalisation across pack formats, nutritional panel extraction, and Australian residential proxy requirements all demand domain-specific pipeline design. DataFlirt absorbs that complexity so your category managers and researchers can focus on the insights — not the infrastructure.

Technical Spec

Woolworths scraper — technical capabilities

Everything supported by our woolworths.com.au scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Full Playwright sessions — required for dynamic pricing, availability, and promo widgets
Supported
CAPTCHA bypass
Automated 2Captcha + CapSolver integration with fallback to manual queue
Supported
Australian residential proxies
ISP-grade AU residential IPs — for geo-consistent availability and Rewards pricing
Supported
Weekly specials cadence
Wednesday reset capture + full promotion lifecycle from start to end date
Supported
Unit price normalisation
Per-100g, per-100ml, per-litre, and per-unit formats normalised consistently
Supported
Nutritional panel extraction
Structured extraction of energy, macros, sodium, health star rating, and allergens
Supported
Everyday Rewards pricing
Member-exclusive pricing captured alongside shelf price per product
Supported
Own-brand detection
Woolworths Macro, Gold, and Essentials private label flagged at product level
Supported
Country of origin capture
Country of origin extracted from product details and label text
Supported
Change detection (diffs)
Hash-based diff: only emit records with changed fields since last run
Supported
In-store stock by location
Store-level stock availability requires store-specific session management on request
Supported
Everyday Rewards account data
Purchase history and personalised offers require authenticated Everyday Rewards login
Partial
Infrastructure

Infrastructure powering the Woolworths pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverAU Residential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles Woolworths' JavaScript-rendered product pages, promotional widgets, and nutritional panel tabs.

Australian Residential Proxy Infrastructure

We maintain pools of Australian ISP residential proxies to ensure geo-consistent availability, Everyday Rewards pricing, and delivery option data. Rotation happens per-request with sticky sessions where required.

Cloud-Native Orchestration

Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling aligned with Woolworths' Wednesday reset cycle, dependency management, and SLA alerting.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested — schema versioned per run
CSV
Flat file with typed columns — Excel/Sheets compatible
Parquet
Columnar format for BigQuery, Snowflake, Athena
S3
Direct bucket delivery — compatible with any data lake
BigQuery
Streamed directly into your dataset with schema auto-detect
Webhook
HTTP POST per record for real-time downstream processing
Postgres
Upsert into your existing schema with conflict resolution
Snowflake
Stage + COPY INTO workflow — incremental or full-replace
// faq

Common questions.

About woolworths.com.au scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Woolworths legal?

Scraping publicly available information from Woolworths is generally permissible under Australian law — consistent with international precedents such as hiQ v. LinkedIn. DataFlirt targets only public, non-authenticated product, pricing, nutritional, and promotional data. We do not extract personal data, circumvent authentication walls, or violate the Australian Privacy Act. We recommend clients review Woolworths' ToS independently and consult legal counsel for specific use cases.

When do Woolworths weekly specials reset, and how do you handle that?

Woolworths resets its weekly specials on Wednesdays. Our pipeline schedules a full promotional sweep within hours of Wednesday midnight, capturing the incoming special type, promo price, was price, and end date. Outgoing specials are flagged as expired with their last-seen data preserved — giving you a full promotion lifecycle history.

Can you capture Everyday Rewards member pricing?

Yes. Everyday Rewards member prices are surfaced publicly on Woolworths product pages alongside regular shelf prices. We capture both, allowing you to quantify the loyalty discount layer per product and analyse the difference between member and non-member pricing over time.

How do you normalise unit prices across different pack formats?

Woolworths expresses unit prices in formats that vary by category — per 100g, per 100ml, per litre, per unit, per sheet. Our pipeline normalises these into a consistent format per product category, so cross-pack-size and cross-brand comparisons are directly queryable without additional transformation.

Can you extract full nutritional panels?

Yes — including energy (kJ and kcal), protein, total fat, saturated fat, carbohydrates, sugars, dietary fibre, sodium, health star rating, allergens, and dietary claims. Nutritional panel coverage depends on what Woolworths surfaces for each product, and we flag null rates per field in every delivery.

What's the minimum viable engagement?

Our smallest packages start at a defined product or category set (typically 2,000–20,000 SKUs) with weekly delivery. For full-catalogue monitoring, nutritional research programmes, or custom schema requirements, we price based on volume and cadence.

Can you compare Woolworths and Coles pricing in the same pipeline?

Yes. We operate Woolworths and Coles pipelines with a shared normalised schema — allowing direct price, unit price, and promotion comparisons between the two major chains from a single data delivery. Contact us to scope a dual-retailer programme.

Can I request a sample dataset before committing?

Absolutely. We provide a sample run of up to 500 products including pricing, promotions, and nutritional data as part of the pre-engagement scoping process — so you can validate schema fit and field completeness before signing any contract.

$ dataflirt scope --new-project --source=woolworths.com.au ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need weekly specials tracking, a full nutritional database, or a real-time grocery price monitor across 40K SKUs — we scope, build, and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Services

Data Extraction for Every Industry

View All Services →