We extract grocery listings, ZIP-code specific pricing, nutritional facts, allergens, and stock levels from rewe.de. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Product Listings objects from rewe.de. All fields typed and schema-versioned.
"product_id": "8273645", "title": "ja! Haltbare Vollmilch 3,5% Fett 1l", "brand": "ja!", "price": 1.09, "base_price": 1.09, "base_unit": "1 l", "nutri_score": "B", "category": "Molkereiprodukte"
| # | product_id | title | brand | category | sub_category | price |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Nutritional & Ingredients objects from rewe.de. All fields typed and schema-versioned.
"product_id": "8273645", "energy_kj": 267, "energy_kcal": 64, "fat_g": 3.5, "saturated_fat_g": 2.3, "protein_g": 3.3, "salt_g": 0.11, "allergens_list": "['Milch']"
| # | product_id | ingredients_text | allergens_list | energy_kj | energy_kcal | fat_g |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Regional Pricing & Stock objects from rewe.de. All fields typed and schema-versioned.
"product_id": "8273645", "store_id": "19384", "zip_code": "10115", "price": 1.09, "in_stock": true, "promotion_active": false, "max_order_qty": 24
| # | product_id | store_id | zip_code | price | discount_price | in_stock |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Categories & Taxonomy objects from rewe.de. All fields typed and schema-versioned.
"category_id": "c-1234", "name": "Käse", "parent_category": "Molkereiprodukte", "level": 2, "product_count": 412, "is_active": true, "url": "/c/molkereiprodukte-kaese/"
| # | category_id | name | parent_category | level | url | product_count |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Store Locations objects from rewe.de. All fields typed and schema-versioned.
"store_id": "19384", "name": "REWE City", "type": "City", "zip_code": "10115", "city": "Berlin", "has_delivery": true, "has_pickup": true
| # | store_id | name | type | address | zip_code | city |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
REWE's frontend heavily relies on regional context and dynamic single-page application loading. We handle the zip-code specific session cookies, pagination, and data hydration to extract clean FMCG records.
Extract both the final retail price and the standard base price (Grundpreis) per kg/litre to enable accurate cross-brand unit cost comparisons.
REWE prices vary by region. We inject specific PLZ (Postleitzahl) cookies and session tokens to extract exact local pricing and stock availability.
Parse detailed macro-nutritional tables (kJ, kcal, fat, sugar, protein), additive lists, and Nutri-Score metadata directly from product detail pages.
Extract structured ingredient lists and flag mandatory European allergen declarations (e.g., gluten, lactose, nuts) for dietary analysis.
Track temporary price drops, multi-buy discounts, and promotional validity windows across the entire catalogue.
Extract store coordinates, opening hours, facility types (REWE Center vs City), and fulfillment options (Abholservice vs Lieferservice).
Monitor real-time out-of-stock indicators and maximum order quantities by specific delivery region or pickup store.
Map the entire category tree to understand product placement, sub-category depth, and shelf-share metrics.
Run daily diffs against the catalogue to isolate price changes, new product listings, and delisted items without processing the full dataset.
Brief in. Clean data out.
Provide target categories, zip codes, or specific FMCG brand names. We design the extraction schema together.
We configure Scrapy / Playwright crawlers, proxy rotation, PLZ session management, and rate-limit handling for rewe.de.
Schema validation, null-rate checks, price-outlier detection, and nutritional data formatting before full launch.
JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.
Extracting from modern grocery platforms requires managing complex state and regional contexts. Here is how we maintain data integrity.
REWE requires a valid market context (Postleitzahl) to display accurate pricing and stock. Our crawlers manage isolated session containers per target region, injecting the correct location cookies before executing search or category traversal.
The REWE frontend relies on Next.js and React. Instead of brittle DOM scraping, we intercept the underlying JSON hydration state and GraphQL API responses, ensuring structural stability even when the UI layout changes.
To avoid geo-blocking and rate limits, we route requests through German ISP residential proxies. This ensures the traffic profile matches domestic consumer behaviour, maintaining high success rates.
Grocery data is notoriously messy with mixed units (grams, kilograms, pieces, litres). We parse and normalise base quantities and pricing into standard decimal formats for immediate database ingestion.
We monitor output for extreme price variance or sudden drops in catalogue size. If a category layout changes and null rates spike, our observability stack triggers an alert for immediate selector repair.
Consumer goods brands monitor their retail pricing, promotional frequency, and base price positioning against direct competitors.
Economic analysts track a fixed basket of grocery items across multiple zip codes to measure real-time food inflation metrics.
Health tech applications ingest Nutri-Score, macro-nutrients, and allergen data to power dietary recommendation engines.
Suppliers monitor regional out-of-stock indicators to optimise their distribution and detect fulfillment bottlenecks.
Delivery startups map REWE's pricing and assortment density to optimise their own dark-store inventory and pricing strategy.
Analysts calculate shelf-share by measuring the total SKU count of a specific brand versus the total category size.
"Grocery data is highly fragmented by region. Without precise session management, you are extracting generic placeholder prices rather than reality."
Extracting data from REWE requires more than simple HTTP requests. You must maintain regional state, handle complex unit conversions, and intercept dynamic application data. DataFlirt manages the session orchestration and proxy routing so you receive clean, normalised FMCG records ready for analysis.
Everything supported by our rewe.de scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, cookie sessions, and interaction flows. Combined via scrapy-playwright middleware.
We maintain pools of residential ISP proxies across DE regions. Rotation happens per-request with sticky sessions where required to maintain PLZ continuity.
Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.
Data delivered to where your team already works — no new tooling required.
About rewe.de scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available pricing, product, and nutritional information is generally permissible under applicable law. DataFlirt targets only public, non-authenticated data. We do not extract personal data or circumvent authentication walls. Clients should consult legal counsel for their specific use case.
We initiate separate crawling sessions for each target postal code (PLZ). The crawler injects the required location cookies before loading category or product pages, ensuring the extracted price and stock status accurately reflect that specific region.
Yes. We extract the full nutritional table, ingredient lists, highlighted allergens, and the official Nutri-Score rating directly from the product detail pages.
For targeted categories or specific competitor brand tracking, we can configure pipelines to run daily or intra-day. Full catalogue refreshes typically complete within a 12-hour window.
Yes. We extract both the retail price and the calculated base price (e.g., €/kg or €/L) to allow for direct comparison across different package sizes and brands.
Our smallest packages start at tracking specific categories or a defined list of FMCG brands across a set number of zip codes. Contact us with your specific volume requirements for a precise quote.
Yes. We provide a sample run of specific categories or search terms as part of the pre-engagement scoping process, allowing you to validate schema fit and data quality.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a daily monitor of FMCG pricing across key German cities or a one-off extraction of nutritional data — we scope, build, and operate the pipeline. Tell us what you need.
