SYSTEM all green source khadims.com queue 4,192 pages p99 latency 214ms dataflirt.com · scraper/khadims-com
RUN · 14 active pipelines · khadims.com live

Khadims catalogue data,
at warehouse scale.

We extract footwear listings, pricing signals, size availability matrices, and physical store inventory from khadims.com. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Products extracted
12,481 /run
Price updates
4,105 /24h
Size availability checks
89,240 /day
Store locations
842
Uptime
99.94%
Data Dictionary

Every field we extract from khadims.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Product Listings objects from khadims.com. All fields typed and schema-versioned.

skutitlebrandcategorysub_categorypricemrpdiscount_pctcoloursizes_availablematerialsole_materialdescriptionimage_urlsurl
product_listings
● 200 OK
"sku": "KHD-M-54321",
"title": "Khadim Men Brown Formal Slip-On Shoes",
"brand": "Khadim",
"category": "Men",
"price": 1499.0,
"mrp": 2199.0,
"discount_pct": 31,
"colour": "Brown",
"sizes_available": "['6', '7', '8', '9', '10']"
# skutitlebrandcategorysub_categoryprice
1
2
3

Complete list of extractable fields for Pricing & Inventory objects from khadims.com. All fields typed and schema-versioned.

skupricemrpdiscount_pctdiscount_absin_stockstock_status_textavailable_sizesout_of_stock_sizespromotion_badgeprice_timestamp
pricing_& inventory
● 200 OK
"sku": "KHD-M-54321",
"price": 1499.0,
"mrp": 2199.0,
"discount_pct": 31,
"in_stock": true,
"available_sizes": "['6', '7', '8', '9', '10']",
"out_of_stock_sizes": "['11', '12']",
"promotion_badge": "End of Season Sale",
"price_timestamp": "2023-10-27T08:15:00Z"
# skupricemrpdiscount_pctdiscount_absin_stock
1
2
3

Complete list of extractable fields for Product Specifications objects from khadims.com. All fields typed and schema-versioned.

skuouter_materialinner_materialsole_materialclosureshoe_typeheel_heightoccasioncare_instructionsorigin_country
product_specifications
● 200 OK
"sku": "KHD-M-54321",
"outer_material": "Synthetic Leather",
"inner_material": "Textile",
"sole_material": "PU",
"closure": "Slip-on",
"shoe_type": "Formal",
"heel_height": "Low",
"occasion": "Office/Formal",
"care_instructions": "Wipe with a clean, dry cloth"
# skuouter_materialinner_materialsole_materialclosureshoe_type
1
2
3

Complete list of extractable fields for Store Locations objects from khadims.com. All fields typed and schema-versioned.

store_idstore_nameaddresscitystatepincodephonelatitudelongitudeoperating_hoursstore_type
store_locations
● 200 OK
"store_id": "STR-KA-042",
"store_name": "Khadim's - Indiranagar",
"city": "Bengaluru",
"state": "Karnataka",
"pincode": "560038",
"phone": "+91-80-12345678",
"latitude": 12.9783,
"longitude": 77.6408,
"operating_hours": "10:30 AM - 09:30 PM"
# store_idstore_nameaddresscitystatepincode
1
2
3

Complete list of extractable fields for Categories objects from khadims.com. All fields typed and schema-versioned.

category_idnameparent_categoryurlproduct_countbanner_image_urlmeta_titlemeta_descriptionscraped_at
categories
● 200 OK
"category_id": "CAT-MEN-FORMAL",
"name": "Men's Formal Shoes",
"parent_category": "Men",
"url": "https://www.khadims.com/category/men-formal",
"product_count": 342,
"meta_title": "Buy Men's Formal Shoes Online | Khadims",
"scraped_at": "2023-10-27T08:15:33Z"
# category_idnameparent_categoryurlproduct_countbanner_image_url
1
2
3

Capabilities

Extract the complete Khadims retail footprint

Our Khadims scraper targets the specific complexities of footwear retail: size availability matrices, colour variant grouping, specification tables, and physical store locators.

Full Catalogue Extraction

Capture every SKU across Men, Women, Kids, and Accessories categories, including title, description, and high-resolution image URLs.

Size Matrix Mapping

Extract size availability per SKU in real time. Differentiate between in-stock, low-stock, and out-of-stock sizes for accurate inventory tracking.

Price & Discount Tracking

Monitor Selling Price against MRP. Capture discount percentages, promotional badges, and flash sale pricing.

Colour Variant Grouping

Map parent-child relationships for shoes available in multiple colours, ensuring accurate cross-referencing of styles.

Material Specifications

Parse unstructured specification tables into clean JSON fields: sole material, upper material, closure type, and occasion.

Store Locator Scraping

Extract details of all physical retail outlets nationwide, including address, coordinates, phone numbers, and operating hours.

Daily Delta Delivery

Configure pipelines to run daily, outputting only the SKUs where price or size availability has changed since the last run.

Image Asset Extraction

Collect primary and secondary product image URLs, normalising them for ingestion into your own PIM or analytics dashboard.

Category Hierarchy Mapping

Traverse the entire site navigation tree to maintain accurate taxonomy data, from root categories down to specific sub-segments.

// engagement pipeline

From target list to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Specify categories, specific SKUs, or the entire catalogue. We design the extraction schema together.

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers, handle pagination, and manage proxy rotation for khadims.com.

Validation & QA
d 4–6

Schema validation, null-rate checks, and size-matrix testing before full pipeline deployment.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

Handling the complexities of footwear scraping

Footwear retail sites present unique scraping challenges, particularly around dynamic inventory and variant mapping. Here is how we ensure data accuracy.

pipeline-monitor · khadims.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Dynamic rendering
JavaScript execution for size selectors

Size availability and price updates on khadims.com are often hydrated via JavaScript after page load. We use Playwright to execute JS and interact with size selectors, ensuring we capture the true stock state rather than stale HTML.

Variant mapping
Parent-child grouping

A single shoe style may exist in multiple colours, sometimes as separate URLs and sometimes as dynamic toggles. Our extraction logic maps these relationships accurately, grouping variants under a single parent identifier.

Rate limiting
Residential proxy rotation

To extract the entire catalogue without triggering IP blocks or getting cached responses, we route requests through Indian residential proxies, maintaining low concurrency and human-like request patterns.

Schema normalisation
Cleaning specification tables

Product specifications (material, closure, care instructions) often vary in format across different categories. Our pipeline normalises these tables into consistent, strongly-typed JSON fields.

Store API extraction
Reverse-engineering locator endpoints

Instead of scraping complex mapping interfaces, we intercept the underlying API calls powering the Khadims store locator to extract precise geographic coordinates and store metadata directly.

Applications

Who uses Khadims data — and how

Teams across industries use khadims.com data to build competitive products and smarter operations.

01
Competitor Pricing Analysis

Footwear brands track Khadims' MRP and discount strategies across categories to optimise their own promotional calendars.

02
Assortment Planning

Retail buyers analyse category depth, colour availability, and material trends to inform seasonal purchasing decisions.

03
Inventory Gap Analysis

Analysts monitor size-level stockouts to identify supply chain constraints and popular size distributions in the Indian market.

04
Retail Footprint Mapping

Real estate and expansion teams extract store locator data to map geographic presence and identify underserved retail catchments.

05
Marketplace Benchmarking

Aggregators compare Khadims' direct-to-consumer pricing against their listings on third-party platforms like Myntra and Amazon.

06
Material & Trend Research

Product teams track the distribution of materials (PU, leather, synthetic) and styles across the catalogue to identify market shifts.

Why DataFlirt

"Footwear retail moves on size availability and discount cycles. You cannot optimise pricing without knowing precisely which SKUs are out of stock on your competitor's site."

Scraping footwear catalogues requires handling complex size-colour matrices and dynamic inventory states. DataFlirt manages the JavaScript execution and proxy rotation required to track thousands of SKUs across khadims.com, delivering clean, normalised datasets so your analytics team can focus on strategy rather than pipeline maintenance.

Technical Spec

Khadims scraper — technical capabilities

Everything supported by our khadims.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Playwright sessions for dynamic size selection and stock checks
Supported
Size matrix mapping
Extraction of all available and out-of-stock sizes per SKU
Supported
Store locator API
Direct extraction of geographic coordinates and store metadata
Supported
High-res image URLs
Capture of primary and gallery image assets
Supported
Daily delta diffs
Only emit records with changed price or stock since last run
Supported
Webhook delivery
HTTP POST per record for real-time inventory updates
Supported
Category traversal
Automated pagination through all sub-categories
Supported
User cart data
Session-specific cart contents and checkout flows
Partial
Loyalty points data
Authenticated access to Khadim Rewards balances
Partial
Infrastructure

Infrastructure powering the Khadims pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration and deduplication. Playwright handles JavaScript rendering for dynamic size and colour selectors.

Residential Proxy Infrastructure

We maintain pools of Indian residential ISP proxies to avoid geographic blocking and rate limits during full catalogue extracts.

Cloud-Native Orchestration

Pipelines run on AWS Lambda and ECS. Airflow handles daily scheduling and diff computation. All state stored in managed Postgres.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested arrays for variants
CSV
Flat file with typed columns for analytics teams
XLS
Excel format for manual review and buying teams
Parquet
Columnar format for BigQuery, Snowflake, Athena
AWS S3
Direct bucket delivery on defined schedules
Webhook
HTTP POST per record for downstream processing
API
REST endpoint to query your latest scraped dataset
PostgreSQL
Direct upsert into your relational database
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About khadims.com scraping, legality, and pipeline operations.

Ask us directly →
Can you extract size availability for every shoe?

Yes. Our pipeline iterates through the available size options on the product page, recording exactly which sizes are in stock and which are sold out at the time of the scrape.

How often can the data be updated?

For the entire catalogue, we typically recommend daily runs. For specific high-priority categories or SKUs, we can configure hourly pipelines to track fast-moving inventory and flash sales.

Do you scrape all physical store locations?

Yes. We extract the complete list of Khadims retail outlets via their store locator, including addresses, PIN codes, operating hours, and latitude/longitude coordinates.

How do you handle shoes with multiple colours?

We map parent-child relationships. If a shoe has three colour variants, we extract each as a distinct record but link them with a shared parent ID to maintain catalogue integrity.

Is it legal to scrape khadims.com?

Scraping publicly available product, pricing, and store data is generally permissible. DataFlirt extracts only public, non-authenticated information. We do not access user accounts, cart data, or loyalty programs.

Can I get historical pricing data?

We begin accumulating historical data from the day your pipeline is commissioned. Every run is timestamped, allowing you to build a time-series database of price changes over time.

What formats do you deliver in?

We deliver in JSON, CSV, XLS, and Parquet. We can push directly to AWS S3, Snowflake, BigQuery, or trigger Webhooks for real-time integration.

$ dataflirt scope --new-project --source=khadims.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off store location export or continuous price and size monitoring across the entire catalogue — we scope, build, and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in shoes and footwear

Services

Data Extraction for Every Industry

View All Services →