SYSTEM all green source croftmill.co.uk queue 1,842 pages p99 latency 215ms dataflirt.com · scraper/croftmill-co.uk
RUN · 14 active pipelines · croftmill.co.uk live

Croftmill fabric data,
structured for scale.

We extract fabric listings, material compositions, pricing, and stock availability from Croftmill. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your schedule.

Fabrics extracted
12.4K /run
Price updates
1.2K /24h
Stock signals
8.9K /day
Active pipelines
14
Uptime
99.94%
Data Dictionary

Every field we extract from croftmill.co.uk

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Fabric Listings objects from croftmill.co.uk. All fields typed and schema-versioned.

skutitlecategorysub_categorycompositionwidth_cmweight_gsmprice_per_metrein_stockimage_url
fabric_listings
● 200 OK
"sku": "CM-8472",
"title": "Navy Blue Cotton Poplin",
"category": "Dress Fabric",
"composition": "100% Cotton",
"width_cm": 112,
"weight_gsm": 130,
"price_per_metre": 6.5,
"in_stock": true
# skutitlecategorysub_categorycompositionwidth_cm
1
2
3

Complete list of extractable fields for Pricing & Stock objects from croftmill.co.uk. All fields typed and schema-versioned.

skupricecurrencydiscount_pctstock_statuslow_stock_warningminimum_cut_metressale_badgescraped_at
pricing_& stock
● 200 OK
"sku": "CM-8472",
"price": 6.5,
"currency": "GBP",
"discount_pct": 0,
"stock_status": "In Stock",
"minimum_cut_metres": 0.5,
"sale_badge": false
# skupricecurrencydiscount_pctstock_statuslow_stock_warning
1
2
3

Complete list of extractable fields for Product Specs objects from croftmill.co.uk. All fields typed and schema-versioned.

skucolourpatternfabric_typestretchwashing_instructionsironingsuitable_fordescription
product_specs
● 200 OK
"sku": "CM-8472",
"colour": "Navy Blue",
"pattern": "Solid",
"fabric_type": "Poplin",
"stretch": "None",
"washing_instructions": "30 Degree Machine Wash",
"suitable_for": "Shirts, Dresses, Quilting"
# skucolourpatternfabric_typestretchwashing_instructions
1
2
3

Complete list of extractable fields for Remnants & Clearance objects from croftmill.co.uk. All fields typed and schema-versioned.

remnant_idoriginal_skulength_metrespriceoriginal_pricedefect_notesclearance_statusstock_counturl
remnants_& clearance
● 200 OK
"remnant_id": "REM-1029",
"original_sku": "CM-8472",
"length_metres": 1.2,
"price": 5.0,
"original_price": 7.8,
"clearance_status": true,
"stock_count": 1
# remnant_idoriginal_skulength_metrespriceoriginal_pricedefect_notes
1
2
3

Complete list of extractable fields for Haberdashery objects from croftmill.co.uk. All fields typed and schema-versioned.

item_idtitlecategorybrandpricedimensionsmaterialstock_statusimage_url
haberdashery
● 200 OK
"item_id": "HAB-441",
"title": "Gutermann Sew All Thread 100m",
"category": "Threads",
"brand": "Gutermann",
"price": 2.1,
"material": "100% Polyester",
"stock_status": "In Stock"
# item_idtitlecategorybrandpricedimensions
1
2
3

Capabilities

Extract structured textile data without the parsing headaches

Croftmill product pages contain dense, unstructured text. We handle the normalisation of fabric weights, widths, and compositions into clean, queryable columns.

Composition Normalisation

We parse unstructured descriptions to extract exact percentage breakdowns of cotton, polyester, viscose, and elastane.

Metric Conversion

Widths and weights are often mixed between imperial and metric. We standardise all outputs to centimetres and GSM.

Price Per Metre Tracking

Capture base pricing, sale discounts, and minimum cut requirements for every fabric listing in the catalogue.

Colour & Pattern Extraction

Map fabrics to primary colour groups and pattern types based on metadata and description text analysis.

Remnant Monitoring

Track one-off remnant pieces, capturing specific lengths and clearance pricing before they sell out.

Stock Availability Signals

Monitor out-of-stock flags and low-stock warnings to forecast inventory depletion rates across categories.

Suitability Mapping

Extract recommended uses for each fabric, categorising them for dresses, upholstery, quilting, or outerwear.

Care Instructions

Parse washing temperatures, ironing limits, and tumble dry suitability into structured boolean or enum fields.

High-Res Image Scraping

Extract URLs for the highest resolution fabric texture images available, bypassing thumbnail compression.

// engagement pipeline

From fabric catalogue to data warehouse

Brief in. Clean data out.

Define Scope
d 0

Select target categories like dressmaking fabrics, remnants, or haberdashery. We define the extraction schema.

Pipeline Build
d 2–4

We configure crawlers to handle Croftmill's pagination, category structures, and unstructured text fields.

Validation & QA
d 4–6

Schema validation, null-rate checks, and unit conversion testing before full production launch.

Delivery
ongoing

Clean JSON, CSV, or Parquet delivered to your S3 bucket or data warehouse on your required schedule.

Under the hood

Overcoming textile data extraction challenges

Extracting data from niche retailers requires custom parsing logic to turn descriptive text into structured metrics.

pipeline-monitor · croftmill.co.uk · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Text Parsing
Extracting specs from paragraphs

Fabric compositions and weights are often buried in paragraph descriptions rather than neat tables. We use regex and NLP to extract these values into structured columns.

Unit Standardisation
Handling mixed metric and imperial units

Textile sites frequently mix inches and centimetres, or ounces and GSM. Our pipeline includes a normalisation layer to output consistent metric units.

Dynamic Stock
Monitoring fast-moving remnants

Remnants are unique items that sell out quickly. We configure higher-frequency crawls for clearance sections to capture data before URLs 404.

Image Extraction
Capturing accurate textures

We locate the source URLs for zoomable, high-resolution images, essential for AI training or visual competitor analysis.

Change Detection
Tracking price and stock diffs

We maintain a hash index of previous runs to only deliver records where pricing or availability has changed, reducing your processing load.

Applications

Who uses Croftmill data

Teams across industries use croftmill.co.uk data to build competitive products and smarter operations.

01
Competitor Price Monitoring

Other UK fabric retailers track Croftmill's price per metre across common compositions to adjust their own pricing strategies.

02
Inventory Forecasting

Analysts monitor stock depletion rates on specific fabric types to identify seasonal trends and demand spikes.

03
AI Material Classification

Machine learning teams use high-resolution texture images paired with composition data to train fabric recognition models.

04
Fashion Trend Analysis

Designers track which colours, prints, and materials are entering clearance versus which remain at full price.

05
Wholesale Sourcing Models

Procurement teams use retail pricing data to negotiate better rates with textile mills and wholesalers.

06
Market Research

Textile industry analysts aggregate data across retailers to map the availability of sustainable fabrics like organic cotton or Tencel.

Why DataFlirt

"Croftmill holds decades of curated textile data, but extracting clean composition and pricing metrics requires custom parsing logic."

Parsing fabric specifications from unstructured descriptions is complex. DataFlirt extracts, cleans, and standardises weight, width, and composition metrics so your procurement algorithms have structured inputs. We handle the web scraping infrastructure entirely, letting you focus on textile analysis.

Technical Spec

Croftmill scraper — technical details

Everything supported by our croftmill.co.uk scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

Category pagination
Traverses all pages within fabric and haberdashery categories
Supported
Composition parsing
Extracts percentage breakdowns from unstructured text
Supported
Unit normalisation
Converts inches to cm and oz to GSM automatically
Supported
High-res images
Extracts base image URLs without compression parameters
Supported
Remnant tracking
Captures unique lengths and prices for off-cuts
Supported
Change detection
Only emits records with updated price or stock fields
Supported
Wholesale pricing
Requires authenticated B2B trade account access
Partial
User purchase history
Private data gated behind customer login walls
Partial
Infrastructure

Infrastructure powering the extraction

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles efficient category traversal while Playwright executes JavaScript required for dynamic stock and image loading.

Residential Proxy Infrastructure

We route requests through UK-based residential IPs to prevent rate-limiting and ensure uninterrupted catalogue extraction.

Cloud-Native Orchestration

Pipelines are orchestrated via Apache Airflow on Kubernetes, ensuring reliable daily or weekly deliveries with automated retries.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Nested structures ideal for complex composition data
CSV
Flat files for immediate spreadsheet analysis
XLS
Excel format for non-technical procurement teams
Parquet
Columnar format for efficient querying
AWS S3
Direct delivery to your cloud storage bucket
Webhook
HTTP POST for real-time stock alerts
API
REST endpoints to query your extracted datasets
BigQuery
Direct ingestion into Google Cloud data warehouses
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About croftmill.co.uk scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Croftmill legal?

Scraping publicly available product, pricing, and stock information is generally permissible. DataFlirt only extracts public data and does not bypass authentication to access wholesale pricing or user accounts. Clients should review terms of service and consult legal counsel.

How do you handle unstructured fabric descriptions?

We build custom parsing rules using regex and natural language processing to extract specific data points like composition percentages, widths, and care instructions from paragraph text.

Can you track remnants and clearance items?

Yes. We can target the remnants category specifically, extracting the unique length and discounted price for each one-off piece.

How frequently can the data be updated?

For a catalogue of Croftmill's size, we typically recommend daily or weekly runs. Higher frequency runs can be configured for fast-moving categories like remnants.

Do you standardise the measurement units?

Yes. We convert all imperial measurements to metric (centimetres and metres) and standardise weight formats to GSM to ensure your database remains clean.

Can I get a sample dataset?

Yes. We provide a sample extraction of specific fabric categories so you can evaluate the parsing quality and schema structure before committing to a pipeline.

$ dataflirt scope --new-project --source=croftmill.co.uk ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Stop manually copying fabric specs. We build and maintain the pipeline to deliver clean, structured textile data directly to your systems.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in textile and fabric

Services

Data Extraction for Every Industry

View All Services →