SYSTEM all green source duedil.com queue 12,408 companies p99 latency 218ms dataflirt.com · scraper/duedil-com
RUN * 114 active pipelines * duedil.com live

Duedil data,
at warehouse scale.

We extract company profiles, financial statements, director networks, and corporate structures from Duedil. Delivered as clean JSON, CSV, or Parquet to S3 or PostgreSQL on your cadence.

Companies extracted
4.1M /run
Director profiles
8.7M /run
Financial records
18.2M /month
Active pipelines
114
Uptime
99.94%
Data Dictionary

Every field we extract from duedil.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Company Profiles objects from duedil.com. All fields typed and schema-versioned.

company_namecompany_numberstatusincorporation_datecompany_typeregistered_addresssic_codeswebsitephoneemployee_count
company_profiles
● 200 OK
"company_name": "ACME HOLDINGS LTD",
"company_number": "01234567",
"status": "Active",
"incorporation_date": "2014-06-12",
"company_type": "Private limited with Share Capital",
"registered_address": "123 Business Road, London, EC1A 1BB",
"sic_codes": "['62012', '62020']",
"employee_count": 245
# company_namecompany_numberstatusincorporation_datecompany_typeregistered_address
1
2
3

Complete list of extractable fields for Financial Statements objects from duedil.com. All fields typed and schema-versioned.

company_numberyear_endingturnovergross_profitoperating_profitnet_assetstotal_liabilitiescash_at_bankauditor_namefiling_date
financial_statements
● 200 OK
"company_number": "01234567",
"year_ending": "2025-12-31",
"turnover": 45200000.0,
"gross_profit": 18500000.0,
"operating_profit": 4200000.0,
"net_assets": 12400000.0,
"cash_at_bank": 3100000.0,
"filing_date": "2026-09-14"
# company_numberyear_endingturnovergross_profitoperating_profitnet_assets
1
2
3

Complete list of extractable fields for Director Networks objects from duedil.com. All fields typed and schema-versioned.

director_iddirector_nameroleappointed_dateresigned_datedate_of_birthnationalityactive_appointmentstotal_appointmentsoccupation
director_networks
● 200 OK
"director_id": "DIR-987654321",
"director_name": "Jane Doe",
"role": "Director",
"appointed_date": "2018-03-01",
"date_of_birth": "1975-08",
"nationality": "British",
"active_appointments": 3,
"total_appointments": 7
# director_iddirector_nameroleappointed_dateresigned_datedate_of_birth
1
2
3

Complete list of extractable fields for Corporate Structure objects from duedil.com. All fields typed and schema-versioned.

company_numberparent_companyultimate_parentsubsidiaries_countownership_percentageshareholder_nameshare_classshares_heldvoting_rightsrelationship_type
corporate_structure
● 200 OK
"company_number": "01234567",
"parent_company": "ACME GROUP PLC",
"ultimate_parent": "GLOBAL HOLDINGS INC",
"subsidiaries_count": 4,
"ownership_percentage": 100.0,
"shareholder_name": "ACME GROUP PLC",
"share_class": "Ordinary",
"shares_held": 10000
# company_numberparent_companyultimate_parentsubsidiaries_countownership_percentageshareholder_name
1
2
3

Complete list of extractable fields for Credit & Risk objects from duedil.com. All fields typed and schema-versioned.

company_numbercredit_scorecredit_limitrisk_banddays_beyond_termsccj_countccj_total_valuelatest_ccj_datemortgage_chargessatisfaction_count
credit_& risk
● 200 OK
"company_number": "01234567",
"credit_score": 85,
"credit_limit": 250000.0,
"risk_band": "Low Risk",
"days_beyond_terms": 12,
"ccj_count": 0,
"ccj_total_value": 0.0,
"mortgage_charges": 2
# company_numbercredit_scorecredit_limitrisk_banddays_beyond_termsccj_count
1
2
3

Capabilities

Deep corporate intelligence at scale

Our Duedil scraper extracts complex corporate entity data: deep financial histories, director cross-appointments, and ultimate beneficial ownership structures, delivered as clean relational data.

Full Company Firmographics

Extract registered address, trading addresses, SIC codes, employee brackets, and contact details.

Financial History Extraction

Capture 10-year P&L, balance sheet, and cash flow statements filed at Companies House.

Director & Officer Mapping

Track active and resigned directors, secretaries, and their cross-board appointments.

UBO & Shareholder Data

Identify Ultimate Beneficial Owners and cap table structures with ownership percentages.

Corporate Hierarchy Trees

Map parent companies, subsidiaries, and joint ventures across complex international groups.

Credit Risk Indicators

Extract credit scores, suggested credit limits, and risk band classifications.

CCJ & Mortgage Tracking

Monitor County Court Judgments, active charges, and satisfied mortgages.

Event Triggers

Track status changes, new director appointments, or recent financial filings.

Scheduled Updates

Run continuous pipelines at monthly or quarterly cadences to capture latest filings.

// engagement pipeline

From target list to structured database

Brief in. Clean data out.

Define Scope
d 0

Provide company numbers, SIC codes, or revenue brackets. We design the extraction schema.

Pipeline Build
d 2–4

We configure Scrapy crawlers, proxy rotation, and session management for duedil.com.

Validation & QA
d 4–6

Schema validation, null-rate checks, and financial data integrity tests before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket or PostgreSQL database on agreed cadence.

Under the hood

Overcoming financial data extraction challenges

Extracting structured financial data from business directories requires handling diverse table formats and strict rate limits.

pipeline-monitor · duedil.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Anti-bot layer
UK residential proxy rotation

Duedil monitors request volumes and IP reputation. We route traffic through UK-based residential proxies to maintain access and avoid rate limits.

Authentication handling
Cookie rotation and session management

Extracting deep financial histories requires authenticated sessions. We manage credential pools and cookie rotation to prevent account bans.

Complex DOM structures
Resilient XPath selectors

Financial tables vary wildly by company size and filing type. Our selectors normalise these structures into consistent schemas.

Rate limiting
Distributed crawl architecture

We respect platform rate limits by distributing requests across hundreds of concurrent workers with random jitter.

Data integrity
Automated schema validation

Financial data requires strict typing. We validate all numerical fields and cross-check totals before delivery.

Applications

How teams use Duedil data

Teams across industries use duedil.com data to build competitive products and smarter operations.

01
B2B Lead Generation

Sales teams use firmographic and financial filters to identify high-growth target accounts.

02
Credit Risk Assessment

Lenders ingest financial histories and CCJ records to automate initial credit scoring models.

03
Master Data Management

Enterprise data teams enrich internal CRM records with canonical registered addresses and SIC codes.

04
KYC & AML Compliance

Compliance officers map director networks and UBOs to identify conflicts of interest or sanctioned entities.

05
Market Research

Analysts aggregate sector-level financial performance using SIC codes and turnover data.

06
Competitor Intelligence

Strategy teams monitor rival company filings, subsidiary creation, and director movements.

Why DataFlirt

"Duedil aggregates the UK corporate landscape into a single view, but extracting that network graph requires serious infrastructure."

Most teams underestimate the complexity of scraping financial data. Normalising inconsistent table structures, managing authenticated sessions, and handling pagination across complex corporate trees requires dedicated engineering. DataFlirt manages the extraction pipeline so your data scientists can focus on modelling risk and opportunity.

Technical Spec

Duedil scraper technical specifications

Everything supported by our duedil.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

Firmographics
Registered address, SIC codes, status
Supported
Financials
P&L, balance sheet, cash flow
Supported
Directors
Active and resigned officers
Supported
Ownership
UBOs and shareholders
Supported
Credit Scores
Duedil proprietary risk bands
Supported
CCJs
Court judgment history
Supported
Change detection
Hash-based diffs for new filings
Supported
Webhook delivery
HTTP POST for event triggers
Supported
Bulk API Export
Access to Duedil internal bulk API
Partial
Private Company Cap Tables
Unfiled cap table data
Partial
Infrastructure

Infrastructure powering the pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration. Playwright handles JavaScript rendering for dynamic charts.

UK Residential Proxies

We maintain pools of UK-based residential IPs to ensure high success rates and low latency.

Cloud-Native Orchestration

Pipelines run on AWS ECS. Airflow handles scheduling. All state stored in managed Postgres.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested arrays
CSV
Flat file with typed columns
XLS
Excel compatible format for analysts
Parquet
Columnar format for data warehouses
AWS S3
Direct bucket delivery
Webhook
HTTP POST per record
API
REST endpoint for on-demand querying
PostgreSQL
Direct database upserts
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About duedil.com scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Duedil legal?

Scraping public company data is generally permissible. We target unauthenticated and authenticated data per client requirements, ensuring compliance with UK data protection regulations.

How do you handle Duedil rate limits?

We use distributed crawling, UK residential proxies, and request jitter to stay within acceptable request thresholds and avoid IP bans.

Can you extract data for specific SIC codes?

Yes. We can seed the pipeline with specific industry codes, revenue brackets, or geographical regions.

How fresh is the financial data?

We can schedule pipelines to run daily, capturing new Companies House filings as soon as they are processed by Duedil.

Do you extract full director appointment histories?

Yes. We capture current and historical directorships, including cross-board appointments and resignation dates.

Can I get a sample dataset?

Yes. We provide a sample run of 100 company profiles during the scoping phase to validate schema fit and data quality.

$ dataflirt scope --new-project --source=duedil.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a full UK corporate database export or targeted daily financial updates, we build and manage the infrastructure.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in business directories

Services

Data Extraction for Every Industry

View All Services →