SYSTEM all green source dnb.com queue 12,492 profiles p99 latency 318ms dataflirt.com · scraper/dnb-com
RUN - 84 active pipelines - dnb.com live

D&B firmographics,
at warehouse scale.

We extract company profiles, DUNS numbers, corporate hierarchies, and executive data from Dun & Bradstreet. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Companies extracted
1.2M /month
Executive records
4.8M /month
Hierarchy maps
85K /run
Active pipelines
84
Uptime
99.94%
Data Dictionary

Every field we extract from dnb.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Company Overview objects from dnb.com. All fields typed and schema-versioned.

duns_numbercompany_namelegal_nameaddress_line_1citystatepostal_codecountryphone_numberwebsite_urlyear_foundedcompany_description
company_overview
● 200 OK
"duns_number": "00-123-4567",
"company_name": "Acme Manufacturing Ltd",
"legal_name": "Acme Manufacturing Limited",
"city": "London",
"country": "United Kingdom",
"year_founded": 1985,
"website_url": "https://www.acmemfg.co.uk"
# duns_numbercompany_namelegal_nameaddress_line_1citystate
1
2
3

Complete list of extractable fields for Firmographics objects from dnb.com. All fields typed and schema-versioned.

duns_numberrevenue_estimated_usdemployee_countindustry_primarynaics_codenaics_descriptionsic_codesic_descriptioncompany_typefiscal_year_end
firmographics
● 200 OK
"duns_number": "00-123-4567",
"revenue_estimated_usd": 45000000.0,
"employee_count": 250,
"industry_primary": "Manufacturing",
"naics_code": "332710",
"sic_code": "3599",
"company_type": "Private"
# duns_numberrevenue_estimated_usdemployee_countindustry_primarynaics_codenaics_description
1
2
3

Complete list of extractable fields for Executives objects from dnb.com. All fields typed and schema-versioned.

duns_numberexecutive_idfull_namejob_titledepartmentmanagement_levelboard_memberbiography_snippet
executives
● 200 OK
"duns_number": "00-123-4567",
"full_name": "Jane Doe",
"job_title": "Chief Executive Officer",
"department": "Executive",
"management_level": "C-Level",
"board_member": true
# duns_numberexecutive_idfull_namejob_titledepartmentmanagement_level
1
2
3

Complete list of extractable fields for Corporate Hierarchy objects from dnb.com. All fields typed and schema-versioned.

duns_numbercompany_nameparent_dunsparent_nameultimate_parent_dunsultimate_parent_namehierarchy_levelsubsidiary_countbranch_count
corporate_hierarchy
● 200 OK
"duns_number": "00-123-4567",
"parent_duns": "00-987-6543",
"parent_name": "Acme Global Holdings",
"ultimate_parent_duns": "00-987-6543",
"hierarchy_level": "Subsidiary",
"subsidiary_count": 2
# duns_numbercompany_nameparent_dunsparent_nameultimate_parent_dunsultimate_parent_name
1
2
3

Complete list of extractable fields for Location Data objects from dnb.com. All fields typed and schema-versioned.

duns_numberlocation_typeaddress_fullstreetcitystate_provincepostal_codecountry_isolatitudelongitude
location_data
● 200 OK
"duns_number": "00-123-4567",
"location_type": "Headquarters",
"city": "London",
"country_iso": "GB",
"latitude": 51.5074,
"longitude": -0.1278,
"postal_code": "EC1A 1BB"
# duns_numberlocation_typeaddress_fullstreetcitystate_province
1
2
3

Capabilities

Everything you need from Dun & Bradstreet

Our D&B scraper handles the business directory layer: firmographic profiles, corporate hierarchies, executive lists, and industry classifications, with bypass mechanisms for regional blocks and strict rate limits.

Firmographic Extraction

Extract revenue estimates, employee headcounts, year founded, and company descriptions for millions of public and private entities.

DUNS Number Mapping

Capture the unique nine-digit Data Universal Numbering System identifier to match records against your existing CRM data.

Corporate Family Trees

Map parent, subsidiary, and branch relationships to understand ultimate beneficial ownership and corporate structures.

Executive Leadership

Extract key principals, C-suite executives, and board members associated with specific corporate entities.

Industry Classification

Standardise your data with extracted NAICS, SIC, and proprietary D&B industry codes for precise market segmentation.

Global Directory Coverage

Scrape regional D&B directories across North America, Europe, and Asia to build international prospect lists.

Location Mapping

Capture headquarters addresses, branch locations, and geographic coordinates for spatial analysis.

Change Detection

Monitor specific DUNS numbers for changes in executive leadership, revenue brackets, or corporate structure over time.

High-Volume Pagination

Navigate deep category and geographic search results to extract entire industry verticals without hitting display limits.

// engagement pipeline

From directory search to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Provide target industries, geographies, or specific company names. We design the extraction schema together.

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers, proxy rotation, session management, and CAPTCHA handling for dnb.com.

Validation & QA
d 4–6

Schema validation, null-rate checks, and DUNS format verification before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

How our D&B pipeline handles the hard parts

Dun & Bradstreet protects its directory with aggressive rate limiting and bot detection. Here is how we maintain extraction stability.

pipeline-monitor · dnb.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Anti-bot layer
Residential proxy rotation and rate limiting

D&B monitors request velocity strictly. We distribute requests across large residential ISP proxy pools and enforce strict delays between requests to mimic organic browsing behaviour and prevent IP bans.

JavaScript rendering
Playwright execution for directory results

Many directory pages load content dynamically via JavaScript. We use Playwright to execute scripts, trigger lazy-loaded elements, and extract data that simple HTTP requests cannot see.

Pagination handling
Deep crawl strategies

Directory search results often cap visible pages. We use granular search parameters across postal codes and sub-industries to reduce result sets below the cap, ensuring complete coverage of a target sector.

Schema stability
Resilient DOM parsing

We maintain multiple fallback selectors for firmographic data points, as D&B frequently updates profile layouts and obfuscates class names in their frontend code.

Data normalisation
Standardised outputs

Revenue figures, employee counts, and addresses are cleaned and cast into correct data types before delivery, ensuring they are immediately ready for database insertion.

Applications

Who uses D&B data and how

Teams across industries use dnb.com data to build competitive products and smarter operations.

01
CRM Enrichment

Sales operations teams append DUNS numbers, revenue data, and accurate industry codes to incomplete Salesforce or HubSpot records.

02
Master Data Management

Data engineering teams use D&B profiles as a source of truth to deduplicate and standardise vendor and customer databases.

03
Lead Generation

Marketing teams extract target accounts by specific NAICS codes and revenue brackets to build highly segmented outbound campaigns.

04
Risk & Compliance

Compliance officers map corporate hierarchies to identify ultimate beneficial owners and assess third-party risk exposure.

05
Market Sizing

Private equity firms analyze aggregate employee and revenue data across specific sectors to model total addressable market size.

06
Vendor Onboarding

Procurement teams automate vendor verification by matching submitted details against public D&B registry records.

Why DataFlirt

"Dun & Bradstreet is the foundational registry of global commerce, but mapping millions of corporate entities requires purpose-built extraction infrastructure."

Most teams underestimate the investment required: reliable dnb.com scraping requires residential proxies, strict rate-limit management, CAPTCHA handling, and daily selector maintenance. DataFlirt absorbs that complexity so your engineers can focus on the analysis, not the infrastructure.

Technical Spec

D&B scraper technical capabilities

Everything supported by our dnb.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Playwright sessions for dynamic directory loading and search results
Supported
CAPTCHA bypass
Automated solver integration for strict perimeter defenses
Supported
Residential proxy rotation
Geographically targeted IPs to access regional D&B directories
Supported
Corporate hierarchy mapping
Extract parent and subsidiary links where publicly listed
Supported
Executive extraction
Capture listed leadership and board members
Supported
Change detection
Track updates to specific company profiles over time
Supported
Webhook delivery
HTTP POST for real-time CRM enrichment workflows
Supported
Industry code mapping
Extract standard NAICS and SIC classifications
Supported
D&B Paydex / Credit Scores
Financial risk scores require authenticated D&B Finance Analytics access
Partial
D&B Hoovers deep financials
Detailed historical financial statements are gated behind paid logins
Partial
Infrastructure

Infrastructure powering the D&B pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, cookie sessions, and interaction flows. Combined via scrapy-playwright middleware.

Residential Proxy Infrastructure

We maintain pools of residential ISP proxies across target regions. Rotation happens per-request with sticky sessions where required. IP score monitoring prevents blacklisted pool contamination.

Cloud-Native Orchestration

Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested arrays for complex hierarchies
CSV
Flat file with typed columns for spreadsheet analysis
XLS
Excel format for business users and analysts
Parquet
Columnar format for BigQuery, Snowflake, Athena
AWS S3
Direct bucket delivery compatible with any data lake
Webhook
HTTP POST per record for real-time processing
API
REST endpoints to query your extracted datasets
BigQuery
Streamed directly into your dataset
Snowflake
Stage and copy workflow for warehouse ingestion
PostgreSQL
Direct database insertion with upsert logic
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About dnb.com scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Dun & Bradstreet legal?

Scraping publicly available directory information is generally permissible under applicable law. DataFlirt targets only public, non-authenticated company profiles and executive data. We do not circumvent authentication walls to access paid credit reports or proprietary financial scores. Clients should review terms of service and consult legal counsel for specific use cases.

How do you handle rate limits on dnb.com?

We use large residential ISP proxy pools and throttle request concurrency to stay within acceptable thresholds. Our infrastructure mimics human browsing behaviour to prevent automated blocks.

Can you extract full corporate hierarchies?

Yes, we can map parent and subsidiary relationships where they are explicitly listed in the public directory profiles, providing a structured view of corporate family trees.

Do you provide D&B credit scores or Paydex data?

No. D&B credit scores and detailed financial risk metrics are gated behind paid authentication walls (such as D&B Finance Analytics). We only extract publicly accessible firmographic data.

Can I match my existing list of companies to DUNS numbers?

Yes. You can provide a list of company names and addresses, and we will configure the pipeline to search the directory and return the corresponding DUNS numbers and profile data.

How fresh is the data?

We can configure pipelines to run on your required cadence, whether that is a one-off historical extraction or a monthly refresh to detect changes in leadership or company status.

What is the minimum viable engagement?

Our minimum engagement typically starts at 10,000 target companies or a specific industry vertical. Contact us with your target criteria for a custom scoping and quote.

$ dataflirt scope --new-project --source=dnb.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off firmographic dump or a continuous CRM enrichment feed across millions of entities, we scope, build, and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Services

Data Extraction for Every Industry

View All Services →