SYSTEM all green source ftc.gov queue 12,408 documents p99 latency 312ms dataflirt.com · scraper/ftc-gov
RUN · 42 active pipelines · ftc.gov live

FTC regulatory data,
at warehouse scale.

We extract enforcement cases, warning letters, HSR early termination notices, and public dockets from ftc.gov. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Cases extracted
18,294 /total
Warning letters
4,192 /total
HSR notices
142 /week
Active pipelines
42
Uptime
99.98%
Data Dictionary

Every field we extract from ftc.gov

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Enforcement Actions objects from ftc.gov. All fields typed and schema-versioned.

case_namecase_numberdate_filedmatter_typeindustrydefendantscourtcase_summarydocument_urlsstatus
enforcement_actions
● 200 OK
"case_name": "Epic Games, Inc.",
"case_number": "1923203",
"date_filed": "2023-02-07",
"matter_type": "Consumer Protection",
"industry": "Technology",
"status": "Settled",
"court": "Federal District Court"
# case_namecase_numberdate_filedmatter_typeindustrydefendants
1
2
3

Complete list of extractable fields for Warning Letters objects from ftc.gov. All fields typed and schema-versioned.

company_nameissue_dateletter_idrecipientsubjectviolation_typepdf_urltext_contentresponse_deadline
warning_letters
● 200 OK
"company_name": "HealthCorp Solutions",
"issue_date": "2024-01-15",
"letter_id": "WL-2024-001",
"violation_type": "Deceptive Advertising",
"response_deadline": "2024-01-30",
"pdf_url": "https://www.ftc.gov/system/files/warning-letters/wl-2024-001.pdf"
# company_nameissue_dateletter_idrecipientsubjectviolation_type
1
2
3

Complete list of extractable fields for HSR Early Terminations objects from ftc.gov. All fields typed and schema-versioned.

transaction_numberdate_clearedacquiring_partyacquired_partytarget_entityindustry_codenotice_urlclearance_type
hsr_early terminations
● 200 OK
"transaction_number": "20240182",
"date_cleared": "2024-03-12",
"acquiring_party": "Alpha Holdings LLC",
"acquired_party": "Beta Technologies Inc",
"industry_code": "511210",
"clearance_type": "Granted"
# transaction_numberdate_clearedacquiring_partyacquired_partytarget_entityindustry_code
1
2
3

Complete list of extractable fields for Public Comments objects from ftc.gov. All fields typed and schema-versioned.

docket_idcomment_idsubmitter_nameorganizationdate_submittedcomment_textattachmentsagency_contact
public_comments
● 200 OK
"docket_id": "FTC-2023-0042",
"comment_id": "FTC-2023-0042-0193",
"submitter_name": "Jane Doe",
"organization": "Consumer Rights Group",
"date_submitted": "2023-11-05",
"attachments": "['comment_brief.pdf']"
# docket_idcomment_idsubmitter_nameorganizationdate_submittedcomment_text
1
2
3

Complete list of extractable fields for Press Releases objects from ftc.gov. All fields typed and schema-versioned.

release_idtitledate_issuedtopicsbureaucontact_inforelease_bodyrelated_casesmedia_urls
press_releases
● 200 OK
"release_id": "PR-2024-089",
"title": "FTC Takes Action Against Deceptive Marketer",
"date_issued": "2024-04-10",
"bureau": "Consumer Protection",
"topics": "['Advertising', 'Health Claims']",
"related_cases": "['1923203']"
# release_idtitledate_issuedtopicsbureaucontact_info
1
2
3

Capabilities

Regulatory intelligence, structured for analysis

Our ftc.gov scraper navigates federal search interfaces, extracts text from scanned legal filings, and monitors new dockets. We handle the infrastructure so you can focus on compliance and risk analysis.

Enforcement Action Tracking

Capture case dockets, defendants, resolutions, and civil penalty amounts across all FTC bureaus.

HSR Early Termination Feed

Track Hart-Scott-Rodino (HSR) antitrust clearance notices for M&A intelligence and financial modelling.

PDF Text Parsing

Automated OCR and text extraction for legal filings, consent orders, and warning letters.

Warning Letter Extraction

Identify targeted companies, regulatory violations, and response deadlines from official correspondence.

Public Docket Monitoring

Track comments, rulemaking updates, and workshop materials across all active FTC dockets.

Consumer Sentinel Reports

Aggregate scam, fraud, and identity theft statistics published by the Consumer Sentinel Network.

Change Detection

Only emit new or updated case files and dockets, reducing downstream processing load.

Cross-Referenced Metadata

Link press releases directly to enforcement actions and underlying legal documentation.

Scheduled Delivery

Hourly or daily pipeline runs to feed compliance systems and risk management platforms.

// engagement pipeline

From federal docket to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Provide target bureaus, document types, or keyword sets. We design the extraction schema together.

Pipeline Build
d 2–4

We configure crawlers, PDF parsers, and rate-limit handling for ftc.gov infrastructure.

Validation & QA
d 4–6

Schema validation, null-rate checks, and PDF OCR accuracy testing before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket or Snowflake stage on agreed cadence.

Under the hood

How our FTC pipeline handles the hard parts

Federal websites present unique extraction challenges. Here is how we build resilient pipelines for government data.

pipeline-monitor · ftc.gov · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Rate Limiting
Respecting federal infrastructure limits

Government websites employ strict IP quotas and anti-DDoS measures via Cloudflare or Akamai. We use distributed US-based residential proxies and polite request pacing to ensure reliable access without triggering security blocks.

Unstructured Data
PDF parsing and OCR

A massive volume of FTC enforcement data exists only as scanned PDF documents. Our pipeline includes automated OCR and NLP to extract structured entities like defendants, penalties, and dates from unstructured legal text.

Legacy Systems
Navigating inconsistent docket formats

The FTC archives span decades, resulting in inconsistent HTML structures and legacy ASP.NET search interfaces. Our selector strategy uses fallback chains to normalise data across different eras of the website.

State Management
Pagination and search state

Extracting complete dockets requires maintaining complex session states and handling JavaScript-heavy pagination. Playwright handles the interaction flows to ensure zero dropped records.

Efficiency
Stateful change detection

We maintain a hash index of all known cases and documents. Subsequent runs only extract and deliver new filings or status updates, providing a clean changelog rather than redundant full exports.

Applications

Who uses FTC data — and how

Teams across industries use ftc.gov data to build competitive products and smarter operations.

01
M&A Risk Analysis

Financial analysts track HSR early terminations for real-time merger intelligence and regulatory clearance signals.

02
Compliance Monitoring

Corporate compliance teams ingest warning letters and enforcement actions into risk platforms to monitor industry precedents.

03
Legal Research

Law firms build searchable, internal databases of FTC enforcement precedents and consent orders.

04
Consumer Protection AI

Machine learning teams train risk models on scam alerts and Consumer Sentinel Network statistics.

05
Vendor Risk Management

Procurement platforms screen suppliers against federal enforcement actions and deceptive practice warnings.

06
Regulatory News Feeds

Data providers power financial terminals with real-time FTC press releases and docket updates.

Why DataFlirt

"Regulatory intelligence requires structured data, but federal websites are built for human reading, not machine extraction."

Extracting data from ftc.gov means navigating legacy search interfaces, unstructured PDF filings, and strict government rate limits. DataFlirt handles the extraction, OCR, and normalisation so your compliance teams can focus on risk analysis instead of document retrieval.

Technical Spec

FTC scraper — technical capabilities

Everything supported by our ftc.gov scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

HSR Early Terminations
Real-time tracking of antitrust clearance notices
Supported
PDF OCR & Text Extraction
Automated parsing of scanned consent orders and filings
Supported
Enforcement Action Dockets
Full case history, defendants, and resolutions
Supported
Warning Letter Metadata
Recipient details, violation types, and deadlines
Supported
Consumer Sentinel Aggregates
Fraud and identity theft statistics by category
Supported
Change Detection (Diffs)
Only emit records with changed fields since last run
Supported
Webhook Delivery
HTTP POST per record for real-time alerting
Supported
Non-Public Investigation Files
Confidential or sealed FTC investigations
Partial
Unredacted Consumer Complaints
Raw consumer complaints containing PII
Partial
Infrastructure

Infrastructure powering the FTC pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheusTesseract OCRPyPDF2
PDF Processing Pipeline

Integrated Tesseract OCR and PyPDF2 handle the massive volume of unstructured legal filings, converting scanned images into structured text for downstream NLP.

Rate-Limit Compliant Crawling

We maintain pools of US residential ISP proxies. Rotation is tuned to respect federal infrastructure limits, ensuring reliable access without triggering DDoS protections.

Cloud-Native Orchestration

Pipelines run on Kubernetes. Airflow handles scheduling, dependency management, and SLA alerting. All state is stored in managed Postgres for precise change detection.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested — schema versioned per run
CSV
Flat file with typed columns — Excel/Sheets compatible
XLS
Excel format for direct analyst consumption
Parquet
Columnar format for BigQuery, Snowflake, Athena
AWS S3
Direct bucket delivery — compatible with any data lake
Webhook
HTTP POST per record for real-time downstream processing
API
REST endpoints for on-demand data retrieval
PostgreSQL
Upsert into your existing schema with conflict resolution
Snowflake
Stage + COPY INTO workflow — incremental or full-replace
BigQuery
Streamed directly into your dataset with schema auto-detect
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About ftc.gov scraping, legality, and pipeline operations.

Ask us directly →
Is scraping ftc.gov legal?

Yes. The FTC publishes public records, enforcement actions, and press releases for public consumption. DataFlirt extracts only publicly available information, respects robots.txt directives, and strictly avoids any restricted or non-public endpoints.

How do you handle PDF documents?

Our pipeline automatically downloads PDF files linked in dockets or warning letters. We use OCR (Optical Character Recognition) and text extraction libraries to convert the documents into machine-readable text, which is then mapped to structured JSON fields.

Can you track HSR early terminations in real-time?

Yes. We configure pipelines to poll the HSR early termination notices at high frequency (e.g., hourly) and deliver updates via webhook for immediate analyst review.

Do you extract historical enforcement actions?

Yes. We can perform a full historical backfill of all available enforcement actions and dockets on ftc.gov, followed by daily delta updates to capture new filings.

How fresh is the data?

Depending on the endpoint, we can configure hourly or daily runs. Press releases and HSR notices can be polled frequently, while deep docket crawls typically run on a daily cadence.

What formats do you deliver?

We deliver structured data in JSON, CSV, XLS, and Parquet formats. Files can be pushed directly to AWS S3, Snowflake, BigQuery, or delivered via API and Webhook.

$ dataflirt scope --new-project --source=ftc.gov ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a historical backfill of enforcement actions or real-time HSR tracking — we build and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in government and public data

Services

Data Extraction for Every Industry

View All Services →