We extract news articles, opinion pieces, market updates, and author metadata from Folha de S.Paulo. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Article Content objects from folha.uol.com.br. All fields typed and schema-versioned.
"url": "https://www1.folha.uol.com.br/mercado/2026/05/banco-central-mantem-taxa.shtml", "headline": "Banco Central mantém taxa Selic", "author": "Maria Silva", "published_at": "2026-05-12T14:30:00Z", "category": "Mercado", "paywalled": false, "word_count": 842, "tags": "['Banco Central', 'Economia', 'Selic']"
| # | url | headline | subheadline | author | published_at | updated_at |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Author Metadata objects from folha.uol.com.br. All fields typed and schema-versioned.
"author_id": "colunista-492", "name": "João Pereira", "column_name": "Opinião Econômica", "twitter_handle": "@jpereira_econ", "article_count": 341, "role": "Colunista", "latest_article_url": "https://www1.folha.uol.com.br/colunas/joaopereira/2026/05/inflacao.shtml"
| # | author_id | name | bio | twitter_handle | column_name | article_count |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Category Feeds objects from folha.uol.com.br. All fields typed and schema-versioned.
"category_name": "Poder", "sub_category": "Eleições", "url": "https://www1.folha.uol.com.br/poder/", "top_headline": "Congresso aprova nova lei eleitoral", "article_count_24h": 45, "feed_updated_at": "2026-05-12T15:00:00Z", "trending_topics": "['Congresso', 'Votação', 'Senado']"
| # | category_name | sub_category | url | top_headline | top_headline_url | article_count_24h |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Market News objects from folha.uol.com.br. All fields typed and schema-versioned.
"article_url": "https://www1.folha.uol.com.br/mercado/2026/05/petrobras-lucro.shtml", "ticker_mentions": "['PETR4', 'PETR3']", "company_mentions": "['Petrobras']", "market_sector": "Energia", "publication_date": "2026-05-12T09:15:00Z", "paywalled": true, "sentiment_indicator": "positive"
| # | article_url | ticker_mentions | company_mentions | publication_date | author | related_links |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Search Results objects from folha.uol.com.br. All fields typed and schema-versioned.
"keyword": "reforma tributária", "position": 1, "article_url": "https://www1.folha.uol.com.br/poder/2026/04/reforma.shtml", "headline": "Entenda os pontos da reforma", "publication_date": "2026-04-28T10:00:00Z", "author": "Redação", "scraped_at": "2026-05-12T15:05:00Z"
| # | keyword | position | article_url | headline | publication_date | relevance_score |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Our Folha scraper handles dynamic feeds, multi-section layouts, and article archives. We manage the infrastructure required to extract clean text and metadata from one of Latin America's largest news portals.
Extract body text, subheadings, blockquotes, and embedded media links from news articles and opinion pieces.
Capture publication timestamps, modification dates, author attribution, and internal tagging systems for every URL.
Track front page and section fronts (Poder, Mercado, Esporte, Ilustrada) to record article placement and layout prominence.
Map articles to specific columnists, extracting bio data, social handles, and historical publication frequencies.
Traverse Folha's deep archives to build longitudinal datasets for NLP training and historical trend analysis.
Identify whether an article is freely available, metered, or strictly paywalled, logging the state at the time of extraction.
Run recurring queries against Folha's internal search engine to track specific entities, politicians, or corporate brands.
Configure pipelines to poll breaking news sections at sub-15 minute intervals for real-time media monitoring.
Strip navigation, sidebar ads, and footer content to deliver pure editorial text ready for machine learning ingestion.
Brief in. Clean data out.
Provide target sections, author lists, or historical date ranges. We map the extraction schema to your requirements.
We configure Scrapy crawlers, handle dynamic section loading, and implement boilerplate removal algorithms.
Schema validation, null-rate checks on author fields, and text-cleanliness verification before full launch.
JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.
Major news outlets employ dynamic layouts and strict access controls. Here is how we ensure reliable data extraction from Folha.
Folha uses different DOM structures for standard news, opinion columns, interactive graphics, and long-form features. Our selector strategy uses fallback chains to ensure text extraction succeeds regardless of the template deployed.
To extract publicly available content without triggering aggressive rate limits or soft paywalls prematurely, we rotate IPs and clear session cookies aggressively across our residential proxy pool.
Scraping years of historical data requires handling legacy HTML structures. We maintain specific parser rules for older article formats that differ from the modern front-end stack.
News stories evolve after initial publication. We hash the body text and monitor the 'updated_at' timestamps, emitting diff records when an article is corrected or expanded.
We deploy deterministic algorithms to strip inline advertisements, 'read more' injection blocks, and social sharing widgets, ensuring the output text contains only the editorial content.
AI labs ingest high-quality Portuguese editorial text to train foundational language models and improve regional dialect comprehension.
PR agencies and corporate communications teams track brand mentions, executive visibility, and sentiment across major news cycles.
Consultancies monitor the 'Poder' section to quantify coverage of legislative changes, tracking entity co-occurrence for risk models.
Quantitative funds extract data from the 'Mercado' section, correlating corporate news volume and sentiment with B3 market movements.
Sociologists and political scientists analyse decades of archive data to study shifts in public discourse and media framing.
Rival media organisations track Folha's publication velocity, author output, and section focus to benchmark their own editorial operations.
"Folha de S.Paulo remains the definitive record of Brazilian political and economic discourse - but extracting that historical corpus requires serious infrastructure."
News aggregators and LLM builders underestimate the complexity of scraping major publishers. Reliable extraction requires residential proxies, strict session management to navigate metered access, and persistent selectors for legacy archive pages. DataFlirt handles the extraction so you can focus on the NLP models.
Everything supported by our folha.uol.com.br scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, cookie sessions, and interaction flows. Combined via scrapy-playwright middleware.
We maintain pools of residential ISP proxies. Rotation happens per-request with sticky sessions where required. IP score monitoring prevents blacklisted pool contamination.
Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.
Data delivered to where your team already works — no new tooling required.
About folha.uol.com.br scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available factual data and headlines is generally permissible. DataFlirt targets only public and metered content. We do not circumvent hard paywalls without client-provided credentials, nor do we extract PII from user accounts. Clients must ensure their downstream use of copyrighted text complies with fair use doctrines.
We manage session state and proxy rotation to extract content that is accessible via metered allowances or public links. For content strictly behind a hard paywall, extraction requires the client to supply valid subscription credentials.
Yes. We can traverse Folha's sitemaps and internal search architecture to extract articles dating back to the limits of their digital archive.
For continuous monitoring, we configure pipelines to poll section fronts (like Poder or Mercado) at sub-15 minute intervals, pushing new URLs to the extraction queue immediately.
Yes. Our parsers strip out navigational elements, inline advertisements, social sharing prompts, and 'read more' blocks, delivering clean editorial text optimised for NLP training.
Our minimum engagement starts at 5,000 URLs per run or a continuous monitoring setup for specific categories. We price based on compute volume and delivery frequency.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a historical archive dump for NLP training or a continuous feed of market news - we scope, build, and operate the pipeline. Tell us what you need.
