We extract article bodies, author profiles, publication timestamps, and section metadata from publico.pt. Delivered as clean JSON, CSV, or Parquet to S3 or PostgreSQL on your scheduled cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Articles objects from publico.pt. All fields typed and schema-versioned.
"url": "https://www.publico.pt/2026/05/12/politica/noticia/exemplo-artigo-123456", "title": "Novo pacote legislativo aprovado no Parlamento", "author": "Maria João", "publish_date": "2026-05-12T14:30:00Z", "section": "Política", "paywall_status": "premium", "tags": "['Parlamento', 'Legislação', 'Governo']"
| # | url | title | subtitle | author | publish_date | update_date |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Authors objects from publico.pt. All fields typed and schema-versioned.
"author_id": "maria-joao-789", "name": "Maria João", "profile_url": "https://www.publico.pt/autor/maria-joao", "role": "Jornalista", "article_count": 412, "twitter_handle": "@mariajoaopt"
| # | author_id | name | profile_url | role | bio | article_count |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Topics & Tags objects from publico.pt. All fields typed and schema-versioned.
"tag_id": "t-parlamento", "name": "Parlamento", "url": "https://www.publico.pt/parlamento", "article_count": 8942, "category": "Política", "last_updated": "2026-05-12T15:00:00Z"
| # | tag_id | name | url | article_count | related_tags | trending_status |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Comments & Engagement objects from publico.pt. All fields typed and schema-versioned.
"article_url": "https://www.publico.pt/2026/05/12/politica/noticia/exemplo-artigo-123456", "comment_count": 145, "share_count": 320, "engagement_score": 8.4, "discussion_status": "open", "scraped_at": "2026-05-12T16:45:12Z"
| # | article_url | comment_count | share_count | top_comment | top_commenter | engagement_score |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Multimedia objects from publico.pt. All fields typed and schema-versioned.
"article_url": "https://www.publico.pt/2026/05/12/politica/noticia/exemplo-artigo-123456", "main_image_url": "https://imagens.publico.pt/imagens.aspx/123456", "image_caption": "Sessão plenária na Assembleia da República", "image_credit": "Lusa / António Silva", "gallery_count": 4, "media_type": "image_gallery"
| # | article_url | main_image_url | image_caption | image_credit | video_url | podcast_link |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Our publico.pt scraper captures the full editorial output: standard articles, opinion pieces, P3 youth content, and Ípsilon culture reviews. We handle infinite scroll pagination and article template variations.
Extract headline, subtitle, body text, publication date, and update timestamps across all news sections.
Capture journalist names, contributor roles, and profile URLs to track editorial output per author.
Distinguish between standard news, opinion columns, P3, and Ípsilon content with strict schema mapping.
Identify and flag Premium articles versus open-access content to maintain dataset consistency.
Extract native tagging, keywords, and related topics to understand editorial categorisation.
Capture main image URLs, captions, photo credits, and embedded video or podcast links.
Monitor discussion activity by extracting total comment counts and engagement metrics per article.
Text is delivered clean, stripped of HTML boilerplate, and UTF-8 encoded for immediate ingestion into language models.
Configure hourly sweeps of the homepage and section fronts to capture breaking news as it publishes.
Brief in. Clean data out.
Specify target sections, authors, or date ranges. We map the required data fields to your schema.
We deploy Scrapy and Playwright spiders configured to handle publico.pt pagination and layout variations.
We verify text cleanliness, date parsing accuracy, and proper handling of Premium article truncations.
Clean JSON or Parquet files pushed directly to your S3 bucket or PostgreSQL database.
News platforms present unique extraction challenges including dynamic loading, varied article templates, and strict anti-bot measures.
Category pages and author feeds on publico.pt rely on infinite scroll. We use Playwright to simulate user scrolling, triggering subsequent API calls and rendering full article lists without missing historical entries.
Longform features, opinion pieces, and standard news articles use different DOM structures. Our selectors employ multi-layer fallback chains to ensure consistent data extraction regardless of the editorial layout.
Publico.pt truncates body text for Premium articles when viewed without an active subscription. We detect the paywall state and flag the record, preventing truncated text from contaminating your full-text corpus.
To prevent IP bans and rate limiting from media firewalls, we route requests through Portuguese residential proxies, mimicking legitimate domestic reader traffic.
We strip inline advertisements, newsletter sign-up widgets, and related-article injects from the body text, delivering a contiguous string of editorial content ready for NLP processing.
PR agencies and corporate communications teams track brand mentions, sentiment, and crisis developments across major Portuguese news outlets.
AI research teams ingest high-quality editorial Portuguese text to train language models and improve translation algorithms.
Think tanks and researchers monitor opinion columns, editorial bias, and topic frequency during election cycles.
Other media organisations track publication velocity, author output, and section engagement to benchmark editorial performance.
Sociologists and linguists analyse long-term cultural trends and discourse evolution using historical article archives.
Analysts track keyword frequency and tag associations in the P3 and Ípsilon sections to identify emerging cultural trends.
"Publico.pt represents a critical corpus of Portuguese journalism, but extracting structured text across varied article templates requires dedicated infrastructure."
Media monitoring and LLM training require clean, structured text. We handle the infinite scrolls, paywall logic, and layout variations so your data science teams receive normalised JSON ready for ingestion.
Everything supported by our publico.pt scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy manages the broad crawl logic and deduplication, while Playwright handles infinite scroll and dynamic widget rendering on category pages.
We maintain pools of Portuguese residential IPs, rotating them per request to avoid triggering media firewalls and rate limits.
Pipelines run on AWS ECS with Airflow managing schedules. This ensures high availability for continuous news monitoring.
Data delivered to where your team already works — no new tooling required.
About publico.pt scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available news articles is generally permissible for non-consumptive uses like NLP training and media monitoring. DataFlirt extracts only public data and respects standard web scraping guidelines. We do not bypass authentication to steal Premium content. Clients must ensure their specific use case complies with copyright law and the publisher's terms of service.
Our standard pipeline extracts the publicly available text. For Premium articles, we extract the visible truncated text, metadata, and tags, and flag the record with a 'premium' status. We do not circumvent the paywall to extract gated text unless you provide valid enterprise credentials for authenticated scraping.
Yes. We can configure a backfill pipeline to traverse category archives and search results to extract historical articles based on specific date ranges or keywords.
For time-sensitive media monitoring, we can configure pipelines to poll the homepage, RSS feeds, and main section URLs every 15 minutes, delivering new articles via Webhook immediately upon detection.
Yes. We monitor the 'update_date' metadata field. If an article is revised post-publication, our pipeline detects the change, extracts the updated text, and pushes a new version to your database.
Our minimum engagement typically starts with daily extraction of specific sections or keyword alerts. For full-site historical backfills, we price based on total article volume and compute requirements.
Yes. We provide a sample dataset of recent articles across different sections (Standard, Opinion, Premium) so your team can validate the text cleanliness and schema structure before committing.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a historical archive export or a continuous daily feed of Portuguese news, we scope, build, and operate the pipeline. Tell us what you need.
