We extract news articles, Financial Post columns, author metadata, and comment threads from nationalpost.com. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Articles objects from nationalpost.com. All fields typed and schema-versioned.
"url": "https://nationalpost.com/news/politics/example-article", "headline": "Parliament debates new fiscal policy measures", "author": "John Ivison", "publish_date": "2026-05-12T14:30:00Z", "category": "Politics", "paywalled": false
| # | url | headline | subheadline | author | publish_date | update_date |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Financial Post objects from nationalpost.com. All fields typed and schema-versioned.
"ticker_mentions": "['TSX:RY', 'TSX:TD']", "market_category": "Banking", "headline": "Canadian banks report quarterly earnings beat", "author": "Barbara Shecter", "publish_date": "2026-05-12T09:15:00Z", "related_companies": "['Royal Bank of Canada', 'TD Bank']"
| # | url | ticker_mentions | market_category | headline | author | publish_date |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Authors objects from nationalpost.com. All fields typed and schema-versioned.
"author_id": "auth_49281", "name": "Kelly McParland", "role": "Columnist", "twitter_handle": "@KellyMcParland", "article_count": 842, "latest_article_url": "https://nationalpost.com/opinion/example"
| # | author_id | name | role | bio | twitter_handle | article_count |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Comments objects from nationalpost.com. All fields typed and schema-versioned.
"comment_id": "cmt_993821", "user_name": "CanuckReader99", "comment_text": "This policy will have significant impacts on the housing market.", "timestamp": "2026-05-12T15:45:22Z", "upvotes": 42, "downvotes": 3
| # | comment_id | article_url | user_name | user_id | comment_text | timestamp |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Multimedia & Tags objects from nationalpost.com. All fields typed and schema-versioned.
"article_url": "https://nationalpost.com/news/canada/example", "primary_tag": "Canadian Politics", "secondary_tags": "['Housing', 'Interest Rates', 'Bank of Canada']", "seo_title": "Housing market reacts to Bank of Canada rate decision", "meta_description": "A detailed look at how the latest interest rate announcement affects mortgages.", "reading_time_mins": 4
| # | article_url | image_urls | video_ids | primary_tag | secondary_tags | seo_title |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
News sites deploy complex layouts, programmatic advertising, and infinite scrolls. Our pipelines normalise this into clean, NLP-ready datasets.
Extract body paragraphs while stripping programmatic ads, newsletter signups, and related-article injection modules.
Target specific market, investing, and economy verticals with ticker mention extraction and sector classification.
Capture contributor bios, roles, social links, and historical article counts across the Postmedia network.
Render third-party comment frames (like Viafoura) via Playwright to capture user discourse, upvotes, and reply threads.
Detect and flag premium gated content versus free-to-read articles to maintain dataset integrity.
Scrape primary tags, secondary topics, SEO titles, and meta descriptions used for internal taxonomy.
Extract hero image URLs, inline image captions, and embedded video IDs.
Paginate through category feeds and author pages that rely on dynamic JavaScript loading.
Run hourly or daily diffs to capture newly published articles and updated timestamps on developing stories.
Deep scrape past articles by year and month to build extensive NLP training corpora.
Brief in. Clean data out.
Provide section URLs, author pages, or date ranges. We design the extraction schema together.
We configure Scrapy / Playwright crawlers, proxy rotation, session management, and CAPTCHA handling for nationalpost.com.
Schema validation, null-rate checks, ad-stripping verification, and sample datasets before full launch.
JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.
Modern news sites prioritise ad delivery and dynamic engagement over static HTML. Here is how we extract clean data.
National Post articles interleave body text with dynamic ad containers, newsletter prompts, and read-more widgets. Our parsers target specific article-body classes and filter out injected nodes, ensuring your text data is contiguous and NLP-ready.
Comments are not present in the initial HTML payload. We use Playwright to execute page scripts, wait for the comment provider frame to load, and simulate scroll events to capture full discussion threads.
Category pages and author profiles use infinite scroll rather than standard pagination. Our crawlers intercept the underlying GraphQL or REST API calls triggered by scroll events to paginate efficiently without rendering the full DOM.
News articles are frequently updated after initial publication. We hash the article content and track 'updated_at' timestamps, emitting a new record only when substantive changes occur to the headline or body text.
We detect paywall overlays and metadata flags to accurately categorise articles as premium. This prevents your dataset from being polluted with truncated article summaries.
PR agencies and corporate communications teams track brand mentions, sentiment, and narrative development across national news.
Quant funds parse Financial Post columns to gauge market sentiment on Canadian equities and macroeconomic policy.
AI researchers ingest decades of Canadian political discourse and journalistic text to train regional language models.
Think tanks and academic researchers analyse opinion columns and comment threads to map ideological shifts.
Other media organisations track publication velocity, author output, and topic coverage to benchmark editorial strategy.
Marketing teams analyse tag frequency and article volume to identify emerging cultural and economic trends.
"The National Post archive represents decades of Canadian political discourse and financial reporting — but extracting clean text from an ad-heavy DOM requires precision engineering."
News publishers rely on complex, ad-injected DOM structures and third-party scripts that break standard HTTP parsers. DataFlirt executes full browser sessions to render comment frames, bypass infinite scrolls, and extract pure article text. We handle the infrastructure so your data science teams receive clean NLP-ready text.
Everything supported by our nationalpost.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, infinite scrolls, and interaction flows for complex news layouts.
Custom middleware strips non-content nodes — ads, tracking pixels, and inline widgets — ensuring the output text is contiguous and ready for natural language processing.
Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.
Data delivered to where your team already works — no new tooling required.
About nationalpost.com scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available news articles and metadata is generally permissible. DataFlirt targets only public, non-authenticated text and metadata. We do not circumvent hard paywalls or extract personally identifiable information from user accounts. Clients should review Postmedia's ToS and consult legal counsel for specific use cases.
We extract the publicly visible metadata (headline, author, tags, publication date) and flag the article as 'paywalled'. We do not bypass hard paywalls to extract premium body text without valid credentials.
No. Our parsers use specific CSS and XPath selectors to target article body paragraphs while explicitly excluding programmatic ad containers, newsletter signups, and related-article injection modules.
Yes. We use headless browsers to execute the necessary JavaScript to load third-party comment frames, capturing the text, timestamps, and vote counts of the discussion.
Yes. The Financial Post is integrated into the nationalpost.com domain. We extract specific market categories, author data, and explicit stock ticker mentions embedded in the text.
Yes. We can configure crawlers to traverse category archives and sitemaps to extract years of historical articles, subject to availability on the site.
Pipelines can run at hourly or daily cadences. We use change detection to emit updated records when a developing story's text or timestamp changes.
Yes. By stripping ads, HTML tags, and inline widgets, we deliver contiguous strings of article text formatted specifically for ingestion into language models and sentiment classifiers.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a historical archive dump or a continuous feed of Financial Post columns — we scope, build, and operate the pipeline. Tell us what you need.
