We extract articles, author profiles, breaking news feeds, and comment sections from NZ Herald. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for Articles objects from nzherald.co.nz. All fields typed and schema-versioned.
"url": "https://www.nzherald.co.nz/business/reserve-bank-ocr-decision/", "headline": "Reserve Bank holds Official Cash Rate at 5.5%", "author": "Liam Dann", "publish_date": "2024-02-28T14:00:00Z", "section": "Business", "is_premium": true, "tags": "['Economy', 'Reserve Bank', 'Interest Rates']"
| # | url | headline | subheadline | author | publish_date | update_date |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Authors objects from nzherald.co.nz. All fields typed and schema-versioned.
"name": "Liam Dann", "role": "Business Editor at Large", "twitter_handle": "@liamdann", "article_count": 1452, "latest_article_url": "https://www.nzherald.co.nz/business/reserve-bank-ocr-decision/", "profile_image_url": "https://www.nzherald.co.nz/resizer/liam-dann.jpg"
| # | author_id | name | role | twitter_handle | bio | article_count |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Comments objects from nzherald.co.nz. All fields typed and schema-versioned.
"comment_id": "c_982341", "article_url": "https://www.nzherald.co.nz/business/reserve-bank-ocr-decision/", "user_name": "KiwiObserver", "timestamp": "2024-02-28T14:45:12Z", "upvotes": 42, "replies_count": 3, "comment_text": "Expected outcome given the latest inflation data."
| # | comment_id | article_url | user_name | timestamp | comment_text | upvotes |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Sections objects from nzherald.co.nz. All fields typed and schema-versioned.
"section_name": "Politics", "url": "https://www.nzherald.co.nz/nz/politics/", "top_story_url": "https://www.nzherald.co.nz/nz/politics/parliament-urgency/", "trending_articles": 5, "last_updated": "2024-02-28T15:10:00Z", "layout_type": "grid"
| # | section_name | url | top_story_url | trending_articles | total_articles | last_updated |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Media objects from nzherald.co.nz. All fields typed and schema-versioned.
"image_id": "img_77392", "article_url": "https://www.nzherald.co.nz/business/reserve-bank-ocr-decision/", "image_url": "https://www.nzherald.co.nz/resizer/rbnz-building.jpg", "caption": "The Reserve Bank building in Wellington.", "credit": "Mark Mitchell", "format": "jpeg"
| # | image_id | article_url | image_url | caption | credit | width |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Our NZ Herald scraper handles the entire newsroom platform: breaking news feeds, deep article archives, author profiles, and comment sections — with JavaScript rendering and session management built in.
Headline, body text, subheadlines, publish dates, and tags — scraped accurately regardless of article template variations.
Track author output, extract bios, capture social links, and monitor specific journalists across all sections.
High-frequency polling on section feeds to capture breaking news URLs and headline updates within minutes.
Extract user sentiment, upvote counts, and nested reply threads from heavily JavaScript-rendered comment sections.
Accurately flag NZ Herald Premium content and extract available free-tier metadata without failing the pipeline.
Extract high-resolution image URLs, captions, and photographer credits associated with every article.
Map articles to their primary and secondary sections (Business, Politics, Sport, Lifestyle) for precise categorisation.
Traverse deep pagination and sitemaps to build comprehensive historical datasets spanning years of publication.
Capture OpenGraph tags, meta descriptions, and hidden keywords used by editorial teams for search visibility.
Configure Webhook delivery for immediate push notifications the moment a new article hits the target section.
Brief in. Clean data out.
Provide target sections, author lists, or historical date ranges. We design the extraction schema together.
We configure Scrapy / Playwright crawlers, handle caching layers, and implement change detection for updates.
Schema validation, null-rate checks, and text-encoding verification before full launch.
JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.
News sites employ aggressive caching and anti-bot layers to protect content. Here's how we stay resilient.
Media sites use edge protection to block datacenter IPs. Our crawlers route through residential and mobile ISP proxies to maintain uninterrupted access to article feeds.
NZ Herald Premium alters the DOM structure significantly. Our selectors detect paywall flags instantly, capturing complete metadata and partial text without generating errors.
Comment sections load asynchronously via JavaScript. We use full browser rendering to trigger lazy loads and expand nested threads before extraction.
News articles are frequently updated post-publication. We maintain hash indexes of article bodies to detect and emit diffs when stories evolve.
Interactive features, video embeds, and long-form layouts break standard parsers. We use fallback chains to extract core text regardless of the presentation layer.
Agencies monitor brand mentions, executive quotes, and crisis coverage across national news in real time.
AI teams ingest structured article text to train models on New Zealand-specific vernacular, geography, and context.
Think tanks and researchers analyse comment sections and editorial tone to gauge public reaction to policy announcements.
Publishers track publishing velocity, author output, and section engagement to benchmark their own newsroom performance.
Universities compile longitudinal datasets of news coverage to study media bias, topic prominence, and cultural shifts.
Quant funds parse the Business section for immediate signals on RBNZ announcements, corporate earnings, and macroeconomic data.
"NZ Herald holds the definitive daily record of New Zealand's political and business landscape — but extracting it at scale requires a resilient pipeline."
Most teams underestimate the investment required: reliable news scraping requires handling hard paywalls, dynamic comment loading, continuous breaking news polling, and aggressive caching layers. DataFlirt absorbs that complexity so your engineers can focus on the analysis — not the infrastructure.
Everything supported by our nzherald.co.nz scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, cookie sessions, and interaction flows. Combined via scrapy-playwright middleware.
We maintain pools of residential ISP proxies. Rotation happens per-request with sticky sessions where required. IP score monitoring prevents blacklisted pool contamination.
Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.
Data delivered to where your team already works — no new tooling required.
About nzherald.co.nz scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available news articles and metadata is generally permissible. DataFlirt targets only public, non-authenticated content. We do not bypass paywalls to steal premium content, nor do we extract private user data. Clients are responsible for ensuring their use of the data complies with copyright law and fair use provisions.
Our scrapers detect the paywall flag in the DOM. For premium articles, we extract all available public metadata (headline, author, publish date, tags, and visible teaser text) and mark the record with an `is_premium: true` boolean. We do not bypass the paywall to extract hidden body text.
For time-sensitive monitoring, we configure pipelines to poll specific section feeds (e.g., National, Business) at high frequencies. New URLs are typically detected, extracted, and delivered via Webhook within 2-5 minutes of publication.
Yes. We use sitemap traversal and deep pagination to extract articles dating back years. Historical backfills are executed as one-off bulk runs before continuous daily polling begins.
Yes. We use headless browsers to render the comment engine, expand nested reply threads, and extract usernames, timestamps, comment text, and upvote/downvote metrics.
Our minimum engagement typically starts at a defined daily extraction volume or a specific historical backfill requirement. Contact us with your target sections and update frequency for a scoped quote.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a historical corpus for NLP training or a real-time feed for media monitoring — we scope, build, and operate the pipeline. Tell us what you need.
