SYSTEM all green source acnc.gov.au queue 12,408 entities p99 latency 318ms dataflirt.com · scraper/acnc-gov.au
RUN : 14 active pipelines : acnc.gov.au live

Australian charity data,
normalised at scale.

We extract ABN profiles, Annual Information Statements (AIS), responsible persons, and enforcement actions from the ACNC. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake.

Charities extracted
60.2K /run
AIS reports parsed
142K /month
Board members
284K /run
Active pipelines
14
Uptime
99.98%
Data Dictionary

Every field we extract from acnc.gov.au

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Charity Profiles objects from acnc.gov.au. All fields typed and schema-versioned.

abncharity_nameother_namescharity_sizeregistration_datedgr_statustax_concessionswebsiteregistered_addressoperating_status
charity_profiles
● 200 OK
"abn": "11005357522",
"charity_name": "Australian Red Cross Society",
"charity_size": "Large",
"registration_date": "2012-12-03",
"dgr_status": true,
"operating_status": "Registered",
"website": "www.redcross.org.au"
# abncharity_nameother_namescharity_sizeregistration_datedgr_status
1
2
3

Complete list of extractable fields for Financial Data (AIS) objects from acnc.gov.au. All fields typed and schema-versioned.

abnreporting_yeartotal_revenuetotal_expensesnet_surplustotal_assetstotal_liabilitiesemployee_expensesgrant_fundinggovernment_grants
financial_data (ais)
● 200 OK
"abn": "11005357522",
"reporting_year": 2023,
"total_revenue": 845000000.0,
"total_expenses": 842000000.0,
"net_surplus": 3000000.0,
"government_grants": 412000000.0,
"employee_expenses": 380000000.0
# abnreporting_yeartotal_revenuetotal_expensesnet_surplustotal_assets
1
2
3

Complete list of extractable fields for Responsible Persons objects from acnc.gov.au. All fields typed and schema-versioned.

abnperson_namepositionappointment_datecessation_daterelated_entitiesboard_roleis_active
responsible_persons
● 200 OK
"abn": "11005357522",
"person_name": "John Smith",
"position": "Board Member",
"appointment_date": "2019-11-15",
"is_active": true,
"board_role": "Director",
"related_entities": 2
# abnperson_namepositionappointment_datecessation_daterelated_entities
1
2
3

Complete list of extractable fields for Operations & Beneficiaries objects from acnc.gov.au. All fields typed and schema-versioned.

abnoperating_countriesoperating_statesmain_activitybeneficiary_typesvolunteer_countemployee_countgrant_programmes
operations_& beneficiaries
● 200 OK
"abn": "11005357522",
"main_activity": "Emergency Relief",
"operating_states": "['NSW', 'VIC', 'QLD', 'WA', 'SA', 'TAS', 'ACT', 'NT']",
"employee_count": 2450,
"volunteer_count": 15000,
"beneficiary_types": "['General community in Australia', 'Victims of disaster']"
# abnoperating_countriesoperating_statesmain_activitybeneficiary_typesvolunteer_count
1
2
3

Complete list of extractable fields for Compliance & History objects from acnc.gov.au. All fields typed and schema-versioned.

abnenforcement_actionaction_dateaction_summaryrevocation_daterevocation_reasoncompliance_statusreporting_overdue
compliance_& history
● 200 OK
"abn": "11005357522",
"compliance_status": "Up to date",
"reporting_overdue": false,
"enforcement_action": "None",
"action_date": "None",
"revocation_date": "None",
"revocation_reason": "None"
# abnenforcement_actionaction_dateaction_summaryrevocation_daterevocation_reason
1
2
3

Capabilities

Extract the complete Australian non-profit landscape

Our ACNC scraper handles WAF rate limits, complex search session states, and unstructured PDF financial reports. We convert regulatory filings into queryable databases.

Charity Register Search

Extract ABN, legal name, aliases, registration date, and charity size across the entire 60,000+ entity catalogue.

AIS Financial Extraction

Parse structured financial data from the Annual Information Statement, including revenue, expenses, assets, and liabilities.

DGR & Tax Status

Track Deductible Gift Recipient (DGR) endorsements and tax concessions linked to each registered entity.

Responsible Persons Mapping

Extract board members, trustees, and directors. Map individuals across multiple charity boards to identify network clusters.

PDF Financials OCR

Download attached financial reports and apply OCR to extract key metrics from unstructured audited statements.

Enforcement Action Alerts

Monitor changes in compliance status, overdue reporting flags, and formal enforcement actions issued by the ACNC.

Beneficiary Profiling

Categorise charities by target demographics, main activities, and operating locations down to the state or territory level.

Historical Data Tracking

Capture historical AIS filings to build longitudinal financial models of non-profit sector growth.

Scheduled Diffs

Run pipelines monthly to capture new registrations, board changes, and fresh financial filings without re-processing the entire register.

// engagement pipeline

From ACNC search parameters to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Specify target segments by charity size, state, DGR status, or provide a list of ABNs. We map the extraction schema.

Pipeline Build
d 2–4

We configure Scrapy crawlers, handle ACNC session cookies, implement rate limiting, and build PDF parsing logic.

Validation & QA
d 4–6

Schema validation, cross-checking financial totals, and null-rate monitoring before production deployment.

Delivery
ongoing

JSON, CSV, or Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on your required cadence.

Under the hood

Overcoming government registry extraction limits

Government portals prioritise stability over accessibility. Here is how we extract ACNC data reliably without triggering WAF blocks.

pipeline-monitor · acnc.gov.au · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Rate limiting
Strict concurrency controls

Government firewalls block aggressive crawling. We enforce strict per-IP concurrency limits and randomised delays, distributing requests across Australian IP pools to maintain access without degrading site performance.

Session management
Handling stateful ASP.NET forms

The ACNC advanced search relies on complex viewstates and session cookies. Our crawlers maintain persistent sessions and handle form hydration to paginate deep into search results without losing context.

Document parsing
Extracting data from audited PDFs

Many charities submit financial data as scanned PDFs rather than structured web forms. We download attachments, apply Tesseract OCR, and use heuristic parsing to extract revenue and expense lines from unstructured tables.

Schema normalisation
Standardising legacy records

Reporting requirements change over time. We map historical AIS fields to a unified modern schema, allowing you to query 2014 financial data alongside 2023 filings without writing custom transformations.

Entity resolution
Mapping board members

Names are entered inconsistently. We extract responsible persons and apply normalisation rules to help you track individuals across multiple non-profit boards accurately.

Applications

Who uses ACNC data and how

Teams across industries use acnc.gov.au data to build competitive products and smarter operations.

01
B2B Sales & Marketing

Software vendors and service providers target non-profits based on revenue size, employee count, and technology budgets.

02
Grant-Making Due Diligence

Philanthropic foundations verify compliance status, DGR endorsement, and financial health before approving grant funding.

03
Academic & Economic Research

Universities analyse sector-wide financial trends, volunteer participation rates, and the economic impact of charities.

04
Compliance & Risk Monitoring

Financial institutions ingest ACNC data for KYB (Know Your Business) processes and to monitor enforcement actions.

05
Wealth Management

Private banks identify high-net-worth individuals and influential directors serving on major charity boards.

06
Government Policy Analysis

State and federal agencies track service delivery gaps by mapping charity operating locations and beneficiary types.

Why DataFlirt

"The ACNC register holds the financial reality of Australia's non-profit sector, but accessing it systematically requires navigating strict WAFs and unstructured filings."

Extracting charity data at scale involves more than simple HTTP requests. You must handle complex pagination, parse legacy PDF financial reports, and map responsible persons across multiple entities. DataFlirt manages the extraction, normalisation, and delivery, so your analysts can focus on due diligence rather than writing custom parsers.

Technical Spec

ACNC scraper technical specifications

Everything supported by our acnc.gov.au scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

WAF bypass
AU-based proxy rotation with conservative concurrency limits
Supported
AIS extraction
Structured financial data from web forms
Supported
PDF OCR parsing
Tesseract integration for scanned financial reports
Supported
Historical AIS data
Access to filings from previous reporting years
Supported
Change detection (diffs)
Hash-based diff: only emit records with changed fields
Supported
Webhook delivery
HTTP POST per record for real-time compliance alerts
Supported
Unredacted personal addresses
Residential addresses of board members are withheld by the ACNC
Partial
Confidential charity submissions
Certain financial reports are granted privacy exemptions by the regulator
Partial
Infrastructure

Infrastructure powering the ACNC pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheusTesseract OCRPyPDF2
Scrapy + Playwright Stack

Scrapy handles crawl orchestration and deduplication. Playwright manages complex ASP.NET session states and viewstate hydration required for deep search pagination.

OCR & Document Parsing

Pipelines integrate Tesseract OCR and PyPDF2 to download, parse, and extract structured tables from scanned annual reports attached to charity profiles.

Cloud-Native Orchestration

Pipelines run on AWS ECS. Airflow handles monthly scheduling for AIS updates. All state and historical diffs are stored in managed Postgres.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested array format
CSV
Flat file with typed columns for spreadsheet analysis
Parquet
Columnar format for BigQuery, Snowflake, Athena
S3
Direct bucket delivery
Webhook
HTTP POST per record for compliance alerts
API
REST endpoint to query extracted records
BigQuery
Streamed directly into your dataset
PostgreSQL
Upsert into your existing schema
// faq

Common questions.

About acnc.gov.au scraping, legality, and pipeline operations.

Ask us directly →
Is scraping the ACNC legal?

Yes. The ACNC Charity Register is a public database designed for transparency. DataFlirt extracts only publicly available information. We adhere strictly to rate limits to ensure we do not disrupt government infrastructure.

How often is the data updated?

Charities submit their Annual Information Statements at different times depending on their financial year. We typically run ACNC pipelines on a monthly cadence to capture new registrations, board changes, and recent AIS filings.

Can you extract data from the PDF financial reports?

Yes. While basic financial metrics are available in the web-based AIS, detailed audits are often uploaded as PDFs. We download these attachments and use OCR to extract specific line items, though accuracy depends on the quality of the scanned document.

Do you track historical changes to board members?

We capture the 'appointment date' and 'cessation date' for responsible persons as listed on the registry. By running periodic pipelines, we build a timeline of board composition changes.

How do you handle ABN validation?

We extract the ABN directly from the ACNC profile. If required, we can cross-reference this with the Australian Business Register (ABR) to append additional corporate structure data.

What is the minimum viable engagement?

We support full-register extractions (approx 60,000 entities) or targeted subsets based on specific criteria (e.g., all charities in Victoria with revenue over $1M). Contact us with your parameters for a scoped quote.

$ dataflirt scope --new-project --source=acnc.gov.au ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off database of all registered charities or a monthly feed of financial filings, we build and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in government and public data

Services

Data Extraction for Every Industry

View All Services →