SYSTEM all green source meinestadt.de queue 14,892 pages p99 latency 312ms dataflirt.com · scraper/meinestadt-de
RUN · 18 active pipelines · meinestadt.de live

German local data,
at warehouse scale.

We extract business directories, regional job postings, real estate listings, and local events from meinestadt.de. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Businesses extracted
4.2M /month
Job postings
380K /week
Real estate listings
215K /run
Active pipelines
18
Uptime
99.94%
Data Dictionary

Every field we extract from meinestadt.de

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Business Directory objects from meinestadt.de. All fields typed and schema-versioned.

business_idnamecategorysub_categorystreet_addresscityzip_codephone_numberwebsite_urlemail_addressopening_hourslatitudelongituderatingreview_count
business_directory
● 200 OK
"business_id": "MS-BIZ-847291",
"name": "Müller & Co. Sanitärtechnik",
"category": "Handwerk",
"city": "Köln",
"zip_code": "50667",
"phone_number": "+49 221 1234567",
"website_url": "https://www.mueller-sanitaer-koeln.de",
"rating": 4.6
# business_idnamecategorysub_categorystreet_addresscity
1
2
3

Complete list of extractable fields for Job Postings objects from meinestadt.de. All fields typed and schema-versioned.

job_idtitlecompany_namelocationzip_codeemployment_typeposted_datedescription_textrequirementscontact_emailapplication_urlsalary_indication
job_postings
● 200 OK
"job_id": "MS-JOB-59281",
"title": "Anlagenmechaniker SHK (m/w/d)",
"company_name": "Müller & Co. Sanitärtechnik",
"location": "Köln",
"employment_type": "Vollzeit",
"posted_date": "2023-10-14T08:00:00Z",
"application_url": "https://jobs.mueller-sanitaer-koeln.de/apply"
# job_idtitlecompany_namelocationzip_codeemployment_type
1
2
3

Complete list of extractable fields for Real Estate objects from meinestadt.de. All fields typed and schema-versioned.

property_idlisting_typetitlestreetcityzip_codeprice_eurroomsarea_sqmbroker_namebroker_phonedescriptionenergy_rating
real_estate
● 200 OK
"property_id": "MS-RE-10482",
"listing_type": "Miete",
"title": "Helle 3-Zimmer Wohnung mit Balkon",
"city": "München",
"price_eur": 1450.0,
"rooms": 3,
"area_sqm": 85.5,
"broker_name": "Bavaria Immobilien GmbH"
# property_idlisting_typetitlestreetcityzip_code
1
2
3

Complete list of extractable fields for Local Events objects from meinestadt.de. All fields typed and schema-versioned.

event_idtitlecategoryvenue_namestreet_addresscityevent_datestart_timeticket_price_eurdescriptionticket_urlorganizer_name
local_events
● 200 OK
"event_id": "MS-EV-99312",
"title": "Weihnachtsmarkt am Dom",
"category": "Märkte & Feste",
"city": "Köln",
"event_date": "2023-12-01",
"start_time": "11:00",
"ticket_price_eur": 0.0,
"organizer_name": "Stadt Köln"
# event_idtitlecategoryvenue_namestreet_addresscity
1
2
3

Complete list of extractable fields for Vehicles objects from meinestadt.de. All fields typed and schema-versioned.

listing_idvehicle_makevehicle_modelprice_eurmileage_kmregistration_yearfuel_typetransmissiondealer_namedealer_citycontact_phonelisting_url
vehicles
● 200 OK
"listing_id": "MS-CAR-48291",
"vehicle_make": "Volkswagen",
"vehicle_model": "Golf VIII 1.5 TSI",
"price_eur": 24500.0,
"mileage_km": 15000,
"registration_year": 2021,
"fuel_type": "Benzin",
"dealer_city": "Berlin"
# listing_idvehicle_makevehicle_modelprice_eurmileage_kmregistration_year
1
2
3

Capabilities

Extract the complete German local data graph

Meinestadt.de aggregates multiple distinct verticals under one domain. We build custom extraction schemas for their Branchenbuch, Stellenmarkt, and Immobilien sections, normalising the output into a unified warehouse format.

Branchenbuch Extraction

Extract comprehensive local business listings across all German municipalities, including addresses, contact details, and categories.

Stellenmarkt Data

Capture regional job postings, apprenticeships, and employment opportunities with full descriptions and application links.

Immobilien Listings

Scrape residential and commercial real estate data, including pricing, square footage, energy ratings, and broker details.

Veranstaltungen & Events

Aggregate local event calendars, ticket prices, venue information, and scheduling data across thousands of cities.

Geo-spatial Mapping

Extract and normalise latitude and longitude coordinates for precise mapping of businesses, properties, and venues.

Dynamic Contact Parsing

Execute JavaScript to unmask protected phone numbers and email addresses hidden behind interactive elements.

Taxonomy Traversal

Navigate complex regional hierarchies from federal states (Bundesländer) down to specific postal codes and districts.

Regional Proxy Routing

Utilise German residential IPs to bypass geo-blocking and rate limits specific to DACH-region traffic.

Scheduled Delta Updates

Maintain a hash index of previously scraped records to deliver only new or modified listings on subsequent runs.

// engagement pipeline

From regional targeting to structured data

Brief in. Clean data out.

Define Scope
d 0

Specify target cities, postal codes, and data verticals (jobs, real estate, businesses).

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers and DE-specific residential proxy pools.

Validation & QA
d 4–6

Schema validation, null-rate checks, and contact detail verification before launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery, or Snowflake stage.

Under the hood

Overcoming Meinestadt.de extraction challenges

Extracting data from multi-vertical directory sites requires distinct handling for different page structures and strict regional compliance.

pipeline-monitor · meinestadt.de · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Regional IPs
DE-specific residential proxy rotation

Meinestadt.de employs strict rate limiting and geo-blocking for non-DACH traffic. We route all requests through high-quality German residential proxies to maintain access and prevent IP bans.

Dynamic content
JavaScript execution for contact details

Critical data points like phone numbers and email addresses are often masked or loaded asynchronously. We use Playwright to execute page scripts and trigger the necessary DOM events to reveal full contact information.

Schema variance
Normalising multi-vertical DOM structures

The DOM structure for a job posting differs entirely from a real estate listing or a business profile. We maintain separate, vertical-specific parsing logic that maps disparate HTML layouts into a unified, predictable JSON schema.

Pagination depth
Deep traversal of regional taxonomies

Local data is deeply nested within city and district categories. Our crawlers systematically traverse these hierarchical pagination structures to ensure comprehensive data capture without missing nested records.

Data freshness
Delta extraction for volatile listings

Job postings and real estate listings expire quickly. We implement change-detection hashing to identify new, updated, or removed listings, delivering precise delta feeds rather than redundant full dumps.

Applications

Who uses Meinestadt.de data — and how

Teams across industries use meinestadt.de data to build competitive products and smarter operations.

01
Local SEO & Citation Building

Marketing agencies aggregate business directories to monitor local search presence, verify NAP (Name, Address, Phone) consistency, and build citations.

02
B2B Lead Generation

Sales teams extract newly registered businesses and contact details to build targeted regional outreach lists.

03
Real Estate Market Analysis

PropTech firms analyse listing prices, square footage, and regional availability to model property valuation trends.

04
Regional Labor Market Intelligence

HR consultancies and economists track job posting volume, required skills, and salary indications across different German municipalities.

05
Event Aggregation

Media companies and tourism boards compile local event data to populate regional calendars and recommendation engines.

06
Competitor Mapping

Retailers map the geographic distribution of competing businesses to identify underserved areas for expansion.

Why DataFlirt

"Meinestadt.de holds the definitive graph of German local commerce and regional employment — but extracting it requires navigating complex taxonomies and strict regional rate limits."

Most teams fail at German directory scraping because they rely on data centre IPs. We deploy DE-specific residential proxy pools, handle dynamic contact detail rendering via Playwright, and normalise disparate schema formats across jobs, real estate, and business verticals. We manage the infrastructure so you receive clean, queryable records.

Technical Spec

Meinestadt.de scraper — technical capabilities

Everything supported by our meinestadt.de scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

DE Residential Proxies
Traffic routed through German ISP IPs to bypass regional blocks
Supported
Phone number unmasking
Automated interaction to reveal hidden contact details
Supported
Cross-vertical schema normalisation
Unified output format across jobs, real estate, and businesses
Supported
Geo-coordinate extraction
Capture of latitude and longitude for mapping applications
Supported
Change detection (diffs)
Emit only new or modified records since the last extraction run
Supported
Webhook delivery
HTTP POST per record or batch for real-time integration
Supported
Job application form bypassing
Automated submission of candidate data through internal forms
Partial
Private user message extraction
Access to direct messages between users and brokers/employers
Partial
Real estate broker backend data
Extraction of non-public listing metrics and inquiry volumes
Partial
Infrastructure

Infrastructure powering the Meinestadt.de pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration and deduplication. Playwright handles JavaScript execution to unmask contact details and render dynamic map elements.

DE-Targeted Proxy Infrastructure

We maintain dedicated pools of German residential proxies to ensure high success rates and prevent geo-blocking during large-scale regional extraction.

Cloud-Native Orchestration

Pipelines run on containerised infrastructure. Airflow handles scheduling across different verticals, ensuring timely delivery of delta updates.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested — schema versioned per run
CSV
Flat file with typed columns — Excel/Sheets compatible
XLS
Formatted spreadsheet for non-technical stakeholders
Parquet
Columnar format for BigQuery, Snowflake, Athena
AWS S3
Direct bucket delivery — compatible with any data lake
Webhook
HTTP POST per record for real-time downstream processing
API
REST endpoints to query extracted historical data
BigQuery
Streamed directly into your dataset with schema auto-detect
Snowflake
Stage + COPY INTO workflow — incremental or full-replace
Postgres
Upsert into your existing schema with conflict resolution
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About meinestadt.de scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Meinestadt.de compliant with German regulations?

We extract only publicly available business, job, and real estate data. We do not extract personal user data or circumvent authentication mechanisms. Clients must ensure their use of the data complies with GDPR and local regulations, particularly regarding unsolicited B2B communication.

How do you handle masked phone numbers and emails?

We utilise headless browsers (Playwright) to simulate user interaction, clicking 'reveal' buttons and executing the underlying JavaScript required to expose the full contact details before extraction.

Can I target specific cities or postal codes?

Yes. We can configure pipelines to target specific municipalities, federal states (Bundesländer), or a defined list of postal codes to match your exact regional requirements.

How fresh is the job and real estate data?

We configure pipelines to run at your required cadence—daily or weekly. Change-detection logic ensures that expired listings are flagged and new listings are captured promptly.

Do you support all verticals on Meinestadt.de?

Yes. Our schemas cover the Branchenbuch (business directory), Stellenmarkt (jobs), Immobilien (real estate), Veranstaltungen (events), and Auto (vehicles) sections.

Why do I need residential proxies for a directory site?

German directories aggressively block traffic originating from known data centre IP ranges (like AWS or DigitalOcean) to prevent scraping. Residential IPs routed through German ISPs are necessary to maintain reliable access.

$ dataflirt scope --new-project --source=meinestadt.de ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a full export of the Branchenbuch or daily delta feeds of regional job postings — we scope, build, and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in business directories

Services

Data Extraction for Every Industry

View All Services →