We extract university profiles, undergraduate course catalogues, WUSCA rankings, and verified student reviews from Whatuni. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.
Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.
Complete list of extractable fields for University Profiles objects from whatuni.com. All fields typed and schema-versioned.
"university_id": "U8472", "name": "University of Manchester", "location": "Manchester", "wusca_ranking": 12, "overall_rating": 4.2, "review_count": 3412, "accommodation_rating": 3.9, "facilities_rating": 4.5
| # | university_id | name | location | region | institution_type | wusca_ranking |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Course Catalogues objects from whatuni.com. All fields typed and schema-versioned.
"course_id": "C91823", "title": "BSc Computer Science", "study_mode": "Full-time", "duration": "3 Years", "qualification": "BSc (Hons)", "tuition_fee_uk": 9250, "tuition_fee_intl": 25500, "placement_year": true
| # | course_id | university_id | title | study_mode | duration | qualification |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Student Reviews objects from whatuni.com. All fields typed and schema-versioned.
"review_id": "R99281", "reviewer_status": "Current Student", "year_of_study": "Year 2", "overall_rating": 5, "course_lecturers_rating": 4, "review_text": "The lecturers are highly engaging and the campus facilities are excellent.", "date_posted": "2023-11-14", "verified_status": true
| # | review_id | university_id | course_id | reviewer_status | year_of_study | overall_rating |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Entry Requirements objects from whatuni.com. All fields typed and schema-versioned.
"course_id": "C91823", "a_level_grades": "AAA-AAB", "btec_grades": "DDD", "ib_points": 36, "gcse_requirements": "Minimum 5 GCSEs at grade 6/B including Maths and English.", "interview_required": false, "portfolio_required": false
| # | course_id | a_level_grades | btec_grades | ib_points | scottish_highers | gcse_requirements |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Complete list of extractable fields for Open Days objects from whatuni.com. All fields typed and schema-versioned.
"event_id": "EV4412", "event_type": "Undergraduate Open Day", "date": "2024-06-15", "start_time": "09:00", "end_time": "16:00", "virtual_event": false, "location": "Main Campus", "booking_url": "https://www.whatuni.com/open-days/book/4412"
| # | event_id | university_id | event_type | date | start_time | end_time |
|---|---|---|---|---|---|---|
| 1 | ||||||
| 2 | ||||||
| 3 |
Our Whatuni scraper handles university profiles, dynamic course searches, UCAS requirements, and deep review pagination. We manage the JavaScript rendering and proxy rotation required to extract accurate data at scale.
Extract module structures, study modes, durations, and placement year availability across all undergraduate and postgraduate listings.
Capture demographic data, student population statistics, campus facilities, and Whatuni Student Choice Awards (WUSCA) rankings.
Paginate through thousands of verified student reviews. Capture granular ratings for job prospects, student union, and accommodation.
Parse A-level, BTEC, and IB point requirements alongside specific GCSE prerequisites and portfolio mandates.
Track upcoming physical and virtual open days, capturing dates, times, and registration links per university.
Extract domestic (UK) and international tuition fees for every course, monitoring year-on-year changes.
Scrape student hall details, weekly rent costs, location proximity to campus, and student accommodation ratings.
Run continuous pipelines that diff course updates, fee changes, and new reviews against previous runs to output clean changelogs.
Automate complex search filters and JavaScript-rendered result pages to ensure zero data omission across the catalogue.
Brief in. Clean data out.
Provide specific universities, course categories, or regions. We design the extraction schema together.
We configure Scrapy and Playwright crawlers, proxy rotation, and session management for whatuni.com.
Schema validation, null-rate checks, and data normalisation rules are applied before full launch.
JSON, CSV, or Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.
Extracting comprehensive course data requires navigating dynamic filters and unstructured text. Here is how we ensure data integrity.
Whatuni relies on client-side rendering for complex course filtering. We use Playwright to execute JavaScript, interact with dropdowns, and hydrate the DOM before extraction.
Universities can have thousands of reviews spread across hundreds of pages. Our crawlers manage state and session cookies to reliably paginate through the entire historical review corpus.
We utilise UK-based residential proxies and spoofed TLS fingerprints to mimic standard student browsing behaviour, preventing IP blocks and CAPTCHA loops.
Entry requirements are often presented in free-text format. We apply post-processing rules to extract structured A-level grades and UCAS points from unstructured descriptions.
Instead of re-delivering the entire 60,000+ course catalogue, our change detection engine only emits records where fees, requirements, or modules have changed.
Universities track tuition fees, entry requirements, and WUSCA rankings against peer institutions to adjust market positioning.
Course discovery platforms syndicate Whatuni course catalogues and module data to enrich their own search directories.
Analysts aggregate student review sentiment to identify trends in student satisfaction, accommodation quality, and teaching standards.
Student housing providers analyse university population sizes and accommodation ratings to target new development locations.
Researchers track changes in international tuition fees and UCAS entry thresholds over time to evaluate higher education accessibility.
Advisory services integrate structured course requirements into matching algorithms to help students find suitable degree programmes.
"Whatuni holds the most comprehensive dataset of UK undergraduate courses and student sentiment, but extracting it requires navigating complex search states."
Extracting higher education data from Whatuni requires managing dynamic search filters, pagination across thousands of courses, and parsing unstructured review text. DataFlirt handles the proxy rotation, JavaScript rendering, and schema normalisation so your data engineering team receives clean, queryable records.
Everything supported by our whatuni.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.
Open-source tooling on proven cloud infra — no vendor lock-in, full observability.
Scrapy handles crawl orchestration and deduplication. Playwright handles JavaScript rendering, cookie sessions, and interaction flows for complex course searches.
We maintain pools of UK residential ISP proxies. Rotation happens per-request with sticky sessions where required to maintain search state.
Pipelines run on AWS Lambda and ECS. Airflow handles scheduling, dependency management, and SLA alerting. All state is stored in managed Postgres.
Data delivered to where your team already works — no new tooling required.
About whatuni.com scraping, legality, and pipeline operations.
Ask us directly →Scraping publicly available information from Whatuni is generally permissible. DataFlirt targets only public, non-authenticated course listings, university profiles, and reviews. We do not extract personal user data or circumvent authentication walls.
We utilize Playwright to execute JavaScript and interact with the site's search filters programmatically, ensuring all paginated results are rendered and captured accurately.
Course catalogues typically require weekly or monthly refreshes. Review data can be streamed daily. We configure the pipeline cadence based on your specific monitoring requirements.
Yes. We paginate through the entire review history for any given university or course, capturing the date posted, verified status, and granular ratings.
Yes. Every pipeline run produces timestamped snapshots. We maintain a time-series record for tuition fees and UCAS entry requirements from the date your pipeline starts.
Our minimum engagement typically starts with a defined subset of universities or specific degree categories with monthly delivery. Contact us for a scoped quote based on volume.
Yes. We provide a sample run of up to 50 university profiles or 500 courses during the scoping process, allowing you to validate schema fit and data completeness.
20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off course catalogue export or continuous monitoring of university reviews and fees, we build and operate the pipeline. Tell us what you need.
