SYSTEM all green source skillshare.com queue 12,408 URLs p99 latency 184ms dataflirt.com · scraper/skillshare-com
RUN · 41 active pipelines · skillshare.com live

Skillshare data,
at warehouse scale.

We extract class catalogues, instructor profiles, student projects, and engagement metrics from Skillshare. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Classes extracted
41.2K /run
Instructor profiles
18.5K /run
Student projects
312K /month
Active pipelines
41
Uptime
99.98%
Data Dictionary

Every field we extract from skillshare.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Classes objects from skillshare.com. All fields typed and schema-versioned.

class_idtitleurlinstructor_nameinstructor_idduration_minutesstudent_countproject_countreview_scoreskill_tagsis_originalis_staff_pickpublish_date
classes
● 200 OK
"class_id": "84729103",
"title": "Graphic Design Basics: Core Principles for Visual Design",
"instructor_name": "Ellen Lupton",
"duration_minutes": 35,
"student_count": 142851,
"is_original": true,
"is_staff_pick": true,
"skill_tags": "['Graphic Design', 'Typography', 'Creative']"
# class_idtitleurlinstructor_nameinstructor_idduration_minutes
1
2
3

Complete list of extractable fields for Instructors objects from skillshare.com. All fields typed and schema-versioned.

instructor_idnameprofile_urlheadlinebiofollower_countfollowing_countteacher_statstotal_studentsclasses_taughtsocial_links
instructors
● 200 OK
"instructor_id": "1948271",
"name": "Aaron Draplin",
"headline": "Graphic Designer, Draplin Design Co.",
"follower_count": 89412,
"total_students": 215491,
"classes_taught": 8,
"social_links": "['instagram.com/draplin', 'draplin.com']"
# instructor_idnameprofile_urlheadlinebiofollower_count
1
2
3

Complete list of extractable fields for Lessons objects from skillshare.com. All fields typed and schema-versioned.

lesson_idclass_idtitlesequence_numberduration_secondsvideo_preview_urldiscussion_countresources_included
lessons
● 200 OK
"lesson_id": "491029",
"class_id": "84729103",
"title": "Introduction to Typography",
"sequence_number": 2,
"duration_seconds": 412,
"discussion_count": 48,
"resources_included": true
# lesson_idclass_idtitlesequence_numberduration_secondsvideo_preview_url
1
2
3

Complete list of extractable fields for Student Projects objects from skillshare.com. All fields typed and schema-versioned.

project_idclass_idstudent_namestudent_idproject_titleproject_urllike_countcomment_countpublish_dateimage_urls
student_projects
● 200 OK
"project_id": "948172",
"class_id": "84729103",
"student_name": "Sarah Jenkins",
"project_title": "My First Brand Identity",
"like_count": 124,
"comment_count": 14,
"publish_date": "2024-03-12T14:22:00Z"
# project_idclass_idstudent_namestudent_idproject_titleproject_url
1
2
3

Complete list of extractable fields for Reviews objects from skillshare.com. All fields typed and schema-versioned.

review_idclass_idstudent_idrating_expectationsrating_clarityrating_actionabilityreview_texthelpful_votespost_date
reviews
● 200 OK
"review_id": "104827",
"class_id": "84729103",
"rating_expectations": "Exceeded",
"rating_clarity": "High",
"rating_actionability": "High",
"helpful_votes": 42,
"post_date": "2024-02-18T09:11:00Z"
# review_idclass_idstudent_idrating_expectationsrating_clarityrating_actionability
1
2
3

Capabilities

Everything you need from Skillshare - nothing you don't

Our Skillshare scraper navigates the entire platform taxonomy: class metadata, instructor metrics, student projects, and engagement data - with automated pagination and dynamic content hydration built in.

Full Class Catalogues

Extract class titles, duration, student counts, project counts, and required skill tags across all main categories and subcategories.

Instructor Intelligence

Capture instructor bios, follower counts, total students taught, and external social links to identify top creators.

Student Project Extraction

Scrape project galleries including project titles, like counts, comment metrics, and image URLs to gauge course engagement.

Lesson & Module Structures

Map the internal structure of classes, tracking sequence numbers, lesson durations, and resource availability.

Skillshare Originals & Staff Picks

Track platform endorsements by capturing 'Skillshare Original' and 'Staff Pick' badges across the catalogue.

Review & Expectation Metrics

Extract granular review data including clarity, actionability, and expectation ratings from verified students.

Category & Skill Taxonomy

Map the entire tag ecosystem to understand how skills are categorised and discover emerging topics.

Discussion Boards

Extract questions, answers, and general engagement metrics from class discussion tabs.

Scheduled Change Detection

Run pipelines at daily or weekly cadences to track follower growth and new class publications via diffing.

// engagement pipeline

From URL list to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Provide categories, instructor IDs, or skill tags. We design the extraction schema together.

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers, proxy rotation, session management, and CAPTCHA handling for skillshare.com.

Validation & QA
d 4–6

Schema validation, null-rate checks, metric outlier detection, and sample payloads before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

How our Skillshare pipeline handles the hard parts

Skillshare relies on dynamic React components and strict rate limits. Here is how we maintain data flow.

pipeline-monitor · skillshare.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Anti-bot layer
Residential proxy rotation

Skillshare employs edge protection that blocks aggressive datacenter IPs. Our crawlers use residential ISP proxies with realistic browser fingerprints and randomised request timing.

JavaScript rendering
React hydration via Playwright

Class modules, student projects, and dynamic metric counters require JavaScript execution. We run full Playwright browser sessions to trigger lazy-loads and hydrate the DOM before extraction.

Schema stability
Resilient selectors with fallback chains

Skillshare updates its UI components frequently. Our strategy uses multiple fallback chains per field - CSS selectors, XPath, and internal JSON state extraction - ensuring high data completeness.

Pagination handling
Deep project gallery extraction

Popular classes have thousands of student projects spread across infinite-scroll interfaces. We automate the scrolling and API interception to extract the complete gallery without memory bloat.

Monitoring & alerting
24/7 pipeline health

Every run emits structured logs to our observability stack. We alert on null-rate spikes, missing tags, and coverage drops - responding before your downstream systems are affected.

Applications

Who uses Skillshare data - and how

Teams across industries use skillshare.com data to build competitive products and smarter operations.

01
Competitor Intelligence

EdTech platforms track Skillshare's catalogue size, category growth, and instructor acquisition rates to benchmark their own offerings.

02
Creator Talent Sourcing

Agencies and competing platforms identify top-performing instructors by tracking student counts, follower growth, and project engagement.

03
Trend Analysis

Market researchers track rising skill tags and new class volume to identify trending topics in design, business, and technology.

04
Content Gap Analysis

Content strategists analyse highly searched topics with low course counts to identify underserved niches for new course creation.

05
Market Research

Investors evaluate the health of the creator economy by monitoring aggregate student enrollment and active instructor metrics.

06
AI Training Data

ML teams use structured course taxonomy and module sequencing data to train educational content generation models.

Why DataFlirt

"Skillshare represents a massive taxonomy of the modern creator economy, mapping exactly what creative professionals are teaching and learning right now."

Extracting this data requires navigating dynamic React applications, strict rate limits, and complex pagination across thousands of user-generated projects. DataFlirt manages the proxy rotation, JavaScript execution, and schema maintenance so your team receives structured, query-ready datasets without the infrastructure overhead.

Technical Spec

Skillshare scraper - technical capabilities

Everything supported by our skillshare.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Full Playwright sessions - required for class modules, project galleries, and dynamic metrics
Supported
CAPTCHA bypass
Automated solver integration for edge protection and rate-limit challenges
Supported
Residential proxy rotation
ISP-grade residential IPs rotated per request to avoid IP bans
Supported
Class taxonomy mapping
Extraction of primary categories, subcategories, and specific skill tags
Supported
Student project gallery scraping
Deep pagination of infinite-scroll project lists per class
Supported
Instructor profile extraction
Aggregation of total students, followers, and full class lists per teacher
Supported
Change detection (diffs)
Hash-based diff: only emit records with changed metrics since last run
Supported
Webhook delivery
HTTP POST per record or batch for real-time integration
Supported
Premium video file download
Extraction of raw DRM-protected video files and premium streaming assets
Partial
Private student messages
Access to authenticated user inboxes and direct messages
Partial
Infrastructure

Infrastructure powering the Skillshare pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration, deduplication, and retry logic. Playwright handles JavaScript rendering, infinite scroll, and interaction flows. Combined via scrapy-playwright middleware.

Residential Proxy Infrastructure

We maintain pools of residential ISP proxies. Rotation happens per-request with sticky sessions where required. IP score monitoring prevents blacklisted pool contamination.

Cloud-Native Orchestration

Pipelines run on AWS Lambda (burst) and ECS (sustained). Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested - schema versioned per run
CSV
Flat file with typed columns - Excel/Sheets compatible
XLS
Standard spreadsheet format for non-technical teams
Parquet
Columnar format for BigQuery, Snowflake, Athena
AWS S3
Direct bucket delivery - compatible with any data lake
Webhook
HTTP POST per record for immediate downstream processing
API
REST endpoints to query your extracted datasets
PostgreSQL
Upsert into your existing schema with conflict resolution
Snowflake
Stage + COPY INTO workflow - incremental or full-replace
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About skillshare.com scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Skillshare legal?

Scraping publicly available information from Skillshare is generally permissible under applicable law. DataFlirt targets only public, non-authenticated course metadata, instructor profiles, and project galleries. We do not extract personal data behind logins or circumvent DRM video protections. Clients should review Skillshare's ToS and consult legal counsel for specific use cases.

How do you handle rate limits and edge protection?

We use residential ISP proxies, full Playwright browser sessions with realistic fingerprints, and request timing modelled on human behaviour. This prevents IP blocks and mitigates automated security challenges.

Can you extract premium video courses?

No. We extract metadata, lesson titles, duration metrics, and public project galleries. We do not bypass DRM or download premium video content.

How fresh is the student count data?

Pipelines can be configured to run daily or weekly. The student count, follower metrics, and project counts reflect the exact numbers displayed on the platform at the time of extraction.

Can you track specific instructors over time?

Yes. We maintain a time-series table per instructor, capturing changes in follower counts, total students, and new class publications from the date your pipeline starts.

What is the minimum viable engagement?

Our smallest packages start at a defined category or list of instructors with weekly delivery. For full catalogue extraction, we price based on volume and delivery frequency.

Do you extract student projects and images?

Yes. We extract the metadata for student projects, including titles, likes, comments, and the public URLs of the uploaded project images.

$ dataflirt scope --new-project --source=skillshare.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a full catalogue dump or continuous tracking of top instructors and trending skills - we scope, build, and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Services

Data Extraction for Every Industry

View All Services →