SYSTEM all green source smcworld.com queue 112,849 pages p99 latency 314ms dataflirt.com · scraper/smcworld-com
RUN · 41 active pipelines · smcworld.com live

SMC component data,
at warehouse scale.

We extract pneumatic specifications, configuration logic, dimensional parameters, and CAD metadata from smcworld.com. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Parts extracted
1.8M /run
Configurations
8.4M /24h
CAD metadata records
645K /run
Active pipelines
41
Uptime
99.94%
Data Dictionary

Every field we extract from smcworld.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Actuators & Cylinders objects from smcworld.com. All fields typed and schema-versioned.

part_numberseriesbore_size_mmstroke_mmaction_typefluid_typeoperating_pressureproof_pressureambient_tempauto_switch_capable
actuators_& cylinders
● 200 OK
"part_number": "CDQ2B32-50DZ",
"series": "CQ2",
"bore_size_mm": 32,
"stroke_mm": 50,
"action_type": "Double acting",
"operating_pressure": "0.05 to 1.0 MPa"
# part_numberseriesbore_size_mmstroke_mmaction_typefluid_type
1
2
3

Complete list of extractable fields for Directional Control Valves objects from smcworld.com. All fields typed and schema-versioned.

part_numberseriesvalve_typeport_sizevoltagepower_consumptionresponse_timemanual_overrideprotective_classflow_rate_cv
directional_control valves
● 200 OK
"part_number": "SY5120-5LZD-01",
"series": "SY5000",
"valve_type": "5-port solenoid",
"voltage": "24 VDC",
"port_size": "1/8",
"flow_rate_cv": 0.5
# part_numberseriesvalve_typeport_sizevoltagepower_consumption
1
2
3

Complete list of extractable fields for Air Preparation (FRL) objects from smcworld.com. All fields typed and schema-versioned.

part_numberseriescomponent_typeport_sizefiltration_rating_umset_pressure_rangebowl_materialdrain_typemax_flow_capacityweight_kg
air_preparation (frl)
● 200 OK
"part_number": "AW30-03BG-A",
"series": "AW",
"component_type": "Filter Regulator",
"port_size": "3/8",
"filtration_rating_um": 5,
"weight_kg": 0.41
# part_numberseriescomponent_typeport_sizefiltration_rating_umset_pressure_range
1
2
3

Complete list of extractable fields for Fittings & Tubing objects from smcworld.com. All fields typed and schema-versioned.

part_numberseriesconnection_typetube_od_mmthread_sizematerialfluid_compatibilityoperating_pressureoperating_tempseal_material
fittings_& tubing
● 200 OK
"part_number": "KQ2H06-01AS",
"connection_type": "Male Connector",
"tube_od_mm": 6,
"thread_size": "R1/8",
"material": "Brass/PBT",
"operating_pressure": "-100 kPa to 1 MPa"
# part_numberseriesconnection_typetube_od_mmthread_sizematerial
1
2
3

Complete list of extractable fields for Configurator Rules objects from smcworld.com. All fields typed and schema-versioned.

base_modeloption_codedescriptiondependency_ruleconflict_ruleprice_modifierweight_modifierlead_time_modifiercategoryregion
configurator_rules
● 200 OK
"base_model": "CQ2",
"option_code": "D",
"description": "Built-in magnet",
"dependency_rule": "Requires auto switch",
"price_modifier": 0.0,
"region": "Global"
# base_modeloption_codedescriptiondependency_ruleconflict_ruleprice_modifier
1
2
3

Capabilities

Industrial logic extraction at scale

Our SMC scraper handles the complex hierarchy of pneumatic catalogues: base models, configuration logic, technical specifications, and replacement parts, with JavaScript rendering for CAD and sizing tools.

Full Catalogue Extraction

Extract series, base models, technical specifications, and dimensional data across all SMC product categories.

Configurator Logic Mining

Traverse SMC part configuration logic to extract valid permutations, dependencies, and conflict rules for millions of part numbers.

CAD Metadata Capture

Extract 2D and 3D CAD file availability, supported formats, and dimensional metadata linked to specific part configurations.

Replacement Part Mapping

Map maintenance kits, seal kits, brackets, and auto switches to their compatible base actuator or valve models.

Technical Specification Parsing

Extract and type-cast operating pressure, flow rates, temperature ranges, and voltage requirements into queryable numerical fields.

Regional Catalogue Normalisation

Handle catalogue differences between SMC US, SMC EU, and SMC Japan, normalising SI and Imperial units.

Sizing Tool Data Extraction

Capture pneumatic sizing parameters and calculation logic exposed by SMC web configurators.

Cross-Reference Data

Extract competitor interchange mapping where exposed in SMC technical documentation and search tools.

Document Indexing

Capture URLs and metadata for PDF manuals, operation guides, and RoHS compliance certificates.

Scheduled Updates

Run monthly or quarterly diffs to identify new product introductions and discontinued series.

// engagement pipeline

From series list to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Provide target product categories, series, or regions. We design the extraction schema together.

Pipeline Build
d 2–4

We configure Scrapy / Playwright crawlers to handle SMC configurator logic and hierarchical navigation.

Validation & QA
d 4–6

Schema validation, permutation logic checks, and unit normalisation before full launch.

Delivery
ongoing

JSON / CSV / Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on agreed cadence.

Under the hood

How our SMC pipeline handles the hard parts

Industrial configurators are complex to scrape. Here is how we extract structured data from highly variable pneumatic catalogues.

pipeline-monitor · smcworld.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Configurator logic
Traversing combinatorial part numbers

SMC actuators and valves have millions of permutations based on bore size, stroke, port thread, and options. We write custom traversal algorithms to extract the dependency matrix, generating valid part numbers without crawling infinite loops.

JavaScript rendering
Playwright for dynamic sizing tools

SMC relies heavily on client-side JavaScript for CAD viewers and pneumatic sizing tools. We use Playwright to execute this logic, capturing data that headless HTTP clients miss.

Deep hierarchy
Navigating massive category trees

The SMC catalogue is deeply nested. Our Scrapy spiders maintain state across multiple navigation layers, ensuring we do not miss obscure variants hidden in sub-categories.

Data normalisation
Standardising technical units

Pneumatic data mixes MPa, PSI, bar, millimetres, and inches depending on the region. We parse these string values and split them into separate numerical and unit fields for clean database ingestion.

Monitoring & alerting
Pipeline health and schema drift

Industrial sites update their configurator UI infrequently, but when they do, selectors break. We monitor schema coverage and null rates, repairing selectors before data delivery.

Applications

Who uses SMC component data

Teams across industries use smcworld.com data to build competitive products and smarter operations.

01
Distributor eCommerce

Industrial distributors populate their B2B storefronts with accurate SMC part numbers, specifications, and images.

02
Procurement Automation

Enterprise procurement teams integrate valid SMC part logic into their ERPs to prevent invalid orders.

03
Digital Twin & CAD Libraries

Engineering software platforms ingest SMC dimensional metadata to populate their pneumatic component libraries.

04
Competitor Cross-Referencing

MRO distributors map SMC components against Festo or Parker equivalents based on flow rates and port sizes.

05
MRO Inventory Optimisation

Maintenance teams identify common seal kits and spare parts across their installed base of SMC actuators.

06
Automated Quoting Systems

Sales teams use extracted configurator logic to validate part numbers instantly during the RFQ process.

Why DataFlirt

"SMC manufactures over 700,000 base products with millions of permutations. Querying that industrial logic requires dedicated extraction infrastructure."

Most industrial distributors underestimate the complexity of pneumatic catalogues. Extracting SMC data requires rendering heavy configurator JavaScript, mapping complex dependency logic, and normalising technical units across regions. DataFlirt handles the extraction and normalisation, delivering ready-to-query industrial data to your warehouse.

Technical Spec

SMC scraper technical capabilities

Everything supported by our smcworld.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Playwright sessions for configurators and dynamic CAD viewers
Supported
Configurator permutation generation
Extract dependency logic to generate all valid part numbers
Supported
CAD file metadata extraction
Capture available formats and dimensional parameters
Supported
PDF manual indexing
Extract URLs for technical documentation and operation guides
Supported
Unit normalisation
Parse string fields into numerical values with standard units
Supported
Cross-reference mapping
Extract competitor interchange data where publicly available
Supported
Change detection (diffs)
Identify new product series or discontinued models
Supported
Webhook delivery
HTTP POST per batch for downstream ERP ingestion
Supported
Raw 3D CAD STEP file downloads
Requires authenticated CAD portal access and manual EULA acceptance
Partial
Distributor net pricing
Requires authenticated distributor portal credentials
Partial
Infrastructure

Infrastructure powering the SMC pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration and deduplication. Playwright handles JavaScript rendering for SMC configurators and sizing tools.

Proxy Infrastructure

We maintain proxy pools to route requests reliably without triggering rate limits on industrial manufacturer domains.

Cloud-Native Orchestration

Pipelines run on AWS Lambda and ECS. Airflow handles scheduling, dependency management, and SLA alerting. All state stored in managed Postgres.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Nested format ideal for hierarchical configurator logic
CSV
Flat file with typed columns for ERP ingestion
XLS
Excel format for procurement and sales teams
Parquet
Columnar format for BigQuery, Snowflake, Athena
AWS S3
Direct bucket delivery compatible with any data lake
Webhook
HTTP POST per record or batch
API
REST endpoint to query extracted catalogue data
PostgreSQL
Upsert into your existing schema
BigQuery
Streamed directly into your dataset
Snowflake
Stage and COPY INTO workflow
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About smcworld.com scraping, legality, and pipeline operations.

Ask us directly →
Is scraping smcworld.com legal?

Scraping publicly available technical specifications and catalogue data is generally permissible. DataFlirt extracts public product data without bypassing authentication. Clients should review terms of service and consult legal counsel for their specific use case.

How do you handle the SMC part configurator?

Instead of clicking every combination, we extract the underlying dependency matrix and option codes. We then generate the valid part number permutations programmatically, ensuring complete coverage without infinite crawling.

Can you extract actual CAD models?

We extract CAD metadata, dimensional parameters, and preview image URLs. We do not download the raw 3D STEP or IGES files if they are gated behind user registration and EULA acceptance.

Do you support regional SMC sites?

Yes. We can extract data from smcworld.com, smcusa.com, smc.eu, and other regional domains, mapping the regional variations in part numbers and specifications.

How fresh is the data?

Industrial catalogues change slowly. We typically configure SMC pipelines to run monthly or quarterly diffs to identify new product introductions, updated specifications, and discontinued series.

What is the minimum viable engagement?

Engagements typically start with a defined set of product categories or series (e.g., all pneumatic cylinders and directional control valves). Contact us to scope the exact catalogue depth required.

Can you normalise technical units?

Yes. We parse string fields like '0.1 to 1.0 MPa' into numerical minimum and maximum fields, and can convert between metric and imperial units based on your schema requirements.

$ dataflirt scope --new-project --source=smcworld.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off catalogue export or a continuous pipeline to monitor new SMC configurations, we scope, build, and operate the infrastructure. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in industrial and mro

Services

Data Extraction for Every Industry

View All Services →