SYSTEM all green source brilliantearth.com queue 12,408 pages p99 latency 184ms dataflirt.com · scraper/brilliantearth-com
RUN : 31 active pipelines : brilliantearth.com live

Jewellery data,
at warehouse scale.

We extract loose diamond inventories, engagement ring settings, gemstone specifications, and real-time pricing from Brilliant Earth. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Diamonds extracted
248K /run
Price updates
1.2M /24h
Ring settings
4,192 /run
Active pipelines
31
Uptime
99.98%
Data Dictionary

Every field we extract from brilliantearth.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Loose Diamonds objects from brilliantearth.com. All fields typed and schema-versioned.

skushapecaratcutcolourclaritypriceoriginlab_growncertificate_urlmeasurementsfluorescence
loose_diamonds
● 200 OK
"sku": "1489201A",
"shape": "Round",
"carat": 1.05,
"cut": "Super Ideal",
"colour": "D",
"clarity": "VVS1"
# skushapecaratcutcolourclarity
1
2
3

Complete list of extractable fields for Engagement Rings objects from brilliantearth.com. All fields typed and schema-versioned.

skunamemetalpricesetting_typeband_widthavailable_sizesimage_urlsmatching_band_sku
engagement_rings
● 200 OK
"sku": "BE1D14",
"name": "Petite Twisted Vine Diamond Ring",
"metal": "18K White Gold",
"price": 1250.0,
"setting_type": "Prong",
"band_width": "1.5mm"
# skunamemetalpricesetting_typeband_width
1
2
3

Complete list of extractable fields for Gemstones objects from brilliantearth.com. All fields typed and schema-versioned.

skugem_typeshapecaratcolourclaritydimensionspriceoriginenhancement
gemstones
● 200 OK
"sku": "GEM8921",
"gem_type": "Sapphire",
"shape": "Oval",
"carat": 2.15,
"colour": "Blue",
"price": 3400.0
# skugem_typeshapecaratcolourclarity
1
2
3

Complete list of extractable fields for Fine Jewellery objects from brilliantearth.com. All fields typed and schema-versioned.

skucategorytitlemetalpricedescriptionchain_lengthstock_statustotal_carat_weight
fine_jewellery
● 200 OK
"sku": "BE451",
"category": "Necklaces",
"title": "Diamond Bezel Pendant",
"metal": "14K Yellow Gold",
"price": 895.0,
"stock_status": "In Stock"
# skucategorytitlemetalpricedescription
1
2
3

Complete list of extractable fields for Diamond Certificates objects from brilliantearth.com. All fields typed and schema-versioned.

diamond_skureport_numberlabmeasurementspolishsymmetryfluorescenceinscriptionissue_date
diamond_certificates
● 200 OK
"diamond_sku": "1489201A",
"report_number": "GIA-2481920481",
"lab": "GIA",
"polish": "Excellent",
"symmetry": "Excellent",
"fluorescence": "None"
# diamond_skureport_numberlabmeasurementspolishsymmetry
1
2
3

Capabilities

Complete visibility into diamond and jewellery inventory

Our Brilliant Earth scraper extracts deeply nested specifications from dynamic search grids, capturing real-time pricing, 4C parameters, and high-resolution media assets without triggering bot detection.

Comprehensive 4C Extraction

Extract carat, cut, colour, and clarity for every diamond, alongside advanced metrics like table percentage, depth percentage, and culet size.

Dynamic Price Tracking

Capture real-time pricing for loose diamonds and ring settings, monitoring fluctuations driven by market changes and promotional events.

Lab Grown vs Natural Classification

Accurately segment inventory between lab-created and naturally mined diamonds, including specific origin data where provided.

Ring Setting Configurations

Scrape all metal variants, prong styles, band widths, and matching wedding band recommendations for every engagement ring SKU.

Certificate Data Parsing

Extract grading report numbers and associated lab entities like GIA or IGI directly from the diamond listing metadata.

Media Asset Aggregation

Capture URLs for high-resolution product images, 360-degree spin videos, and on-model photography.

Ethical Sourcing Metadata

Extract blockchain tracking links and specific country-of-origin tags associated with Brilliant Earth's Beyond Conflict Free commitment.

Stock and Availability

Monitor shipping timelines, stock status, and available ring sizes to track inventory velocity across specific categories.

Continuous Delta Updates

Run pipelines on a daily schedule, receiving only the records that have changed in price or availability since the last extraction.

// engagement pipeline

From target categories to warehouse delivery

Brief in. Clean data out.

Define Scope
d 0

Specify target categories like loose diamonds, engagement rings, or fine jewellery. We map the required data fields.

Pipeline Build
d 2–4

We configure Playwright to navigate dynamic search grids and intercept API responses for complete inventory capture.

Validation & QA
d 4–6

We verify standardisation of 4C metrics, validate media URLs, and ensure complete coverage of all metal variants.

Delivery
ongoing

Structured JSON, CSV, or Parquet files delivered to your S3 bucket or Snowflake environment on a scheduled cadence.

Under the hood

Overcoming dynamic grid limitations

Brilliant Earth relies on complex JavaScript applications to display thousands of diamonds. We bypass frontend limitations by interacting directly with the underlying data structures.

pipeline-monitor · brilliantearth.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
API Interception
Bypassing frontend DOM scraping

Diamond search grids load via background XHR requests. We intercept these API calls directly, extracting clean JSON payloads before they are rendered into HTML, ensuring zero data loss and faster execution.

Proxy Rotation
Residential IP pools for consistent access

High-frequency requests to inventory endpoints trigger rate limits. We distribute extraction across thousands of residential IPs, maintaining human-like request patterns to prevent IP bans.

Variant Mapping
Structuring complex product matrices

A single ring setting can have dozens of permutations based on metal type and ring size. Our schema normalises these variants into a flat, queryable structure for easy analysis.

Data Standardisation
Normalised 4C values

We clean and standardise text values for colour, clarity, and cut grades, ensuring your downstream analytics tools receive consistent categorical data without manual cleaning.

Media Resolution
Capturing highest quality assets

We parse product page metadata to locate the highest resolution image variants and raw video files, bypassing compressed thumbnail versions displayed on category pages.

Applications

How teams utilise Brilliant Earth data

Teams across industries use brilliantearth.com data to build competitive products and smarter operations.

01
Market Pricing Intelligence

Jewellery retailers monitor Brilliant Earth's pricing strategies for lab-grown versus natural diamonds to adjust their own margins.

02
Inventory Gap Analysis

Merchandising teams analyse the distribution of diamond shapes, carat weights, and colours to identify market trends and supply shortages.

03
Machine Learning Models

Data scientists train pricing algorithms using the vast matrix of 4C specifications and corresponding retail prices.

04
Competitive Benchmarking

Brands track the expansion of Brilliant Earth's fine jewellery categories and new product launches over time.

05
Supply Chain Visibility

Analysts monitor the availability of ethically sourced diamonds from specific origins to gauge global supply constraints.

06
Consumer Preference Tracking

Marketing teams correlate popular ring setting styles and metal choices to inform upcoming seasonal collections.

Why DataFlirt

"Brilliant Earth holds one of the most comprehensive digital inventories of ethically sourced and lab-grown diamonds, but extracting that data requires navigating complex dynamic grids."

Scraping fine jewellery and diamond specifications at scale requires more than basic HTTP requests. You need full JavaScript execution to render diamond search grids, residential proxies to bypass bot protection, and careful schema mapping to standardise the 4Cs across thousands of SKUs. DataFlirt manages this entire infrastructure so you can focus on market analysis.

Technical Spec

Brilliant Earth scraper: technical specifications

Everything supported by our brilliantearth.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

Loose diamond inventory
Full extraction of all available diamonds across all shapes and origins
Supported
Ring setting variants
Capture of all metal types and pricing permutations per setting
Supported
Lab-grown identification
Clear flags distinguishing lab-created from natural diamonds
Supported
High-res media URLs
Extraction of 360-degree video links and primary images
Supported
GIA/IGI certificate data
Report numbers and lab entities parsed from listing details
Supported
API interception
Direct extraction from backend JSON responses for speed and accuracy
Supported
User account wishlists
Extraction of saved items requires authenticated user sessions
Partial
Checkout flow pricing
Final pricing including applied user-specific promo codes and taxes
Partial
Infrastructure

Infrastructure powering the extraction pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Playwright Integration

We utilise headless browsers to execute JavaScript, allowing the dynamic diamond search grids to populate fully before extraction begins.

Proxy Management

Our infrastructure routes requests through extensive residential IP pools, preventing rate limiting and ensuring uninterrupted access to the catalogue.

Automated Orchestration

Pipelines are scheduled and monitored via Apache Airflow, with automated retries and alerting systems to guarantee data delivery SLAs.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Nested structures ideal for complex variant mapping
CSV
Flat files for immediate spreadsheet analysis
XLS
Excel compatible formats for business users
Parquet
Columnar storage optimised for data warehouses
AWS S3
Direct delivery to your cloud storage buckets
Webhook
Real-time HTTP POST for immediate inventory updates
API
On-demand access to extracted datasets
BigQuery
Direct streaming into Google Cloud analytics
Snowflake
Automated staging and ingestion workflows
Postgres
Direct database inserts with schema validation
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About brilliantearth.com scraping, legality, and pipeline operations.

Ask us directly →
Can you extract the entire loose diamond inventory?

Yes. We configure pipelines to paginate through the entire available inventory, capturing all shapes, carat weights, and price points across both natural and lab-grown categories.

How frequently can you update diamond pricing?

We can schedule pipelines to run daily or at specific intra-day intervals to capture pricing adjustments based on market fluctuations.

Do you capture 360-degree video assets?

We extract the direct URLs to the high-resolution media assets, including images and 360-degree spin videos, allowing you to download or reference them in your own systems.

How do you handle the different metal variants for rings?

Our schema maps every available metal type (e.g., 18K White Gold, Platinum) as a distinct variant under the parent ring SKU, capturing the specific price for each configuration.

Can I get a sample of the diamond data?

Yes. We provide a sample dataset of up to 1,000 diamonds during the scoping phase, allowing you to verify the 4C standardisation and schema structure before proceeding.

Is it possible to track inventory changes over time?

Absolutely. By running continuous pipelines, we can provide delta files that highlight new additions, removed items, and price changes, enabling historical trend analysis.

$ dataflirt scope --new-project --source=brilliantearth.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. From complete diamond inventories to specific ring setting variants, we build and manage the pipeline. Define your requirements and we deliver the data.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in jewelry

Services

Data Extraction for Every Industry

View All Services →