SYSTEM all green source bluemercury.com queue 14,392 pages p99 latency 184ms dataflirt.com · scraper/bluemercury-com
RUN · 42 active pipelines · bluemercury.com live

Luxury beauty data,
at warehouse scale.

We extract product formulations, pricing signals, brand intelligence, reviews, and stock depth from Bluemercury. Delivered as clean JSON, CSV, or Parquet to S3, BigQuery, or Snowflake on your cadence.

Products extracted
84K /run
Price updates
120K /24h
Review records
312K /run
Active pipelines
42
Uptime
99.94%
Data Dictionary

Every field we extract from bluemercury.com

Structured, schema-consistent data across all major object types — delivered clean, typed, and ready to query.

Complete list of extractable fields for Product Listings objects from bluemercury.com. All fields typed and schema-versioned.

skutitlebrandcategorysub_categorypricecurrencysize_volumeshade_nameshade_hexis_exclusiveconscious_beautydescriptionhow_to_useimage_urlsurl
product_listings
● 200 OK
"sku": "BM-81723902",
"title": "Crème de la Mer",
"brand": "La Mer",
"price": 380.0,
"currency": "USD",
"size_volume": "2 oz",
"is_exclusive": false,
"conscious_beauty": false
# skutitlebrandcategorysub_categoryprice
1
2
3

Complete list of extractable fields for Pricing & Inventory objects from bluemercury.com. All fields typed and schema-versioned.

skupricelist_pricediscount_pctin_stockstock_levelpromotional_tagsgift_with_purchaseprice_per_ouncecurrencyscraped_at
pricing_& inventory
● 200 OK
"sku": "BM-81723902",
"price": 380.0,
"list_price": 380.0,
"in_stock": true,
"promotional_tags": "['Free Gift with $150 La Mer Purchase']",
"gift_with_purchase": true,
"scraped_at": "2026-05-12T10:15:00Z"
# skupricelist_pricediscount_pctin_stockstock_level
1
2
3

Complete list of extractable fields for Ingredients & Formulations objects from bluemercury.com. All fields typed and schema-versioned.

skubrandkey_ingredientsfull_ingredient_listparaben_freesulfate_freephthalate_freevegancruelty_freefragrance_free
ingredients_& formulations
● 200 OK
"sku": "BM-9921034",
"brand": "M-61",
"key_ingredients": "['Glycolic Acid', 'Salicylic Acid', 'Chamomile']",
"full_ingredient_list": "Water, Glycolic Acid, Sodium Hydroxide...",
"paraben_free": true,
"vegan": true,
"cruelty_free": true
# skubrandkey_ingredientsfull_ingredient_listparaben_freesulfate_free
1
2
3

Complete list of extractable fields for Reviews & Ratings objects from bluemercury.com. All fields typed and schema-versioned.

review_idskureviewer_nicknameratingreview_titlereview_textdate_postedverified_buyerrecommendedhelpful_votes
reviews_& ratings
● 200 OK
"review_id": "REV-998231",
"sku": "BM-81723902",
"rating": 5,
"review_title": "Worth every penny",
"review_text": "This moisturiser completely changed my skin texture.",
"verified_buyer": true,
"recommended": true
# review_idskureviewer_nicknameratingreview_titlereview_text
1
2
3

Complete list of extractable fields for Brands & Categories objects from bluemercury.com. All fields typed and schema-versioned.

brand_idbrand_namebrand_descriptiontotal_productsbestseller_skusnew_arrival_skusbrand_urlis_founder_led
brands_& categories
● 200 OK
"brand_id": "BR-092",
"brand_name": "Oribe",
"total_products": 142,
"bestseller_skus": "['BM-1123', 'BM-1124', 'BM-1125']",
"brand_url": "https://bluemercury.com/collections/oribe",
"is_founder_led": false
# brand_idbrand_namebrand_descriptiontotal_productsbestseller_skusnew_arrival_skus
1
2
3

Capabilities

Everything you need from Bluemercury, nothing you don't

Our Bluemercury pipeline processes the entire beauty catalogue: complex shade variants, unstructured ingredient lists, luxury pricing rules, and dynamic inventory states.

Variant & Shade Extraction

Map parent products to dozens of child shade variants, capturing specific hex codes, colour names, and individual stock status for cosmetics.

Ingredient Parsing

Extract and normalise key ingredients, full formulation lists, and conscious beauty tags like vegan, paraben-free, and cruelty-free.

Luxury Pricing & Promos

Track base prices, price-per-ounce metrics, and complex promotional rules like 'Gift with Purchase' thresholds.

Inventory Monitoring

Monitor stock availability across the entire catalogue to detect out-of-stock items, discontinued lines, and new arrivals.

Review & Sentiment Data

Pull complete review text, star ratings, and verified buyer flags to analyse consumer sentiment on high-ticket skincare.

Brand Intelligence

Audit brand assortments, track exclusive lines like M-61 and Lune+Aster, and monitor category share.

Store & Spa Locations

Extract physical store details, spa service menus, and local event schedules from the Bluemercury store locator.

Automated Change Detection

Receive only the data that changed since the last run. Perfect for high-frequency price and stock monitoring.

Anti-Bot Circumvention

Bypass Cloudflare and custom bot protection using advanced residential proxy rotation and browser fingerprinting.

// engagement pipeline

From brand list to warehouse record

Brief in. Clean data out.

Define Scope
d 0

Provide target brands, categories, or specific SKUs. We design the extraction schema tailored to your cosmetic data needs.

Pipeline Build
d 2–4

We configure crawlers to handle Bluemercury's frontend, managing JavaScript execution, proxy rotation, and variant mapping.

Validation & QA
d 4–6

Schema validation, null-rate checks, and ingredient list normalisation before full production launch.

Delivery
ongoing

JSON, CSV, or Parquet pushed to your S3 bucket, BigQuery dataset, or Snowflake stage on an agreed schedule.

Under the hood

How our Bluemercury pipeline handles the hard parts

Extracting luxury beauty data requires handling complex product hierarchies and strict bot defenses. Here is how we build for reliability.

pipeline-monitor · bluemercury.com · live ● active
// fingerprinting
Identity rotation
TLS fingerprintrandomised
User-agentrotated
IP poolresidential
Challenges blocked0
// pagination
Page coverage
48,291 pages queued running
// observability
Pipeline health
99.9%
uptime
142ms
p99 lat
0.3%
null rate
2
alerts
Bot Protection
Bypassing edge security

Retailers protect their catalogues with strict edge security. Our infrastructure uses residential US proxies and mimics genuine browser TLS fingerprints to ensure uninterrupted data collection.

Data Structure
Taming shade variants

Cosmetics often have 40+ shades per product, each with unique SKUs and stock states. We map this parent-child hierarchy perfectly so your database receives clean, relational records.

Text Parsing
Normalising ingredient lists

Formulations are often presented as unstructured text blocks. We parse these strings into structured arrays, separating active ingredients from inactive bases for easier chemical analysis.

Dynamic Content
JavaScript hydration

Bluemercury relies on frontend JavaScript to load pricing, reviews, and inventory. We execute full Playwright sessions to capture the final rendered state of every product page.

Efficiency
Delta extraction

To monitor stock levels daily, we hash previous results and only deliver records that have changed. This reduces your ingest costs and processing time.

Applications

Who uses Bluemercury data and how

Teams across industries use bluemercury.com data to build competitive products and smarter operations.

01
Competitor Price Monitoring

Beauty retailers track Bluemercury pricing, promotions, and gift-with-purchase offers to remain competitive in the luxury segment.

02
Ingredient Trend Analysis

Cosmetic chemists and formulators mine ingredient lists to identify rising active compounds and clean beauty trends.

03
Brand Assortment Tracking

Emerging beauty brands monitor category saturation and competitor SKU counts to identify gaps in the market.

04
MAP Compliance

Luxury skincare brands audit retailer pricing to ensure strict adherence to Minimum Advertised Price agreements.

05
Supply Chain & Inventory

Analysts track out-of-stock rates across top brands to estimate sales velocity and supply chain bottlenecks.

06
Consumer Sentiment Analysis

Marketing teams aggregate review text to understand consumer reactions to new product formulations and packaging.

Why DataFlirt

"Bluemercury holds the definitive catalogue of luxury skincare and cosmetics, but mapping complex formulations to price points requires dedicated extraction infrastructure."

Cosmetics data extraction is notoriously complex due to endless shade variations, unstructured ingredient lists, and dynamic inventory states. DataFlirt manages the proxy rotation, JavaScript execution, and schema normalisation so your data science teams can focus on market analysis rather than pipeline maintenance.

Technical Spec

Bluemercury scraper: technical capabilities

Everything supported by our bluemercury.com scraper — rendered SPA elements, auth walls, rate-limit evasion and beyond.

JavaScript rendering
Full Playwright sessions required for dynamic inventory and review loading
Supported
CAPTCHA bypass
Automated solving for Cloudflare and edge security challenges
Supported
Residential proxy rotation
ISP-grade residential IPs from US pools rotated per request
Supported
Shade variant mapping
Extracts specific SKUs, hex codes, and names for all colour variants
Supported
Ingredient splitting
Converts raw ingredient text blocks into structured JSON arrays
Supported
Review pagination
Captures historical reviews across all paginated endpoints
Supported
Change detection
Hash-based diffing emits only changed records for inventory tracking
Supported
Webhook delivery
HTTP POST per record or batch for rapid downstream updates
Supported
BlueRewards loyalty points
User-specific loyalty point balances and tier rewards require authentication
Partial
User purchase history
Historical order data is gated behind individual user accounts
Partial
Infrastructure

Infrastructure powering the Bluemercury pipeline

Open-source tooling on proven cloud infra — no vendor lock-in, full observability.

ScrapyPlaywrightPython 3.12RedisPostgreSQLApache AirflowAWS LambdaS3CloudWatch2CaptchaCapSolverResidential ProxiesDockerKubernetesGrafanaPrometheus
Scrapy + Playwright Stack

Scrapy handles crawl orchestration and deduplication. Playwright manages JavaScript execution to render complex product pages and dynamic inventory states.

Residential Proxy Infrastructure

We route requests through US-based residential proxies with sticky sessions to bypass geo-restrictions and edge security.

Cloud-Native Orchestration

Pipelines run on AWS Lambda and ECS. Airflow handles scheduling and dependency management, ensuring data arrives strictly on time.

Output & Delivery

Your data, your destination

Data delivered to where your team already works — no new tooling required.

JSON
Newline-delimited or nested arrays for complex variants
CSV
Flat file with typed columns for immediate analysis
XLS
Excel compatible format for business analysts
Parquet
Columnar format optimised for BigQuery and Snowflake
AWS S3
Direct bucket delivery compatible with any data lake
Webhook
HTTP POST per record for real-time inventory alerts
API
REST endpoints to query your extracted datasets
Snowflake
Stage and COPY INTO workflow for immediate warehousing
S3
Direct bucket delivery — compatible with any data lake
// faq

Common questions.

About bluemercury.com scraping, legality, and pipeline operations.

Ask us directly →
Is scraping Bluemercury legal?

Scraping publicly available product, pricing, and review data is generally permissible. DataFlirt extracts only non-authenticated, public information. We do not bypass login walls to access personal data. Clients must ensure their specific use case complies with applicable laws.

How do you handle variant data for makeup?

Cosmetics require deep variant mapping. We extract the parent product data and create linked child records for every shade, capturing unique SKUs, hex codes, and individual stock availability.

Can you parse ingredient lists?

Yes. We extract raw ingredient text and apply normalisation rules to separate active compounds, inactive bases, and conscious beauty tags like 'paraben-free' into structured arrays.

How fresh is the inventory data?

For targeted SKU lists, we can configure high-frequency pipelines that check stock status hourly. Full catalogue refreshes typically run on a daily cadence.

Can you track exclusive brands like M-61?

Yes. We capture brand metadata and exclusive flags, allowing you to monitor the performance and pricing of Bluemercury's proprietary lines.

Do you extract BlueRewards data?

No. BlueRewards point balances, tier status, and user-specific purchase histories require account authentication, which falls outside our public data extraction scope.

Can I request a sample dataset?

Absolutely. We provide a sample run of up to 500 SKUs during the scoping phase so you can validate the schema, variant mapping, and ingredient parsing before committing.

$ dataflirt scope --new-project --source=bluemercury.com ready

Tell us what
to extract.
We do the rest.

20-minute scoping call. Pilot dataset within the week. Production within two. Whether you need a one-off catalogue dump or continuous inventory monitoring across thousands of SKUs, we scope, build, and operate the pipeline. Tell us what you need.

hello@dataflirt.com · Bengaluru · IST · typical reply < 4h
Related Scrapers

More in beauty and skincare

Services

Data Extraction for Every Industry

View All Services →