You need to match inventory across platforms, track competitive pricing, or feed product catalogs to Google Merchant Center. Relying on product titles or thumbnail images fails at scale. Titles change constantly. Images get compressed or watermarked differently across marketplaces. You need exact, standardized product identifiers to build reliable data pipelines.
Key takeaways
- Product pages often hide GTINs and UPCs from the visible frontend HTML.
- JSON-LD script tags are the most reliable extraction targets for standardized identifiers.
- Amazon relies on proprietary ASINs; mapping them to global GTINs requires structured table parsing.
- Missing identifiers affect roughly 30 percent of online listings.
- Fallback matching strategies using brand, MPN, and category mapping are mandatory for robust pipelines.
- Legal orientation: Scraping public identifiers is generally fair game; always check platform terms of service and consult counsel.
The product identifier hierarchy which one to use and when
Use GTIN-13 for global cross-platform matching, UPC for North American retail, and ASIN exclusively for Amazon ecosystems. MPNs provide a fallback when global standards are absent entirely.
To build a highly accurate pricing intelligence tool, you must understand how these identifiers overlap. They are not interchangeable. Some are globally unique, while others are platform-specific. DataFlirt categorizes these identifiers based on their validation strictness and regional application.
| Identifier | Scope | Where used | Precision |
|---|---|---|---|
| GTIN-13 (EAN) | Global | Worldwide retail | Highest precision |
| GTIN-12 (UPC) | North America | Primary Amazon and Walmart listings | High precision |
| ASIN | Amazon-specific | Amazon catalog exclusively | One-to-many to GTINs |
| MPN | Manufacturer-assigned | Electronics and auto parts | Medium precision |
| ISBN | Books only | Global publishing | Highest precision |
Global Trade Item Numbers
GTINs are the absolute gold standard for product matching. They prevent an algorithm from matching a bulk 12-pack of paper towels to a single unit. Brand owners have uploaded over 200 million unique GTINs to the official GS1 Registry Platform for global product verification.
DataFlirt prioritizes GTIN extraction above all other fields during catalog scraping. When DataFlirt engineers design a pipeline, they look for GTIN-13 or EAN codes first. These 13-digit numbers contain manufacturer prefixes and specific check digits. They guarantee exact product parity across international borders.
Universal Product Codes
UPCs are technically known as GTIN-12s. They dominate North American retail environments. You will find them heavily utilized on platforms ranging from big box hardware stores to local grocery delivery apps.
Having exact UPCs directly impacts sales visibility. Merchants who add exact GTINs to their e-commerce product data feeds experience up to a 20% conversion rate increase. Furthermore, GTIN-matched listings observe a 40 percent higher click-through rate. DataFlirt extracts these codes to help clients syndicate their inventory correctly to shopping aggregates.
Amazon Standard Identification Numbers
ASINs are completely proprietary. Amazon assigns a 10-character alphanumeric code to every single item in its system. You cannot use an ASIN to find a product on a competing platform.
Amazon currently lists an estimated 600 million products across its global marketplace. Over 98 percent of these listings originate from third-party sellers. DataFlirt routinely maps ASINs back to their root UPCs by scraping the backend metadata, allowing cross-platform comparison.
Manufacturer Part Numbers
MPNs are alphanumeric strings assigned directly by the brand. They lack a central registry. You need them desperately for electronics, hardware, and automotive parts.
Because they lack strict formatting rules, MPNs are prone to data entry errors. A vendor might include a dash; another might omit it. DataFlirt sanitizes MPNs by stripping special characters and converting strings to uppercase before executing database joins.
International Standard Book Numbers
ISBNs apply exclusively to published books. They function exactly like a GTIN. Modern ISBNs are 13 digits long and always start with 978 or 979.
When scraping bookstores or educational platforms, DataFlirt configures scrapers to target the ISBN specifically. This prevents confusion when books have multiple editions, paperback versions, or audio equivalents.
Where to find each identifier on product pages
Look for identifiers in the JSON-LD markup, canonical URLs, or backend product attribute tables. Visible HTML text is the least reliable location for extraction.
Product pages often do not show GTINs publicly. How do you get them at scale? You look in the DOM where the browser reads metadata, not where the user reads text. DataFlirt engineers specifically target structured metadata to bypass the chaos of frontend redesigns.
Amazon catalog extraction quirks
Amazon deliberately hides the UPC from the public frontend. While sellers must submit a valid UPC to create a listing, the customer interface only shows the ASIN. To scrape Amazon product identifiers, you must target the URL path itself or the hidden details table.
DataFlirt utilizes automated URL parsing to grab the ASIN reliably. For amazon listings, the ASIN sits predictably in the address bar. Finding the underlying GTIN requires a deeper dive into the product details table, assuming the third-party seller included it in their specification block.
Big box retailers and hidden fields
Retailers like target and walmart handle identifiers differently. They frequently expose UPCs in their specification tables. However, they sometimes render these tables via heavy JavaScript payloads.
DataFlirt intercepts the XHR network requests for these sites rather than trying to parse the fully rendered HTML. This approach captures the raw JSON product data directly from the retailer’s backend API. This raw data almost always contains the clean, unformatted GTIN.
Shopify storefront architecture
The Shopify Storefront API’s ProductVariant object does not always expose the barcode field directly to the client-side frontend by default. It typically must be explicitly output into the theme’s schema markup or mapped to a custom metafield by the store owner to be scrapable.
When DataFlirt targets sites like wayfair or homedepot that might employ headless commerce architectures, our scrapers look for the underlying product manifest. DataFlirt bypasses the visual DOM entirely. We append .json to standard Shopify product URLs to retrieve the structured catalog data directly.
B2B marketplaces and specification tables
Platforms like flipkart and indiamart cater heavily to bulk buyers and suppliers. These sites usually place the MPN prominently in a specifications table. The HSN code or manufacturer part number is critical for B2B procurement.
DataFlirt navigates these massive tables using precise css-selector logic. We target the key-value pairs systematically. If a row header says “Model Number”, DataFlirt captures the adjacent cell as the MPN.
Brand manufacturer websites
Brand websites are often the cleanest source of identifier truth. Manufacturers format their pages with deep SEO optimization in mind. They embed identifiers inside JSON-LD tags to appease search engine crawlers.
When checking a direct brand site like sephora or chewy, DataFlirt ignores the visual styling. DataFlirt goes straight to the page source. The JSON-LD schema is drastically cleaner than relying on scraped visible text.
Extraction method for identifiers
Extract JSON-LD data using a parser, apply regex to canonical URLs, and traverse HTML tables for specific attribute keys. These three methods cover the vast majority of e-commerce websites.
Visible text scraping breaks the moment a site changes its font, layout, or translation logic. Structured data extraction scales cleanly. DataFlirt standardizes these three programmatic methods to guarantee high-fidelity data extraction across thousands of targets.
Parsing schema markup with Python
JSON-LD extraction is the primary weapon in the DataFlirt arsenal. Webmasters inject this data into script tags formatted as application/ld+json. The schema strictly follows schema.org/Product formatting rules.
DataFlirt engineers prefer json-parsing because it avoids HTML fragility completely. Here is how a simplified DataFlirt script locates and extracts a GTIN from a product page.
# DataFlirt JSON-LD Extraction Example
# Requires: pip install beautifulsoup4
import json
from bs4 import BeautifulSoup
def extract_identifiers(html_content):
soup = BeautifulSoup(html_content, 'html.parser')
script_tags = soup.find_all('script', type='application/ld+json')
for tag in script_tags:
try:
data = json.loads(tag.string)
if '@type' in data and data['@type'] == 'Product':
return {
'gtin13': data.get('gtin13'),
'mpn': data.get('mpn'),
'sku': data.get('sku')
}
except json.JSONDecodeError:
continue
return NoneThis basic pattern handles a massive percentage of modern retail sites. DataFlirt hardens this logic to handle nested arrays, malformed JSON, and unexpected key variations.
Extracting ASINs via regex
URL parsing is incredibly fast. When scraping Amazon, the URL structure is highly predictable. You do not need to download the entire multi-megabyte HTML payload just to find the ASIN.
DataFlirt utilizes regex-in-scraping to isolate the 10-character code immediately. The ASIN always follows /dp/ or /product/ in the URL path. A standard DataFlirt regex pattern looks like this:
import re
def extract_asin(url):
pattern = r'(?:dp|o|gp|-|product)/([a-zA-Z0-9]{10})'
match = re.search(pattern, url)
if match:
return match.group(1)
return NoneDataFlirt runs this regex on millions of URLs daily. It isolates the product identity before the crawler even initiates a full page render, saving massive amounts of bandwidth and compute time.
Traversing product specification tables
When metadata fails, you must parse the HTML tables. xpath is essential here. E-commerce tables use generic classes, making them difficult to scrape without structural logic.
DataFlirt isolates the table elements and iterates through the rows. If the first column contains the text “UPC” or “EAN”, DataFlirt pulls the text from the second column. This method requires constant monitoring, as site redesigns will break the XPath queries.
Validating check digits programmatically
Extraction is only the first step. Validation is mandatory. Google Merchant Center strictly validates GTINs against the GS1 check-digit algorithms. If a scraper pulls a malformed number and submits it to Google, it will result in feed disapproval.
DataFlirt never passes raw GTINs to a client without running the modulo 10 validation algorithm. The last digit of any standard GTIN is mathematically derived from the preceding digits.
def validate_gtin_check_digit(gtin_string):
if not gtin_string.isdigit() or len(gtin_string) not in [8, 12, 13, 14]:
return False
payload = gtin_string[:-1]
check_digit = int(gtin_string[-1])
total = 0
# Reverse payload and multiply alternating digits by 3 and 1
for i, digit in enumerate(reversed(payload)):
multiplier = 3 if i % 2 == 0 else 1
total += int(digit) * multiplier
calculated_check = (10 - (total % 10)) % 10
return calculated_check == check_digitIf a merchant attempts to fabricate a missing GTIN, or if the scraper pulls a phone number by mistake, this function catches it. DataFlirt drops invalid identifiers immediately to preserve data integrity.
Handling missing identifiers at scale
Build a fallback logic hierarchy using Brand, MPN, and category mapping when GTINs are absent. Never fabricate missing identifiers to force a platform match.
You will never achieve 100 percent identifier coverage across a massive catalog. Third-party marketplace sellers are notoriously lazy with data entry. DataFlirt builds defensive pipelines assuming that identifiers will occasionally disappear.
The reality of missing data
An estimated ~30% of product listings across retailers lack a visible GTIN. This makes cross-platform product matching difficult without advanced scraping or attribute-based fuzzy matching.
If you blindly trust supplier feeds, your downstream applications will fail. Maintaining high data-quality means acknowledging gaps. DataFlirt flags missing identifiers explicitly. We retain “missingness” as a documented data quality field. DataFlirt never imputes a GTIN silently.
Fallback composite keys
When the GTIN is absent, you must create a composite key. DataFlirt combines the Brand name, the MPN, and a normalized category string.
For example, if you are scraping alibaba and the UPC is missing, DataFlirt extracts the manufacturer part number. “Sony” plus “WH-1000XM4” plus “Headphones” provides a highly reliable match rate against competitor sites. DataFlirt standardizes brand spellings to ensure these composite keys join correctly in your data warehouse.
Fuzzy matching and Levenshtein distance
Title-based fuzzy matching is an absolute last resort. DataFlirt deploys this only when structured metadata and composite keys fail.
We utilize Levenshtein distance calculations to measure how many character edits it takes to turn one product title into another. For extreme edge cases, DataFlirt applies embedding similarity models. This is particularly useful for generating ai-training-data where natural language variance is high.
Consider a data engineer building a repricing engine for 50,000 SKUs across six marketplaces. Every morning, she needs exact matches, not guesses. A pipeline relying purely on fuzzy titles will incorrectly reprice a 64GB phone based on the 128GB model’s data. Identifiers are non-negotiable for pricing automation.
Legal orientation for public identifiers
Scraping product identifiers generally involves public, non-personal data. GTINs and ASINs describe inventory, not human beings. Therefore, they fall outside the strict purviews of GDPR or CCPA frameworks.
However, terms of service matter. Marketplaces routinely deploy anti-bot systems to protect their catalog structures. You should review the legal realities of your target platforms. DataFlirt recommends consulting qualified legal counsel regarding your specific jurisdiction and data utilization plans before initiating high-volume extractions.
DataFlirt for identifier extraction
DataFlirt engineers custom extraction pipelines that prioritize backend metadata schemas over fragile visible DOM elements. We deliver validated, check-digit-verified product identifiers ready for immediate ingestion.
A freelancer on a gig platform might write a quick script to pull visible HTML text. Once you cross into millions of SKUs, or hit Cloudflare-protected sites like ebay, that script will fail. That is the threshold where DataFlirt starts paying for itself.
DataFlirt handles the heavy lifting of what-is-web-scraping at enterprise scale. When DataFlirt provisions a pipeline, we blend JSON-LD parsing, regex URL extraction, and XHR interception to maximize identifier coverage.
DataFlirt supports complex web-scraping-ecommerce-product-data requirements. We run the modulo 10 validation on every GTIN. DataFlirt outputs clean, normalized CSV or JSON payloads directly to your cloud storage. Every delivery from DataFlirt includes confidence scores and missingness flags per identifier field, ensuring your database remains pristine.
FAQ
Which product identifier is the most accurate for global matching?
GTIN-13 (EAN) is the highest precision identifier. It is regulated by GS1 and contains strict manufacturer and product assignments with algorithmic check-digits.
Why can’t I find the UPC on an Amazon product page?
Amazon requires sellers to submit a UPC to the backend, but deliberately hides it from the public frontend. Scrapers must rely on the proprietary ASIN found in the URL or page details.
How do I extract an identifier if it is not in the visible HTML text?
You should target the JSON-LD script tags formatted as schema.org/Product. This backend metadata is explicitly structured for machines and contains clean GTIN and MPN data.
What should I do if a product has no GTIN or UPC?
You should build a fallback composite key using the Brand, MPN, and normalized category. Do not attempt to guess or fabricate a GTIN, as platforms like Google Merchant Center will suspend your account for invalid check-digits.
If you would rather not scope this yourself, DataFlirt’s ecommerce scraping service handles the extraction, QA, and delivery. We navigate the anti-bot systems and validate the metadata automatically. Reach out for a free scoping call.
