Fuzzy Matching for Product Catalogues: A Workflow for Variants, SKUs, and Duplicate Listings
Product DataEcommerce SearchSKU MatchingDuplicate DetectionEntity ResolutionCatalogue Management

Fuzzy Matching for Product Catalogues: A Workflow for Variants, SKUs, and Duplicate Listings

FFuzzy Point Editorial Team
2026-11-10
8 min read

A practical workflow for matching messy product records, detecting duplicates, and protecting genuine variants with explainable fuzzy matching.

Product catalogue data rarely arrives in a consistent shape. One supplier may send Acme Dishwasher Tablets 40 Pack, another may use ACM DW TABS x40, while a marketplace seller lists the same item with a shortened SKU or a different pack-size format. This workflow explains how to use fuzzy matching, field normalisation, candidate blocking, weighted scoring, and human review to match product records safely and detect duplicate listings without merging genuine variants.

Overview

Fuzzy matching for product catalogues is an entity-resolution problem: the goal is to decide whether two imperfect records represent the same product. The decision should not be based on title similarity alone. A reliable process compares several signals, including brand, model or MPN, SKU, barcode, pack size, product type, and selected attributes.

Start by separating three related tasks:

  • Record matching: linking a supplier item to an existing canonical product.
  • Duplicate detection: finding listings that may describe the same product within a catalogue or marketplace.
  • Variant identification: distinguishing legitimate differences such as colour, capacity, flavour, size, or pack quantity.

These tasks can use the same approximate string matching tools, but their thresholds and risk tolerance differ. A duplicate-detection system can send uncertain pairs to review, whereas an automated catalogue import may need to reject a low-confidence match rather than attach a new listing to the wrong product.

Exact matching remains valuable. A verified barcode, a stable manufacturer part number, or a trusted internal product ID should generally be checked before fuzzy methods are applied. Approximate string matching is most useful when identifiers are missing, mistyped, reformatted, or inconsistently stored.

For background on choosing a similarity method, see Fuzzy Matching Algorithms Compared. The main principle is simple: use fuzzy scores to support a controlled decision process, not to replace product-data rules.

What to track

1. Build a field-cleaning checklist

Normalisation should make equivalent values comparable without removing information needed to identify variants. Apply transformations consistently to both sides of every comparison.

  • Trim leading and trailing whitespace and collapse repeated spaces.
  • Use a consistent case, such as lowercase, for comparison fields.
  • Normalise punctuation, hyphens, slashes, and common separators.
  • Convert known unit formats to a standard representation, such as g to kg where appropriate.
  • Separate numbers from units so 500ml, 500 ml, and 0.5 l can be compared deliberately.
  • Standardise common abbreviations only when their meaning is unambiguous.
  • Preserve the original value for display, auditing, and review.
  • Tokenise titles and attributes so word-order changes do not automatically look like different products.

Do not blindly remove numbers. In product data, numbers often carry the most important distinction: 32GB versus 64GB, 250ml versus 500ml, or a two-pack versus a six-pack. It is usually safer to extract numerical attributes into dedicated fields and compare them separately.

2. Track identifiers and their trust level

Record whether a field is present, normalised, validated, and sourced from a trusted feed. Useful fields include GTIN or barcode, manufacturer part number, supplier SKU, brand, model, product title, category, colour, size, capacity, material, and pack quantity.

Identifiers should not all receive the same weight. A supplier SKU may be unique only within that supplier, while a manufacturer part number may be shared incorrectly across listings. Store the source system and validation status alongside each identifier so the matching logic can reflect its reliability.

3. Compare methods before choosing a score

MethodUseful forLimitations
Exact matchingValidated barcodes, IDs, and normalised attributesBreaks when formatting or data entry differs
Token-based similarityTitles with reordered words, brands, and descriptive phrasesCan overvalue shared generic terms
Edit distanceTypos, missing characters, and short codesLess suitable when word order changes substantially
Character n-gram or trigram similarityLong titles and noisy catalogue textNeeds threshold and index tuning
Semantic similarityRelated wording and broader product descriptionsMay treat distinct variants as similar without structured checks

Levenshtein distance can help with a misspelled model code, while token similarity is often more useful for reordered product titles. Jaro-Winkler may be useful for short strings where shared prefixes matter, but it should not be treated as a universal product-matching algorithm. The article Levenshtein vs Jaro-Winkler vs Jaccard provides a useful comparison framework.

4. Record the evidence behind every match

For each candidate pair, retain the component scores, matched tokens, normalised values, rule outcomes, final decision, and decision version. This makes a match explainable and lets the team identify whether errors come from bad input data, weak blocking, an unsuitable metric, or an overly permissive threshold.

Cadence and checkpoints

Treat catalogue matching as an operating process rather than a one-time import task. A practical cadence is to review the system after every major supplier onboarding and on a monthly or quarterly schedule, depending on catalogue volume and change rate.

At every import

  • Check whether new columns, delimiters, units, or naming conventions have appeared.
  • Measure the percentage of rows matched exactly, matched fuzzily, sent to review, and left unmatched.
  • Sample high-confidence matches and low-confidence non-matches.
  • Look for sudden increases in one supplier's unmatched or ambiguous records.
  • Check whether new product categories introduce attributes not covered by the current rules.

Monthly checkpoint

  • Review false merges, particularly where different pack sizes or capacities were combined.
  • Review missed duplicates that received separate canonical records.
  • Inspect the most common unmatched tokens, abbreviations, and unit formats.
  • Compare match rates by supplier, category, brand, and data source.
  • Check the size and age of the manual review queue.

Quarterly or release checkpoint

  • Reassess field weights and thresholds using reviewed decisions.
  • Test new taxonomy categories and seasonal product formats.
  • Rebuild or validate candidate-blocking keys if the catalogue has grown materially.
  • Run a fixed regression set before changing normalisation or scoring logic.
  • Record rule changes so older decisions remain auditable.

Candidate blocking is an important control. Comparing every new record with every existing product is expensive and creates many irrelevant pairs. Instead, create a broad candidate set using signals such as brand, category, barcode prefix, manufacturer, model-code fragments, or selected title tokens. Blocking should reduce comparisons without excluding plausible matches; using several overlapping blocking keys is safer than relying on one strict key.

Implementation choices depend on the data store and application. A Python team may use a string-similarity library such as RapidFuzz, a JavaScript application may use Fuse.js or a custom index, and a PostgreSQL workflow may use trigram operators and indexes. The technology matters less than preserving explainability, testing representative data, and measuring the result. See the fuzzy matching libraries comparison for a broader implementation checklist.

How to interpret changes

A similarity score is evidence, not a verdict. Use at least three decision bands:

  • Automatic match: strong evidence from identifiers or several independent fields, with no conflicting variant attributes.
  • Manual review: plausible evidence, but a missing identifier, conflicting quantity, or close score between multiple candidates.
  • No match: insufficient evidence or a clear conflict such as different capacity, model, colour, or pack count.

A weighted score can be represented as:

overall score = identifier score × identifier weight + title score × title weight + brand score × brand weight + attribute score × attribute weight

The exact weights should be learned from reviewed examples and adjusted by category. For example, model numbers may be decisive for electronics, while pack quantity and size may be more important for groceries. Do not let a highly similar title override a confirmed conflict in a defining attribute.

Keep separate thresholds for matching and duplicate detection. A pair that is similar enough to display as a search suggestion may not be safe to merge. Also distinguish a missing value from a conflicting value: unknown pack size is not the same as a verified different pack size.

Watch for recurring failure patterns. Generic words such as replacement, premium, wireless, or organic can inflate token similarity. Brand names may be misspelled or embedded in titles. Marketplace sellers may add promotional text, warranty wording, or delivery information. Titles may also contain multiple products in a bundle, making a close text score misleading.

When errors increase, inspect the distribution of scores rather than changing the threshold immediately. A new supplier format may have shifted scores, or a taxonomy change may have introduced a new variant attribute. A threshold calculator and a fixed labelled sample can help compare alternatives without relying on intuition alone. For production controls, see Production Fuzzy Search Monitoring.

When to revisit

Revisit the workflow whenever a supplier changes its export, a marketplace adds a new listing format, a category introduces new variants, or a catalogue migration changes identifiers. Also schedule a regular monthly or quarterly review even when imports appear stable. Quiet drift can show up first as a growing review queue, a lower exact-match rate, or more products with near-identical titles.

Before changing production rules, create a labelled evaluation set containing confirmed matches, confirmed non-matches, and difficult variant pairs. Include examples from every important supplier and category. Run the current and proposed rules against that set, then inspect the errors manually. Prioritise preventing false merges: combining two distinct products can damage stock, pricing, fulfilment, and customer-facing content. Missed matches are still important, but they can often be routed to a review queue without corrupting the canonical catalogue.

Use this practical checklist for the next catalogue refresh:

  1. Export the raw records and preserve the import version.
  2. Run normalisation while retaining original field values.
  3. Apply exact identifier rules first.
  4. Generate candidates with more than one blocking key.
  5. Calculate field-level similarity and check variant conflicts.
  6. Route results into automatic match, review, and no-match bands.
  7. Sample decisions by supplier, category, and confidence band.
  8. Log corrections and feed confirmed decisions back into the evaluation set.
  9. Review the process again after the next import or scheduled checkpoint.

This approach makes fuzzy matching ecommerce data more controlled and repeatable. It supports product record deduplication while leaving room for human judgement where the available evidence is incomplete. As catalogue formats, taxonomies, and supplier conventions change, the workflow can be updated without discarding the audit trail that explains earlier decisions.

Related Topics

#Product Data#Ecommerce Search#SKU Matching#Duplicate Detection#Entity Resolution#Catalogue Management
F

Fuzzy Point Editorial Team

SEO and Search Engineering Editors

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.