Abstract
A Python classification and embedding component inside the PartsRadar price-intelligence pipeline. It takes noisy retailer product titles, normalizes them, and resolves them into comparable product entities so downstream price history and comparison views operate on consistent data. The interesting problem is entity resolution across stores, not simple category labeling.
1. What This Is
Product Classifier is a closed-source component of the PartsRadar system. Its job is to take raw or lightly cleaned retailer listing titles and map them to a canonical product entity, so that the same hardware model written differently across stores ends up in one comparison group.
The classifier is deliberately separated from the scraping layer. Model and matching logic can evolve without touching each retailer collector, and the output is a normalized classification result that PartsRadar persistence and analytics stages consume directly.
2. How It Works
The pipeline runs in five sequential stages, from raw listing title to canonical entity assignment.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Ingest listing titles | Raw/cleaned retailer titles from crawler | Python | Structured title records |
| 02 | Normalize naming | Structured title records | Python preprocessing | Consistent brand/model/capacity/variant fields |
| 03 | Embed and classify | Normalized fields | Embeddings + classifier | Semantic match candidates |
| 04 | Resolve entity | Match candidates | Classifier pipeline | Canonical product entity assignment |
| 05 | Hand off to PartsRadar | Canonical entity | SQL persistence | Price history and comparison views |
3. Implementation Notes
3.1 Entity resolution, not just labeling
The hard part is that two listings written differently may refer to the same hardware model. Exact string equality fails on brand aliases, capacity units, and variant suffixes. Embeddings let the classifier compare semantic content rather than raw text, which is what makes cross-store grouping feasible.
3.2 Separation from scraping
The classifier is a standalone component. Each retailer collector produces titles; the classifier consumes them. A new retailer can be added by writing a collector only, and the matching logic does not need to change per store.
3.3 Synthetic and custom dataset
Training and evaluation used a project-specific synthetic or custom dataset rather than a public benchmark. This keeps the component scoped to the PartsRadar product domain but limits how well the classifier generalizes to unseen naming conventions.
4. Constraints
-
Closed-source component
No public repository was available. The specific embedding model, vector store, and classifier architecture are not documented in the reviewed materials, so external verification is not possible.
-
Synthetic data ceiling
The classifier was trained and evaluated on a project-specific dataset. Real-world retailer naming drift, new brands, and edge-case variants will likely produce lower match rates than the evaluation suggests.
-
Ambiguity in entity resolution
Two listings can be close in embedding space but refer to different hardware revisions. Without a confidence threshold and manual review path, mis-grouped entities silently corrupt downstream price comparisons.
-
Single-domain scope
The component is built for the PartsRadar product category. Extending it to unrelated product domains would require retraining or at minimum a new synthetic dataset.
5. Next
- a. Document the specific embedding model, classifier architecture, and vector store in a public README so the component can be independently reviewed.
- b. Add a confidence score to each entity assignment and route low-confidence matches to a manual review queue before they enter the price-history table.
- c. Expand the synthetic dataset with real retailer edge cases (brand aliases, unit mismatches, discontinued variants) to measure and improve cross-store match accuracy.
— end of report —