Matching 50 million songs across platforms that disagree on everything

2018. I built a music aggregation and metadata matching platform. The problem sounds simple: match the same song across different streaming platforms. The problem is not simple.

The mess

Every streaming platform stores metadata differently. Song titles don't match. "Don't Stop Me Now" on one platform is "Don't Stop Me Now - Remastered 2011" on another. Artist names have variations. Is it "The Beatles" or "Beatles, The"? Album art is inconsistent. ISRCs, the supposedly universal identifier for recordings, are unreliable. Different pressings get different codes. Some tracks have no ISRC at all.

You can't just do exact string matching. You'd miss half the catalog. You can't do naive fuzzy matching either. "Live in Paris" and "Live in Paris (Deluxe Edition)" might be different albums with different track listings.

The matching engine

I built a matching engine that normalized and cross-referenced metadata across sources. Fuzzy matching with confidence scoring. String normalization that strips parenthetical additions, handles unicode variations, and deals with the hundred ways platforms format "featuring" in a track title.

High-confidence matches got auto-approved. Low-confidence matches went into a manual review queue. The platform processed millions of records. The auto-match rate was around 85%. The remaining 15% needed human eyes.

The lesson

Data quality problems are never solved by better algorithms alone. You need humans in the loop for the edge cases. The algorithm handles the volume. The human handles the judgment. Trying to automate the last 15% would have taken more effort than the first 85% and produced worse results.

That lesson shows up in every agent framework I build now. Agents handle the searching, the matching, the grunt work. But the system always has a path for human review when confidence is low. Full automation is a trap. Smart automation with human oversight is the goal.