Six engines for one songbook.

I am, mortifyingly, the kind of Taylor Swift listener who has opinions about which 2019 demos should have stayed unreleased.1 So when I built a recommender for her catalogue I was the worst possible audience: any model that surfaced something obvious would feel patronising, and any model that surfaced something stupid would feel personal.1. The original Lover demos. Don't @ me.

§01Where it started: an undergrad free-reign brief

This was my favourite undergraduate project. The brief was unusually generous and pretty much to build anything you like, provided it is written in R. The original was a single collaborative filtering engine wrapped in a Shiny app. Slow (two or three seconds per query), single-user, and opinionated in ways I didn't fully understand yet. But my friends and I had fun with it, and three years later I rebuilt it from scratch. You can find both repos on my Github and below.

§02Six engines, one query

I ended up with six recommendation models, each looking at a different facet of a song. Five base engines plus an ensemble that learned how to weight them.

1. Lyrics-Transformer. Sentence-BERT embeddings over Genius-scraped lyrics. Cosine similarity in the 384-D embedding space.
2. Audio VAE. A variational autoencoder over Spotify audio features, compressed to a 16-dimensional latent.2 Distance in latent space picks up sonic neighbours.
3. Graph node2vec. Build a song-co-listen graph, run node2vec for 64-D embeddings, retrieve nearest neighbours.
4. Neural Collab Filtering. Classic two-tower model on user-song interactions.
5. Contrastive SSL. Self-supervised pretext: same song, two augmentations of its audio, pull together; different songs, push apart.
6. Ensemble. Weighted blend of the five, with a consensus boost for songs surfaced by ≥2 engines.

§03The dataset that made it tractable

The engines run on Taylor's full discography — every era, including Vault tracks — with editorial bridges to neighbouring artists.3 The bridges are metadata, not embeddings: the six engines compute similarity within Taylor's catalogue, and the editorial layer maps outward to artists who share lyrical or sonic DNA.3. Editorial bridges to Kacey Musgraves, Maggie Rogers, Olivia Rodrigo, Gracie Abrams, Lana Del Rey, HAIM, Phoebe Bridgers, Carly Rae Jepsen, Paramore, and others. The mapping is hand-curated — each bridge has a reason.

§04Why the ensemble works

Each engine has a failure mode that the others compensate for. Lyrics-only will surface "songs that mention rain" when you wanted "songs that feel like rain." Audio-only will surface acoustic ballads when you wanted lyrical heartbreak. Collaborative filtering will surface whatever's popular this week. The ensemble is less a fusion of strengths than an averaging-out of weaknesses.

// pullThe ensemble is less a fusion of strengths than an averaging-out of weaknesses.

§05What the demo actually feels like

Type a song. Six engines return their picks in parallel. The UI shows you the ensemble's top 10, but lets you toggle "show me what each engine thought" so you can see the disagreements. The disagreements are fun! The lyrics engine wants you to listen to The Manuscript; the audio VAE thinks you'd rather have You're On Your Own, Kid; the graph engine sends you somewhere else.

The site is at shubz-taylor-recommendation-engine.vercel.app. This is where I've got to so far. I'm still thinking about what a seventh engine would do — and how I can get the editorial bridges to be learned rather than hand-curated. Bring your own opinions about the Lover demos.

— written on the train back from a Tortoise live show.