Recommendation Engines: Choosing Algorithms, Data and Metrics That Work

Recommendations appear everywhere: “customers also bought”, “because you watched”, “suggested for you”. Done well, they help users find relevant items and increase engagement and revenue. Done badly, they repeat the obvious, promote what is already popular or feel intrusive.

This guide explains the main approaches and how to choose and measure them. For streaming-specific architecture, see our guide to AI for streaming platforms.

Algorithm families

Staged architecture

1. Candidate generation: several fast methods — similar items, collaborative filtering, trending, recently viewed — each propose candidates.
2. Ranking: a model scores candidates using user, item and context features.
3. Re-ranking: business rules apply — stock availability, margins, diversity, freshness, compliance and exclusions.
4. Presentation: placement, explanations (“because you viewed…”) and logging for learning.

Data you need

• Interaction events: views, clicks, add-to-cart, purchases, ratings, watch time, returns — with timestamps and context.
• Item metadata: categories, attributes, descriptions, price, availability.
• User context: device, location (coarse), session behaviour; profile data only with appropriate consent.
• Negative signals: skips, returns, hidden items, quick abandonment.

Consistent item and user identifiers across channels are essential. Many recommendation projects stall on data quality rather than modelling.

Cold start

• New users: popular and trending items, onboarding preferences, contextual signals and quick learning from first sessions.
• New items: content-based similarity from metadata and text embeddings; controlled exposure to gather feedback.

Measuring recommendations

Offline results often fail to predict online performance, so run A/B tests with control groups before scaling. Watch for feedback loops where recommendations only reinforce what was already shown.

Worked example: an online bookstore

A hypothetical bookstore with 200,000 titles and moderate traffic starts with “bestsellers in this category” and “customers also bought” based on order history. It then adds content-based recommendations using genre, author and description embeddings, which help new titles appear. Once it has a year of interaction data, it trains a ranking model that combines purchase likelihood, margin and stock, with re-ranking rules to avoid showing the same author five times and to exclude titles the customer already owns.

Each step is tested against a control group. The ranking model lifts conversion from recommendation widgets, but the team also tracks returns and the share of sales coming from outside the top 1,000 titles, to make sure recommendations broaden discovery rather than narrowing it.

Common mistakes

• Recommending what was just bought: exclude purchased items unless they are consumables.
• Ignoring stock: recommending unavailable products frustrates users.
• Over-optimising clicks: clickbait recommendations raise engagement but not satisfaction or revenue.
• No exploration: systems that only exploit known preferences stop learning.
• Leaking test data: evaluating offline on data that overlaps with training produces misleadingly good scores.

Responsible recommendations

• Transparency: explain why items are recommended where possible.
• Control: let users remove items from history and adjust preferences.
• Diversity: avoid narrowing choices excessively.
• Sensitive inferences: avoid recommendations that reveal health, financial or other sensitive information.
• Regulation: data-protection and platform rules may require consent, transparency or non-personalised options in some markets.

A staged roadmap

1. Baseline: popularity, recently viewed and simple item similarity; instrument events.
2. Collaborative filtering or matrix factorisation once enough interaction data exists.
3. Learning to rank combining signals and business objectives.
4. Deep learning and embeddings for sequences, content understanding and scale.
5. Continuous experimentation with a holdout group to measure long-term impact.

Frequently asked questions

How much data do we need?

Content-based and popularity approaches work from day one. Collaborative filtering becomes useful once many users have interacted with many items; the exact threshold depends on catalogue size and behaviour.

Should we buy a recommendation service?

Managed services are a good start for many teams. Build custom when you need special business rules, unusual data or tight control.

Can LLMs replace recommendation algorithms?

Not for large-scale ranking. They are useful for understanding queries, enriching metadata and explaining recommendations, alongside traditional models.

How often should models be retrained?

It depends on how fast behaviour and catalogue change — daily for fast-moving retail, less often for stable catalogues — with monitoring to detect drift.