Recommender Systems: Complete Guide — Collaborative Filtering and Matrix Factorization

Summary — Recommender systems are at the heart of Netflix, Amazon, and Spotify. Collaborative filtering predicts preferences by finding similar users or items, without knowing the content of the items. Matrix Factorization decomposes the user-item matrix into latent factors revealing hidden dimensions like “taste for science-fiction” or “preference for 80s movies.” It is the most influential algorithm in modern e-commerce and entertainment.

Mathematical Principle

1. User-Based Collaborative Filtering

The fundamental idea: users who agreed in the past will also agree in the future.

Cosine similarity between users u and v:

sim(u, v) = (Σ r_ui · r_vi) / (√Σ r_ui² · √Σ r_vi²)

Where the sums are over items rated in common by u and v.

Pearson correlation (better because it centers the ratings):

sim(u, v) = Σ(r_ui - r_u_bar)(r_vi - r_v_bar) / (√Σ(r_ui - r_u_bar)² · √Σ(r_vi - r_v_bar)²)

Prediction of user u’s rating for an unrated item i:

r_hat(u,i) = r_u_bar + Σ sim(u,v) · (r_v,i - r_v_bar) / Σ |sim(u, v)|

Where the sum is over the k nearest neighbors of u who have rated item i.

2. Item-Based Collaborative Filtering

Instead of finding similar users, we find similar items. If you like Inception and Inception is similar to Interstellar, we recommend Interstellar. More stable than user-based because item similarities change less than user tastes.

3. Matrix Factorization (FunkSVD)

The ratings matrix R (u users × m items) is very sparse (> 99% empty). Matrix factorization learns to approximate it by the product of two rank-k matrices where k ≪ min(n, m):

R ≈ U × V^T

Where U ∈ R^{n×k} represents users in the latent space and V ∈ R^{m×k} represents items. Each dimension of the latent space can correspond to an implicit factor like “taste for action” or “importance of positive reviews.”

Regularized cost function:

L = Σ_{(u,i)} (r_ui - u_u · v_i)² + λ(||u_u||² + ||v_i||²)

Where the sum is over observed ratings only (not the empty cells).

4. Optimization by Stochastic Gradient Descent

For each observed rating (u, i):

e_ui = r_ui - u_u · v_i          # error
u_u ← u_u + η(e_ui · v_i - λ u_u)  # user update
v_i ← v_i + η(e_ui · u_u - λ v_i)  # item update

Where η is the learning rate and λ is the regularization.

Intuition

Imagine your cinephile friend. He knows exactly which movies to recommend because he knows your tastes. No need to analyze the genre, director, or budget. He thinks: “You and I like the same things, and I loved this movie so you will too.”

Collaborative filtering is exactly that, but at the scale of millions of users. The friend’s “knack” is replaced by mathematical similarity calculations.

Matrix Factorization goes further: instead of comparing users directly, it discovers “hidden dimensions.” These dimensions are not labeled but can be interpreted a posteriori: dimension 3 could correspond to a love of 90s action movies, dimension 7 to a preference for foreign films. The model finds them on its own, without anyone telling it what an “action movie” is.

Python Implementation

1. FunkSVD from scratch

import numpy as np

# ---- FunkSVD from scratch ----
def funkSVD(R, mask, n_factors=50, n_epochs=100, lr=0.01, reg=0.02):
    """FunkSVD by stochastic gradient descent."""
    n_users, n_items = R.shape
    U = np.random.normal(0, 0.1, (n_users, n_factors))
    V = np.random.normal(0, 0.1, (n_items, n_factors))

    users_idx, items_idx = np.where(mask)

    for epoch in range(n_epochs):
        perm = np.random.permutation(len(users_idx))
        total_loss = 0

        for idx in perm:
            u = users_idx[idx]
            i = items_idx[idx]
            pred = U[u] @ V[i]
            err = R[u, i] - pred
            total_loss += err**2
            U[u] += lr * (err * V[i] - reg * U[u])
            V[i] += lr * (err * U[u] - reg * V[i])

        rmse = np.sqrt(total_loss / len(users_idx))
        if epoch % 20 == 0:
            print(f"Epoch {epoch} | RMSE: {rmse:.4f}")

    return U, V

# Example with a small matrix
ratings_matrix = np.array([
    [5, 3, 0, 1, 4],
    [4, 0, 0, 2, 5],
    [1, 2, 5, 0, 0],
    [0, 1, 4, 5, 3],
    [3, 0, 2, 4, 0],
], dtype=float)

mask = ratings_matrix > 0

print("=== FunkSVD from scratch ===")
U, V = funkSVD(ratings_matrix, mask, n_factors=3, n_epochs=200, lr=0.005)

R_pred = U @ V.T
print("\nOriginal matrix (0 = unrated):")
print(ratings_matrix)
print("\nPredicted matrix:")
print(np.round(R_pred, 1))

# Recommendations for user 0
user = 0
unseen = np.where(~mask[user])[0]
rec_scores = R_pred[user, unseen]
top_item = unseen[np.argmax(rec_scores)]
print(f"\nTop recommendation for user {user}: item {top_item} (score {rec_scores.max():.1f})")

2. SVD with Surprise (MovieLens)

from surprise import Dataset, SVD, accuracy
from surprise.model_selection import train_test_split

# Load MovieLens 100k
data = Dataset.load_builtin("ml-100k", prompt=False)
trainset, testset = train_test_split(data, test_size=0.2, random_state=42)

# SVD = FunkSVD with additional optimizations
algo = SVD(n_factors=100, n_epochs=50, lr_all=0.005, reg_all=0.02)
algo.fit(trainset)

# Predictions on the test set
predictions = algo.test(testset)
accuracy.rmse(predictions, verbose=True)
accuracy.mae(predictions, verbose=True)

# Top 5 recommendations for a user
uid = "196"
all_items = {trainset.to_raw_iid(i) for i in trainset.all_items()}
user_ratings = trainset.ur[trainset.to_inner_uid(uid)]
rated_items = {trainset.to_raw_iid(i) for i, _ in user_ratings}
unrated = sorted(all_items - rated_items)

preds = [(iid, algo.predict(uid, iid).est) for iid in unrated]
top5 = sorted(preds, key=lambda x: x[1], reverse=True)[:5]
print(f"\nTop 5 recommendations for user {uid}:")
for iid, score in top5:
    print(f"  Item {iid}: {score:.2f}")

Hyperparameters

Advantages

1. No metadata needed: Collaborative filtering does not know the content of items. It works with ratings only, unlike content-based filtering.
2. Discovery of implicit patterns: Matrix Factorization discovers latent dimensions that no one has explicitly defined, capturing subtle correlations between preferences.
3. Scalability: Surprise and other frameworks can handle millions of users with hundreds of thousands of items.
4. Foundation of deep learning: The embeddings learned by Matrix Factorization are the basis of modern neural recommender systems (NCF, YouTube DNN).

Limitations

1. Cold Start: Impossible to recommend a new item (never rated) or to a new user (no ratings). Requires hybrid techniques.
2. Sparse matrix: When the vast majority of cells are empty, factorization becomes imprecise. Netflix’s sparsity level (>99%) is a major challenge.
3. Popularity bias: Popular items are over-recommended, creating a “rich get richer” effect that reduces recommendation diversity.
4. Evolving tastes: Preferences change (concept drift). A static model becomes obsolete within a few months.

4 concrete use cases

1. Video on Demand (Netflix, YouTube)

Netflix uses a combination of Matrix Factorization and deep learning for its recommendations. The “Cinematch” algorithm, initiated with SVD, has been replaced by a deep neural network but the factorization principle remains in the architecture. The 3 latent dimensions could correspond to “preference for dramatic genres,” “appetite for foreign films,” and “sensitivity to high ratings.”

2. E-commerce (Amazon, Cdiscount)

“Customers who bought this item also bought…” — this feature relies on item-item collaborative filtering. Amazon pioneered this approach in 1998. Product similarities are precomputed and updated daily, enabling real-time recommendations at massive scale.

3. Music (Spotify)

Spotify combines collaborative filtering (similar listening playlists) and audio analysis (BPM, key, energy). The “Discover Weekly” playlist generates 30 personalized tracks every week using listening co-occurrence in community playlists. Matrix Factorization on the user-tracks listening matrix captures implicit musical tastes.

4. Social Networks (LinkedIn, Twitter)

Friend or connection suggestions use a form of collaborative filtering: if you and another user have many connections in common, you are probably connected. Factorization of the interaction graph matrix allows prediction of which connections are most relevant.

Conclusion

Recommender systems using collaborative filtering and Matrix Factorization remain the most widely deployed technology in the industry. The Netflix Prize (1 million dollars for improving RMSE by 10%) showed that marginal gains in accuracy have a colossal economic impact.

Modern approaches (Neural Collaborative Filtering, Two-Tower Models) are extensions of the factorization principle: instead of a simple dot product u · v, a neural network learns a nonlinear function f(u, v). But the conceptual core remains the same: reducing the user-item space to compact latent representations.

See Also

• Mastering Binomial Product Divisors with Python: Complete Guide
• Mastering Composites with Primary Repunit Property with Python: Guide and Applications