Enhancing Image Recommendations: The DPP Approach to Quality and Diversity

1. Context & Problem Statement

In today’s mobile-first world, users face constant information overload, with countless applications competing for their limited attention. Glance product that operates on lock screen — the first interface users encounter before even unlocking their devices — has emerged as a prime digital real estate for meaningful content delivery.

In content recommendation systems, there exists a fundamental tension between maximizing relevance and ensuring diversity. While relevance ensures users see content that matches their query or interests, diversity provides a broader perspective that captures different facets of the same topic.

Diversity in search results offers several critical benefits:

• it reduces information bubbles by exposing users to varied perspectives; increases the likelihood of serendipitous discovery
• ensures comprehensive coverage of complex topics
• ultimately leads to higher user satisfaction by providing options that address different interpretations of the same query

This challenge of maintaining a balance between relevance and diversity is particularly acute in visual content recommendation, where images must not only be relevant but also offer different visual perspectives on a topic. For products like Glance who rely on compelling imagery to engage users in split-second decisions, solving the quality-diversity equation becomes mission-critical. The effectiveness of our content delivery hinges on selecting the perfect images to accompany news articles — images that must simultaneously capture attention, convey information, and represent the full context of stories.

Glance is not a UGC platform. Therefore, the content publishing process becomes crucial for user’s experience. Our content editorial system has following expectations from our image recommendation system:

1. Maintain high relevance to the article’s content
2. Provide sufficient diversity to give editors meaningful choices
3. Follow the guidelines set by various Original Equipment Manufacturers (Xiaomi, Realme, Oppo, Samsung, etc.)
4. Respect regional and cultural sensitivities across diverse markets

Glance Publishing Process

Traditional top-k retrieval methods often returned highly relevant but redundant results — multiple images that were too similar to each other. For example, a news article about a sporting event might generate recommendations showing nearly identical angles of the same moment, rather than capturing the event’s full narrative range.

Our image recommendation system, while adept at identifying relevant content, suffered from a critical limitation: it consistently delivered visually and semantically similar images by prioritizing individual relevance scores without considering similarity between results. This lack of diversity severely restricted our editorial team’s options — despite each image being technically relevant, the homogeneous recommendation set failed to provide meaningful alternatives, ultimately undermining the quality and versatility of our content delivery.

1.1 Addressing the Diversity Challenge

The industry employs several methods to introduce diversity in recommendation systems.

Maximum Marginal Relevance (MMR) uses a greedy approach to balance relevance and diversity but often lacks global optimization.

Clustering techniques group similar items to select diverse representatives but struggle with optimal grouping definitions.

Re-ranking with explicit diversity constraints offers another solution but requires complex tuning and doesn’t generalize well.

Neural diversity-aware models show promise but demand extensive training data.

After evaluating these options, we selected Determinantal Point Processes (DPP) for our image recommendations, as it provides a mathematically elegant framework that naturally optimizes both relevance and diversity simultaneously, offering superior performance without treating diversity as an afterthought.

2. Solution: Implementing k-Determinental Point Processes for Balanced Diversity-Relevance

2.1 Understanding k-DPP: The Mathematical Foundation

After evaluating several approaches, we implemented k-Determinental Point Processes (k-DPP), a probabilistic model specifically designed to select diverse subsets while maintaining quality. Unlike standard top-k retrieval, which considers items independently, k-DPP models the quality of an entire subset collectively, accounting for both individual item relevance and inter-item similarities.

A k-DPP defines a probability distribution over all possible subsets of size k from a candidate pool, with higher probability assigned to subsets containing high-quality items that are different from each other. This approach has strong theoretical foundations in random matrix theory and provides elegant mathematical guarantees for diversity-aware selection.

Diagram referred from “Determinantal Point Processes for Machine Learning”

2.2 Implementation Architecture

Our implementation consisted of several key components:

1. Image Embedding Generation: We generated semantic embeddings for each image candidate using a pre-trained vision model
2. Quality Score Computation: Each image received a relevance score based on its semantic similarity to the article
3. Kernel Matrix Construction: We built a kernel matrix combining quality scores and similarity information
4. k-DPP Sampling: We sampled the optimal subset of size k using efficient k-DPP algorithms

2.3 Kernel Construction

The kernel matrix is the heart of our k-DPP implementation, encoding both quality and similarity information.

def compute_kernel(embeddings, scores, quality_weight=0.5):
    # Normalize embeddings for cosine similarity
    normed_embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)

    # Compute similarity matrix
    similarity = normed_embeddings @ normed_embeddings.T

    # Normalize quality scores
    quality = (scores - scores.min()) / (scores.max() - scores.min())
    quality_matrix = np.outer(quality, quality)

    # Combine quality and diversity
    L = quality_matrix * ((1 - quality_weight) * np.eye(len(embeddings)) + 
                         quality_weight * similarity)
    return L

The quality_weight parameter controls the relevance-diversity trade-off: values near 0 prioritize relevance (independent item selection), values near 1 emphasize diversity (considering inter-item similarity), and we typically start at 0.5 for balance. We adjust this parameter based on content category needs—higher for visually diverse categories (sports) and lower where relevance dominates (business news).

2.4 k-DPP Sampling Process

Our sampling process followed this pseudo-algorithm:

def sample_dpp(self, kernel: np.ndarray, k: int) -> np.ndarray:
    """
    Sample k items using k-DPP with efficient dual representation.
    """
    N = kernel.shape[0]
    k = min(k, N)  # Ensure k is not larger than N

    # Eigendecomposition with numerical stability
    eigenvals, eigenvecs = np.linalg.eigh(kernel)
    eigenvals = np.maximum(eigenvals, 0)  # Ensure non-negative eigenvalues

    # Sort in descending order
    idx = eigenvals.argsort()[::-1]
    eigenvals = eigenvals[idx]
    eigenvecs = eigenvecs[:, idx]

    # Compute elementary symmetric polynomials
    E = self._compute_elementary_symmetric(eigenvals, k)

    # Sample k eigenvectors
    J = self._sample_k_eigenvectors(eigenvals, k, E)

    # Compute probabilities for all items
    V = eigenvecs[:, list(J)]
    probs = np.sum(V * V, axis=1)

    return probs

def _sample_k_eigenvectors(self, eigenvals: np.ndarray, k: int, E: np.ndarray) -> set:
    """
    Sample k eigenvectors using elementary symmetric polynomials.
    """
    N = len(eigenvals)
    J = set()
    remaining = list(range(N))
    eps = 1e-10  # Small constant for numerical stability

    while len(J) < k:
        # Compute conditional probabilities
        probs = []
        for i in range(len(remaining)):
            rem_size = len(remaining)
            try:
                if len(J) == 0:
                    # First selection
                    numerator = eigenvals[remaining[i]] * E[k-1, rem_size-1]
                    denominator = E[k, rem_size]
                else:
                    # Subsequent selections
                    numerator = eigenvals[remaining[i]] * E[k-len(J)-1, rem_size-1]
                    denominator = E[k-len(J), rem_size]

                # Handle division by zero
                if abs(denominator) < eps:
                    prob = 0.0
                else:
                    prob = numerator / denominator

            except IndexError:
                prob = eigenvals[remaining[i]] if len(J) == k-1 else 0

            probs.append(max(0, prob))  # Ensure non-negative

        # Normalize and handle numerical issues
        probs = np.array(probs)
        sum_probs = np.sum(probs)

        if sum_probs < eps:
            # If all probabilities are effectively zero, use uniform distribution
            probs = np.ones(len(probs)) / len(probs)
        else:
            probs = probs / sum_probs

        # Ensure no NaN values
        if np.any(np.isnan(probs)):
            probs = np.ones(len(probs)) / len(probs)

        # Sample index
        selected = np.random.choice(len(remaining), p=probs)
        J.add(remaining[selected])
        remaining.pop(selected)

    return J

def _compute_elementary_symmetric(self, eigenvals: np.ndarray, k: int) -> np.ndarray:
    """
    Compute elementary symmetric polynomials with numerical stability.
    """
    N = len(eigenvals)
    E = np.zeros((k + 1, N + 1))
    E[0, :] = 1

    for l in range(1, k + 1):
        for n in range(l, N + 1):
            # Use log-sum-exp trick for numerical stability
            with_nth = eigenvals[n - 1] * E[l - 1, n - 1]
            without_nth = E[l, n - 1]
            E[l, n] = with_nth + without_nth

    return E

def diversity_reranker(items, k, quality_weight=0.5):
    # Extract embeddings and scores
    embeddings = np.array([item['embedding'] for item in items])
    scores = np.array([item['score'] for item in items])

    # Compute DPP kernel
    kernel = compute_kernel(embeddings, scores, quality_weight)

    # Sample k items
    dpp_scores = sample_dpp(kernel, k)

    # Return top-k reranked items
    indices = np.argsort(dpp_scores)[::-1][:k]
    return [items[i] for i in indices]

🤔 What is Efficient Dual Representation? Our k-DPP sampling is based on dual representation. This approach transforms the k-DPP sampling problem from the computationally expensive primal approach (working directly with kernel matrices) to the more efficient dual approach using eigendecomposition. This technique represents the kernel’s information through its eigenvalues and eigenvectors, significantly reducing complexity.

The approach relies on Elementary Symmetric Polynomials (ESPs) — mathematical functions that compute sums of products of eigenvalues (e.g., e₂(λ₁,λ₂,λ₃) = λ₁λ₂ + λ₁λ₃ + λ₂λ₃) — to efficiently calculate sequential selection probabilities. By implementing ESPs through dynamic programming, we achieve O(k×N) time complexity versus the exponential complexity of naive approaches, making this method feasible for large-scale production systems while preserving the mathematical guarantees of exact k-DPP sampling.

2.5 Implementation Challenges

• Computational Efficiency: Addressed O(n³) complexity through pre-filtering to 3–5× the target size, optimized matrix operations, and approximation methods for large collections.
• Parameter Tuning: Balanced quality-diversity trade-off via systematic testing, starting at quality_weight=0.5, conducting A/B tests, and incorporating editorial feedback.
• Numerical Stability: Solved eigendecomposition issues with nearly identical images by adding small diagonal terms, implementing stable decomposition methods, and enhanced normalization.

2.6 Why DPP Over Learning-to-Rank (LTR) Methods?

DPP outperformed traditional Learning-to-Rank methods for three key reasons:

• It naturally balances relevance and diversity in a unified framework, unlike LTR's focus on relevance metrics alone
• It elegantly encodes both quality and diversity in the kernel matrix without requiring extensive feature engineering
• It operates effectively with embedding similarities and relevance scores without needing labeled training data—crucial given our limited human-labeled diverse image sets.

While LTR might excel in pure relevance scenarios, our offline pipeline accommodated DPP's computational requirements, making it optimal for diversity-aware image selection.

3. Impact: Transforming Content Selection

Quantitative Results:

• 10% increase in diversity while maintaining relevance within 5% of traditional top-k
• 3x increase in publishing rate (February 2025)

Qualitative Improvements:

• Enhanced visual storytelling through greater perspective diversity
• Better regional appropriateness and significantly higher editorial satisfaction

Performance: Added ~100ms processing time for 1000 candidates with O(n²) memory scaling — well within operational requirements.

Acknowledgements

This project’s success would not have been possible without the collaborative efforts of numerous teams and individuals. Special thanks to:

• The Machine Learning team for their expertise in implementing and optimizing the k-DPP algorithm
• The Editorial team for providing valuable feedback throughout the development process
• Academic researchers whose work on determinental point processes provided the theoretical foundation for our implementation

References

• Kulesza, A., & Taskar, B. (2012). “Determinantal Point Processes for Machine Learning”
• Kathuria, A., et al. (2016). “Efficient Sampling for k-DPP”
• Chen, L., et al. (2018). “Fast k-DPP Sampling”