Self-Information of Popularity#
An information-theoretic weighting that rewards recommending less popular items.
Important
✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.
What it is#
A novelty measure borrowed from information theory. The self-information (surprisal) of recommending an item is the negative base-2 logarithm of its popularity — the probability that a random user has interacted with it. Rare events carry more information, so rare items score high.
The formula#
where \(\text{count}(i)\) is the number of users who consumed item \(i\) and \(|U|\) is the total number of users. A metric averages this over a top-\(N\) list and across users.
What it captures#
Popular items (high probability) have low self-information — they are unsurprising; long-tail items have high self-information — they are novel. It quantifies how much a recommendation tells the user something new.
Theme: Recommender Systems · All terminology
Hint
Mind map — connected ideas
Long-Tail Items · Relevance in Recommender Systems · Intra-List Diversity (ILD) · Catalog Coverage · Diminishing Utility · Dominating in Recommender Systems
Hint
More in Recommender Systems
Catalog Coverage · Cosine Similarity of Item Features · Diminishing Utility · Diversity (in Recommender Systems) · Dominating in Recommender Systems · Genre Overlap · Hit Rate (HR) · Intra-List Diversity (ILD) · Item Coverage · Jaccard index · Novelty (in Recommender Systems) · Relevance in Recommender Systems · User Coverage
See also
Source article Adapted (context, re-expressed) in our own words from: Self-Information of Popularity (insightful-data-lab.com).