🔎 Team Draft Interleaving (TDI)

Team Draft Interleaving (TDI)#

Interleaving where rankers alternately draft items into one list for fair comparison.

Important

✨ AI-generated content. This page was written with the assistance of an AI language model and is provided as a learning aid. Despite careful review, it may still contain mistakes, omissions, or out-of-date information. Whether you are new to the topic, a team lead, or a senior practitioner, treat it as a starting point rather than an authoritative reference: read it critically and independently verify anything you act on (code, commands, figures, and factual claims) against official documentation and primary sources before relying on it.

What it is#

Team-draft interleaving (TDI) is an online method for comparing two ranking algorithms A and B. Like balanced interleaving it merges both into one list shown to users, but it fills the list by a draft pick — exactly like two captains choosing players for a team — deciding which algorithm contributes the next slot.

Why use it#

A/B testing splits traffic and needs large samples; balanced interleaving can favour one ranker depending on overlap and order. TDI randomises the draft so each algorithm gets an equal, unbiased chance to place its items.

How it works#

Take the top-k from A and B; randomly pick who drafts first; then alternate: each algorithm in turn adds its highest-ranked item not already in the list, until the interleaving is full. Show it, and attribute each click to whichever algorithm drafted that item — attribution is deterministic and unambiguous.

Example#

With A = [A1, A2, A3, A4] and B = [B1, B2, B3, B4], a draft might yield [A1, B1, A2, B2, A3, B3, A4, B4]. A click on B2 credits algorithm B, because B drafted it.

Strengths and limits#

TDI is fair (randomised drafting removes systematic bias), efficient (fewer users than A/B testing to detect a difference), gives clear click ownership, and is sensitive to small gaps — which made it an industry standard. Its limits: it compares only two algorithms at once, still assumes clicks equal relevance (noisy), and needs care with ties and overlapping results.

Theme: Ranking & Interleaving · All terminology

Hint

Mind map — connected ideas

Balanced Interleaving · Probabilistic Interleaving · Ranking Algorithms · Online Experimentation Platforms · A/B Testing · NDCG (Normalized Discounted Cumulative Gain)

Hint

More in Ranking & Interleaving

Balanced Interleaving · DCG (Discounted Cumulative Gain) · Interleaving Tests · Mean Average Precision (MAP) · NDCG (Normalized Discounted Cumulative Gain) · Probabilistic Interleaving · Ranking Algorithms · TREC (Text REtrieval Conference)