Interactive · Week 8 · Section 4

End-to-end correctness is bounded by upstream metrics.

A synthetic 100-query eval set. Each query has a hidden retrieval difficulty and a hidden generation difficulty. Move the sliders to set how capable your retrieval and generation are. Watch how end-to-end correctness emerges from the product of the two — and how the bottleneck shifts as you improve one without the other. Most teams overinvest in the easier-to-improve metric while the bottleneck silently caps performance. The visceral lesson: improving the wrong metric doesn't move the user-visible number.

Recall@K

—

retrieval found right chunk

Generation correctness

—

given retrieval succeeded

E2E correctness

—

user-visible quality

Citation accuracy

—

cited source matches claim

Retrieval capability 0.70

embeddings · reranker · hybrid · chunking

Generation capability 0.70

model · prompt · context handling

Citation capability 0.85

grounding · structured output · prompt

100 eval queries · per-query outcome

hover any cell to see its difficulty

End-to-end correct retrieval found the right chunk and generation answered correctly — queries

Generation gap retrieval gave correct context but generation produced wrong answer — queries

Retrieval gap retrieval failed to find the right chunk; answer can't be correct — queries

What's happening here

Each query has a hidden retrieval difficulty and a hidden generation difficulty (independent uniform random in [0,1], deterministically seeded). A query passes retrieval iff retrieval_capability ≥ retrieval_difficulty; it passes generation iff retrieval passed AND generation_capability ≥ generation_difficulty. End-to-end correctness equals the joint pass rate. Citation accuracy is computed independently as the fraction of E2E-correct queries where citation difficulty falls under citation_capability. Source-of-truth lives as <EvalMetricsExplorer/> in the curriculum codebase.