Selection bench · evaluation rev 4.3
Model Assessment
Requirements become hard filters; everything else becomes scoring weight. Effective context is used throughout — the retrieval-verified window, not the advertised ceiling.
Illustrative figuresGET /assessment/candidates · OSAI-Models selection bench, evaluation rev 4.3 (prototype figures) · 2026-09-14
11eligible · 0 filtered out
Primary workload
Weights effective context and grounding hardest — advertised ceilings are discounted.
1M / day60M / day
Drives the blended $/1M used for cost scoring.
Min effective context
TTFT ceiling
Data residency
Excludes closed models with no tuning path.
0 filtered out by requirements
Context fidelity model
Advertised is not usable
0–60%High-fidelity retrieval — near-full recall anywhere in the prompt.
60–80%Degradation — middle-of-context recall drops.
80–100%Unreliable — attention degradation and hallucination risk.