Benchmark Lab

EcoBench v1: … original tasks across 7 categories, each with a deterministic check. Calibration cases warm the router; held-out test cases measure it. Every condition runs the same cases so results are paired.

Connecting…

Configure a run

Same cases, same validators, temperature 0 for every condition.

Conditions (first fixed:large is the baseline)

fixed:small always Nanofixed:medium always Superfixed:large always Ultra (baseline)adaptive router, cold startadaptive_warm router + calibration evidence

SplitLimit casesRepeats

0 cases × 3 conditions × 1 repeat(s). Worst case 0 calls (cap …). Spend stops at the daily budget.

Runs

No runs yet.