Fleet Model Eval — 11-Test Skill Matrix
Every agent/model across the full 11-test battery, compared by task — real per-test results
2026-08-25 · 33 runs · 9 models · source: Fleet-Eval-2026-08 (authoritative)
| Model · Box | 1a HTML pricing | 1b is_prime | 2a kbd copy | 2b rewrite | 3a SVG logo | 3b CSS vars | 4a disk du | 4b crash/OOM | 5b transcript | 7a AZ sales-tax | 7b web-cost | Capable | Quality | Latency | Verdict |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mistral-small:24b midgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 18.9s | STRONG |
odin | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 15.6s | STRONG |
| gemma4:12b midgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 56.2s | STRONG |
overgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.5 | 41.2s | STRONG |
| qwen3.6:latest heimdall | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.5 | 56.2s | STRONG |
midgaard | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | FAIL | 0/11 | — | 180.9s | WEAK |
overgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 5.0 | 38.3s | STRONG |
| qwen3.5:4b asgard | PASS | PASS | FAIL | PASS | FAIL | PASS | PASS | PASS | PASS | FAIL | PASS | 8/11 | 4.5 | 131.1s | MIXED |
heimdall | PASS | PASS | FAIL | FAIL | FAIL | PASS | FAIL | PASS | PASS | FAIL | PASS | 6/11 | 5.0 | 29.4s | MIXED |
m1pro | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.8 | 186.3s | STRONG |
midgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.6 | 89.3s | STRONG |
odin | PASS | PASS | FAIL | FAIL | PASS | FAIL | PASS | PASS | PASS | FAIL | PASS | 7/11 | 5.0 | 57.7s | MIXED |
| gemma4:12b-it-qat asgard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.6 | 48.0s | STRONG |
heimdall | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.6 | 22.4s | STRONG |
midgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.5 | 50.9s | STRONG |
odin | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 39.2s | STRONG |
overgaard | PASS | PASS | FAIL | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 10/11 | 4.6 | 46.4s | STRONG |
| gemma3:4b asgard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 10.6s | STRONG |
heimdall | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.1 | 4.2s | STRONG |
m1pro | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.1 | 15.0s | STRONG |
midgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 7.7s | STRONG |
odin | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 6.3s | STRONG |
overgaard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.1 | 4.9s | STRONG |
| llama3.1:8b asgard | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.1 | 8.8s | STRONG |
odin | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 11/11 | 4.3 | 6.1s | STRONG |
| qwen3.5:9b asgard | PASS | PASS | FAIL | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 10/11 | 4.8 | 171.5s | STRONG |
heimdall | PASS | PASS | FAIL | PASS | FAIL | PASS | FAIL | FAIL | PASS | PASS | PASS | 7/11 | 4.7 | 42.3s | MIXED |
midgaard | PASS | PASS | FAIL | PASS | FAIL | PASS | FAIL | FAIL | PASS | PASS | PASS | 7/11 | 4.7 | 139.1s | MIXED |
odin | PASS | PASS | FAIL | PASS | FAIL | PASS | PASS | FAIL | PASS | PASS | PASS | 8/11 | 4.8 | 78.1s | MIXED |
overgaard | PASS | PASS | FAIL | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 10/11 | 4.8 | 68.4s | STRONG |
| qwen3.8:27b heimdall | PASS | PASS | FAIL | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 10/11 | 4.6 | 173.7s | STRONG |
midgaard | FAIL | PASS | FAIL | PASS | PASS | PASS | PASS | PASS | PASS | PASS | PASS | 9/11 | 4.3 | 243.8s | MIXED |
overgaard | PASS | PASS | FAIL | PASS | PASS | PASS | PASS | PASS | PASS | PASS | FAIL | 9/11 | 4.6 | 106.7s | MIXED |
| PASS rate by task | 94% | 97% | 61% | 91% | 82% | 94% | 88% | 88% | 97% | 88% | 94% | of 33 runs | |||
PASS
FAIL (honest N/A where capable=false)
no record
Capable = tests passed of 11. Quality = avg score 1–5. Latency = avg seconds/test.