PEICO

A benchmark for a customer sales agent at a fictional insurance company.

Each of the 80 tasks is a live back-and-forth conversation with a simulated customer against a real database, and the agent is graded on the full chat: the database must end in exactly the right state and a rubric judge must approve the conversation. Inspired by τ-bench.

Leaderboard
80 tasks × 1 trial · click a model to read its conversationsnote: to keep costs down on this personal project, models were only run once on each task (k=1)
#ModelScoreTrialsCost$ / taskMedian timeFailure modes
1AnthropicClaude Sonnet 5 (adaptive, high)anthropic/claude-sonnet-596.280$5.50$0.0747s1 changeset · 2 judge
2AnthropicClaude Opus 4.8 (adaptive, high)anthropic/claude-opus-4-8@adaptive-high95.080$13.88$0.1758s1 changeset · 3 judge
3AnthropicClaude Opus 4.8 (no thinking)anthropic/claude-opus-4-892.580$11.24$0.1443s3 changeset · 3 judge
4OpenAIGPT-5.5 (medium)openai/gpt-5.592.580$14.02$0.1847s2 changeset · 4 judge
5OpenAIGPT-5.4 (none)openai/gpt-5.490.080$5.35$0.0733s3 changeset · 5 judge
6AnthropicClaude Fable 5 (adaptive, high)anthropic/claude-fable-588.880$29.69$0.3784s5 changeset · 3 judge · 1 refused
7GoogleGemini 3.1 Flash-Lite (minimal)google_genai/gemini-3.1-flash-lite83.880$1.25$0.0219s5 changeset · 1 incomplete · 7 judge · 1× iteration cap
8AnthropicClaude Haiku 4.5 (no thinking)anthropic/claude-haiku-4-570.080$1.71$0.0230s7 changeset · 17 judge
9OpenAIGPT-5.4 mini (none)openai/gpt-5.4-mini70.080$1.84$0.0223s11 changeset · 1 incomplete · 12 judge
10GoogleGemini 3.5 Flash (medium)google_genai/gemini-3.5-flash68.880$31.03$0.3977s11 changeset · 1 incomplete · 13 judge · 9× iteration cap
11OpenAIGPT-5.4 nano (none)openai/gpt-5.4-nano57.580$0.67$0.0132s20 changeset · 14 judge
Score vs. cost
Agent API cost for the full bench (log scale). Per-token pricing is a poor predictor of per-task cost
Score vs. speed
Median wall-clock time for a full task, customer simulator and grading included
AnthropicClaude Fable 5 (adaptive, high)Claude Opus 4.8 (adaptive, high)Claude Opus 4.8 (no thinking)Claude Sonnet 5 (adaptive, high)Claude Haiku 4.5 (no thinking)
OpenAIGPT-5.5 (medium)GPT-5.4 (none)GPT-5.4 mini (none)GPT-5.4 nano (none)
GoogleGemini 3.5 Flash (medium)Gemini 3.1 Flash-Lite (minimal)
Outcomes
Where trials ended, out of 80 per model. Every failure keeps its typed cause

Every case × every model

A cell is green only if the model passed the case; failed cells are colored by the failure type. Click a cell for that model's full transcript and grader verdicts, or a case name for every model's failure reasons. Sorted hardest-first, the top rows are the cases that separate models.

80 of 80 casesAnthropicClaude Sonnet 5adaptive, high77/80AnthropicClaude Opus 4.8adaptive, high76/80AnthropicClaude Opus 4.8no thinking74/80OpenAIGPT-5.5medium74/80OpenAIGPT-5.4none72/80AnthropicClaude Fable 5adaptive, high71/80GoogleGemini 3.1 Flash-Liteminimal67/80AnthropicClaude Haiku 4.5no thinking56/80OpenAIGPT-5.4 mininone56/80GoogleGemini 3.5 Flashmedium55/80OpenAIGPT-5.4 nanonone46/80
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 0/1 passed · Failed: judge rubricGPT-5.4 (none): 0/1 passed · Failed: judge rubricClaude Fable 5 (adaptive, high): 0/1 passed · Failed: judge rubricGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.5 (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: wrong DB end-stateClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: judge rubricGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 0/1 passed · Failed: judge rubricGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 0/1 passed · Failed: judge rubricClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 0/1 passed · Failed: judge rubricClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 0/1 passed · Failed: judge rubricClaude Opus 4.8 (adaptive, high): 0/1 passed · Failed: judge rubricClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: wrong DB end-stateClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 0/1 passed · Failed: wrong DB end-stateClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 0/1 passed · Failed: judge rubricClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: judge rubricGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 0/1 passed · Failed: judge rubricGPT-5.4 (none): 0/1 passed · Failed: judge rubricClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: wrong DB end-stateClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 0/1 passed · Failed: wrong DB end-stateClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: incompleteGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 0/1 passed · Failed: judge rubricGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: incompleteClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 0/1 passed · Failed: judge rubricClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 0/1 passed · Failed: judge rubricClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: judge rubricClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 0/1 passed · Failed: judge rubricClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: incompleteGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 0/1 passed · Failed: judge rubric
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: wrong DB end-stateClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 0/1 passed · Failed: wrong DB end-stateClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: judge rubricGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 0/1 passed · Failed: wrong DB end-stateClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · RefusedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 0/1 passed · Failed: wrong DB end-stateGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 0/1 passed · Failed: judge rubricGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 0/1 passed · Failed: wrong DB end-state
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: wrong DB end-stateGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 0/1 passed · Failed: judge rubricGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 0/1 passed · Failed: wrong DB end-stateGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed
Claude Sonnet 5 (adaptive, high): 1/1 passedClaude Opus 4.8 (adaptive, high): 1/1 passedClaude Opus 4.8 (no thinking): 1/1 passedGPT-5.5 (medium): 1/1 passedGPT-5.4 (none): 1/1 passedClaude Fable 5 (adaptive, high): 1/1 passedGemini 3.1 Flash-Lite (minimal): 1/1 passedClaude Haiku 4.5 (no thinking): 1/1 passedGPT-5.4 mini (none): 1/1 passedGemini 3.5 Flash (medium): 1/1 passedGPT-5.4 nano (none): 1/1 passed