CAAC Go Agent Benchmark
Which AI model makes the best customer-service agent?
A monthly benchmark from Crescendo Lab. Each edition runs the same 150 customer conversations through the production agent of CAAC Go, changes only the model, and has two AI judges from different vendors grade every reply. We publish the misses alongside the wins.
2 editions · 7 models tested · 150 cases in 7 languages per edition · latest run 7 November 2026
Latest edition · November 2026
Gemini 4 Argon catches GPT-6.1. Claude Opus 5.5 leaves them both behind.
Seven models. Gemini 4 Argon is level with GPT-6.1 Sol and ahead of GPT-5.6 Terra, which it trailed in October, and saves far more leads than any GPT model. Claude Opus 5.5 is the only model clearly ahead of the field.
- 1Claude Opus 5.589%
- 2Claude Sonnet 5.581%
- 3Gemini 4 Argon76%
- 4GPT-6.1 Sol76%
- 5Gemini 3.8 Flash75%
- 6GPT-5.574%
- 7GPT-5.6 Terra73%
Share of 150 cases fully correct: the right action, every claim supported by the business’s own information, in the visitor’s language. Gaps under about 7 points are within noise.
Read the November 2026 report →All editions
November 2026
Tested 7 November 2026 · 7 models · 150 cases
Gemini 4 Argon catches GPT-6.1. Claude Opus 5.5 leaves them both behind.
Seven models. Gemini 4 Argon is level with GPT-6.1 Sol and ahead of GPT-5.6 Terra, which it trailed in October, and saves far more leads than any GPT model. Claude Opus 5.5 is the only model clearly ahead of the field.
Claude Opus 5.5 89% · Claude Sonnet 5.5 81% · Gemini 4 Argon 76% · GPT-6.1 Sol 76% · Gemini 3.8 Flash 75% · GPT-5.5 74% · GPT-5.6 Terra 73%
October 2026
Tested 14 October 2026 · 2 models · 150 cases
How Gemini 4 performs inside the CAAC Go customer-service agent
The first edition: Gemini 4 Argon against GPT-5.6 Terra, the model CAAC Go runs in production. Argon saved more leads but made more unsupported claims and was slower; overall success was statistically tied.
GPT-5.6 Terra 73% · Gemini 4 Argon 69%
Analyses
8 November 2026
Which AI model actually saves the lead and books the visitor?
The 24 lead and booking cases of the November edition: three models save almost every one, the GPT models ask for a second contact channel first.
Every model, every edition
The cases and the agent stay the same from one edition to the next, so a model’s score can be compared over time. A single run moves by a few points on its own; each report tests whether a change is larger than that.
| Model | November 2026 | October 2026 |
|---|---|---|
| Claude Opus 5.5 | 89% | not tested |
| Claude Sonnet 5.5 | 81% | not tested |
| Gemini 4 Argon | 76% | 69% |
| GPT-6.1 Sol | 76% | not tested |
| Gemini 3.8 Flash | 75% | not tested |
| GPT-5.5 | 74% | not tested |
| GPT-5.6 Terra | 73% | 73% |
How every edition is run
- The same 150 cases. Six fictional businesses (a clinic, a SaaS product, an online store, a tutoring centre, a cleaning company and a café), 25 cases each, across nine task types and seven visitor languages. The mix follows anonymised patterns from real CAAC Go conversations; no customer data is used.
- The same agent. Every reply comes from CAAC Go’s production agent code, with the same prompt, tools and language rule for every model. Each business’s full knowledge base is in context, so the test measures the model, not search.
- Two judges, both must agree. GPT-5.5 and Gemini 3.1 Pro grade every reply. Whether the agent saved a lead or handed off is checked from what it did, not judged.
- Honest error bars. Each report gives confidence intervals and a case-by-case significance test, plus the limits of that run.
- Any model, one gateway. Models we cannot reach directly, such as Claude and GPT-6.1, run through Agent Lab AI Gateway, also built by Crescendo Lab.
The agent in every test
CAAC Go: an AI agent for your website, live in minutes
It answers visitors from your own website content, saves leads and booking requests, and hands the conversation to your team when it should. Paste your site’s link to try it on your own content. Free, no signup.