Gemini 4 Argon catches GPT-6.1. Claude Opus 5.5 leaves them both behind.
We ran seven models inside the same CAAC Go website agent on 150 customer conversations. Gemini 4 Argon has caught up with OpenAI: it passed 76% of cases, level with GPT-6.1 Sol and ahead of GPT-5.6 Terra, the model it trailed in October. It also saved far more leads than any GPT model. It has not caught up on accuracy or speed: it added details the business never stated in 10% of replies and was among the slowest. Claude Opus 5.5 was the one clear winner, at 89%.
Published by Crescendo Lab · Tested 2026-11-07 · 150 cases × 7 models · two independent AI judges · three models run through Agent Lab AI Gateway
What is Gemini 4 Argon?
The model
The Gemini 4 model from Google that developers can use today, on Vertex AI and the Gemini API. Its model ID is gemini-4-argon. We called it through Vertex AI’s global endpoint.
Same as October
Our October edition called it “Gemini 4”. It is the same model, run on the same 150 cases.
Not Carbon
Google announced a second Gemini 4 model, Gemini 4 Carbon, on 28 October. So far it is open only to selected security teams and partners, with no public model ID, so it is not in this test.
Price
US$2 per million input tokens and US$10 per million output tokens until 2026-11-30, then US$4 and US$20.
Gemini 4 Argon at a glance
01
It has caught up with OpenAI. Argon fully passed 114/150 cases (76%), exactly as many as OpenAI’s newest model in this test, GPT-6.1 Sol, and more than GPT-5.6 Terra (73%). In October it trailed Terra. Only Claude Opus 5.5 (89%) is clearly ahead.
02
It saves leads; the GPT models mostly do not. When a visitor left contact details and asked to be followed up, Argon saved the lead or booking in 21/24 cases. GPT-6.1 Sol managed 7/24 and GPT-5.6 Terra 8/24, usually because they asked for one more contact channel before saving. Only the two Claude models did better.
03
It adds details the business never said. 10% of Argon’s replies contained a business claim the knowledge did not support, against 0.7% for GPT-5.6 Terra. On questions built to lure a wrong answer it held the line in 8/18 cases, against 16/18 for Terra.
04
It is slow, and slower than in October. Argon’s median reply took 5.7 s and the slowest 5% took 28.1 s or more. In our October run the median was 3.0 s. GPT-5.6 Terra answered in 2.0 s.
05
It writes about 2.7 times as much. Its average reply is 271 characters, against 101 for Terra, often listing other services and asking for an email. The judges scored reply quality about the same (4.66 vs 4.55 out of 5).
06
It costs more once the launch price ends. At the tokens it used here, Argon costs US$12.29 per 1,000 conversations at list price and US$6.14 at its introductory price, which ends on 2026-11-30. GPT-5.6 Terra costs US$5.04.
Our take
Argon’s strength is real, and it is acting, not knowing
The test for any new model is whether its advantage survives a second run. Argon’s does. In every lead and booking case, in both editions, Argon called the save action: 24 of 24 in October and 24 of 24 in November. GPT-5.6 Terra called it in 10 and 8, and OpenAI’s newest, GPT-6.1 Sol, in 7. Comparing the two case by case this month, Argon passed 13 lead and booking cases that Terra failed, and Terra passed none that Argon failed (p < 0.001). That is not run-to-run noise. It is how the model behaves.
48/48
lead and booking cases where Argon saved the request, across both editions
18/48
the same for GPT-5.6 Terra, which usually asked for one more contact channel first
13 to 0
lead and booking cases only Argon passed, against only Terra, this month
The same eagerness explains its weak spots. Argon never skipped a save and never missed a case that needed a person. When it fails, it fails by doing too much. In 13 of its failed cases it asked for contact details the visitor had not offered (Terra: 6), and in 16 it added a claim the business never made (Terra: 1). Even its 3 failed lead cases saved the lead. Each lost the point for one invented sentence, such as “Pro plan includes unlimited forms”.
What this means: pick Argon where an action has to happen on the first reply, such as a quote request, a callback or a booking. Give it a knowledge base that spells out limits and exceptions, because Argon fills silence with plausible detail. Where every sentence has to be exactly right, GPT-5.6 Terra and GPT-6.1 Sol are the safer choice (0.7% and 1.3% unsupported claims).
Good fit
Sales and lead-capture flows, where saving the visitor’s details matters more than a short reply.
Booking requests: it recorded 8/8 of them.
Teams that will check new leads by hand anyway, so an extra sentence is cheap to catch.
Watch out
Policy, pricing and medical questions, where one invented condition can mislead a customer.
Live chat where people expect an answer in two or three seconds.
Budget after 2026-11-30, when its list price doubles.
All seven models
Same agent, same 150 cases, same judges. Only the model changes. Models marked † ran through Agent Lab AI Gateway.
Gemini 4 Argon
Claude Opus 5.5 †
Claude Sonnet 5.5 †
GPT-6.1 Sol †
GPT-5.6 Terra
GPT-5.5
Gemini 3.8 Flash
Fully correct
76% (69%–82%)
89% (83%–93%)
81% (74%–86%)
76% (69%–82%)
73% (66%–80%)
74% (66%–80%)
75% (68%–82%)
Replies with an unsupported claim
10.0%
4.0%
4.7%
1.3%
0.7%
4.0%
10.0%
Held the line on fabrication bait
8/18
12/18
13/18
16/18
16/18
14/18
15/18
Saved the lead or booking
21/24
24/24
22/24
7/24
8/24
9/24
12/24
Handed off when it should
100%
100%
88%
92%
88%
88%
92%
Replied in the visitor's language
100%
100%
100%
99%
96%
98%
91%
Stayed safe under prompt attacks
8/12
8/12
9/12
11/12
11/12
11/12
6/12
Median reply time
5.7 s
4.2 s
3.9 s
3.2 s
2.0 s
2.8 s
6.7 s
Slowest 5% of replies
28.1 s
8.0 s
7.4 s
10.9 s
4.9 s
5.9 s
13.5 s
Cost per 1,000 conversations
US$12.29 (US$6.14 intro)
US$21.65
US$10.32
US$5.24
US$5.04
US$14.13
US$8.42 (US$4.21 intro)
One model wins overall: Claude Opus 5.5. Compared case by case, it beat every other model by a margin very unlikely to be chance (p = 0.002 or lower against each; p = 0.01 against Claude Sonnet 5.5). Among the other six, no gap is statistically significant: Gemini 4 Argon against GPT-5.6 Terra is p = 0.64, and against GPT-6.1 Sol it is a dead heat, 114 cases each. Below the top, the useful differences are in how each model fails, how fast it answers and what it costs.
Success rate by task
Share of cases where the reply did the right thing, was fully supported by the business’s own information, and was in the visitor’s language.
We ran the same 150 cases on 2026-10-14 with Gemini 4 Argon and GPT-5.6 Terra. Three things moved.
Argon caught up with GPT-5.6 Terra. In October Argon scored 69% to Terra’s 73% and lost more head-to-head cases than it won (21 to 27). This month it scored 76% to 73% and won more than it lost (22 to 18). Terra itself held still: 8 cases got better and 8 got worse. Argon moved one way: 18 better, 8 worse. With the test unchanged, that points to a change on Google’s side, though one run cannot prove it (p = 0.08).
Argon got slower. Its median reply went from 3.0 s to 5.7 s, measured from the same place on the same endpoint.
GPT-5.6 Terra made fewer things up. Unsupported claims fell from 4% to 0.7%. Neither the cases nor the agent’s instructions changed between the runs, so this is run-to-run variance or an update on the provider’s side that we cannot see.
By visitor language
About 65% of the messages real CAAC Go visitors send contain Chinese, so Traditional Chinese is the second-largest group here.
Visitor language
Cases
Gemini 4 Argon
Claude Opus 5.5
Claude Sonnet 5.5
GPT-6.1 Sol
GPT-5.6 Terra
GPT-5.5
Gemini 3.8 Flash
English
68
69%
88%
78%
74%
66%
69%
74%
Traditional Chinese
60
82%
87%
82%
77%
77%
77%
75%
Japanese
6
100%
100%
100%
100%
100%
83%
100%
Malay
6
100%
100%
100%
67%
83%
83%
67%
Thai
5
60%
100%
80%
80%
80%
80%
80%
Hinglish
4
50%
75%
50%
75%
75%
75%
75%
Korean
1
100%
100%
100%
100%
100%
100%
100%
Gemini 4 Argon vs GPT-5.6 Terra, side by side
Kopi & Crumb · café · Save a lead
“Hi, could someone send me a quote for 50 sourdough loaves for our office event next month? I'm Priya, priya.n@example.com”
Gemini 4 ArgonPass
Hi Priya! I have saved your contact details and passed your request for a quote on 50 sourdough loaves for next month's office event to our team. Normally our sourdough loaves are RM18 each, and someone from our team will email you at priya.n@example.com with a bulk quote and availability soon. Please let me know if I can help you with anything else in the meantime!
saved: capture_lead
GPT-5.6 TerraFail
You’d like a quote for 50 sourdough loaves for an office event next month. Please share your best phone number, and our team will follow up with the quote.
The visitor gave a name, an email and the request. Gemini 4 Argon saved the lead and confirmed it. GPT-5.6 Terra asked for a phone number first and saved nothing on that turn.
Both say the store is online only. Gemini 4 Argon then adds that items can be returned within 14 days, which leaves out the store's “Final sale” exception, so a sale-item buyer would be misled.
Both decline the filler treatment correctly. Gemini 4 Argon also lists other services and asks for an email, which the judges marked as pushing for contact details the visitor did not offer.
The other six models
Claude Opus 5.5 is the clear leader. It passed 133/150 cases, answered every plain question correctly (36/36), saved every lead and booking (24/24) and kept unsupported claims to 4.0%. Its weak spot is the same as Argon’s: on questions it should simply decline, it often asked for an email anyway, so it held the line on fabrication bait in only 12/18. It is also the most expensive model here, at US$21.65 per 1,000 conversations, about four times GPT-5.6 Terra.
Claude Sonnet 5.5 came second at 81% for about half of Opus’s price (US$10.32). It saved 22/24 leads and bookings, but it was weak at turning down off-topic requests (5/12). Its lead over the field is not statistically significant (p = 0.10 against Terra).
GPT-6.1 Sol, OpenAI’s newest model, tied with Argon at 76%. It was as careful as Terra (1.3% unsupported claims, 16/18 on bait) and costs about the same (US$5.24), but like Terra it rarely saved a lead on the first turn (7/24). The two judges agreed on 97% of its replies, the highest of any model.
GPT-5.6 Terra is the one CAAC Go runs in production. It made the fewest unsupported claims (0.7%), answered fastest (2.0 s median) and cost least (US$5.04 per 1,000 conversations). Its weak spot is saving leads on the first turn.
GPT-5.5 scored about the same as Terra (74%) with more unsupported claims (4.0%), slower replies and nearly three times the cost (US$14.13). There is no reason to choose it over Terra or GPT-6.1 Sol for this work.
Gemini 3.8 Flash scored 75% and is cheaper than Argon (US$8.42 list, US$4.21 intro). It was the slowest at the median (6.7 s), and in 10 of 150 cases, all of them lead or booking requests, it returned an empty reply even after a retry. We count those as failures.
Cost and speed
Gemini 4 Argon
Claude Opus 5.5
Claude Sonnet 5.5
GPT-6.1 Sol
GPT-5.6 Terra
GPT-5.5
Gemini 3.8 Flash
Cost per 1,000 conversations (list price)
US$12.29
US$21.65
US$10.32
US$5.24
US$5.04
US$14.13
US$8.42
Cost per 1,000 conversations (intro price)
US$6.14 until 2026-11-30
–
–
–
–
–
US$4.21 until 2026-12-31
Median reply time
5.7 s
4.2 s
3.9 s
3.2 s
2.0 s
2.8 s
6.7 s
Slowest 5% of replies
28.1 s
8.0 s
7.4 s
10.9 s
4.9 s
5.9 s
13.5 s
Input tokens per turn
2,475
3,982
3,895
2,108
1,991
1,981
2,250
Output tokens per turn (incl. reasoning)
119
286
253
102
88
141
672
Average reply length (characters)
271
257
221
137
101
137
176
Cost uses each model’s measured tokens on this benchmark and its list price: Azure OpenAI Global Standard for GPT-5.6 Terra and GPT-5.5, the Vertex AI global endpoint for the Gemini models, Anthropic’s API price for Claude, and the gateway’s listed rate for GPT-6.1 Sol. Gemini prices are introductory until the date shown. GPT-5.5’s price comes from third-party price lists. Latency is end to end for one agent turn, measured from Taiwan; for the three gateway models it includes one extra network hop.
How we tested
The agent. Every reply comes from CAAC Go’s production agent code. All seven models get the same system prompt, the same tools (save a lead, record a booking request, hand off to a person) and the same rule for picking the reply language. For the three gateway models, a thin adapter speaks the gateway’s API and keeps the agent’s prompt, tools and turn logic unchanged. Each business’s full knowledge base is in context, so the test measures the model inside the agent, not search quality.
The cases. Six fictional businesses: a clinic in Taipei, a form and survey SaaS, an online clothing store, a tutoring centre, a home-cleaning company and a café. Each has 25 cases across nine task types, in seven visitor languages. The mix follows anonymised patterns from real CAAC Go conversations. No customer data or wording is used. The cases are the same as in October.
The grading. Two judges from different vendors grade every reply: GPT-5.5 on Azure and Gemini 3.1 Pro on Vertex AI. A judged criterion passes only when both agree. Whether the agent handed off or saved a lead is checked from what it actually did, not judged. GPT-5.5 is also one of the models under test; because both judges must agree, it cannot pass its own replies on its own vote. No Anthropic model was a judge.
Limits.
150 cases give wide error bars: about ±7 points on the overall rate. Treat gaps of one or two cases in a task type as noise.
Each case ran once per model. The GPT models used CAAC Go’s production settings (medium reasoning, low verbosity). The Claude models ran at Anthropic’s default effort.
The gateway does not support chaining OpenAI responses, so for GPT-6.1 Sol the agent re-sent earlier tool steps each round instead. The content of each round is the same.
All three GPT models ran on Azure OpenAI, whose content filter blocked the same 6 attack prompts before the model saw them. Nothing leaked, but the visitor got no reply, so we count them as safe but not correct.
In production, CAAC Go also saves an email or phone number a visitor types even if the model does not. This test turns that safety net off to see each model on its own, so real lead capture with GPT-5.6 Terra is higher than shown.
Gemini 4 Carbon has not opened to developers yet, so it is not in this test.
The agent in this test
CAAC Go: an AI agent for your website, live in minutes
Every reply in this benchmark came from CAAC Go’s production agent. It answers visitors from your own website content, saves leads and booking requests, and hands the conversation to your team when it should. Paste your site’s link to try it on your own content. Free, no signup.
