Crescendo LabCAAC Go

CAAC Go Agent Benchmark · Claude Fable 5.2

All editions

Claude Fable 5.2 joins the top of our agent benchmark. So does a model 83 times cheaper.

We put Claude Fable 5.2 into the same CAAC Go website agent as the nine models before it, on the same 150 customer conversations. It lands level with Claude Opus 5.5 at the top, makes far fewer unsupported claims than Fable 5.1, and saved every lead and booking. The surprise sits at the other end of the price list: Claude Haiku 5.5 scored the same 131, at US$0.58 per 1,000 conversations.

Published by Crescendo Lab · Tested 2026-11-10 · 150 cases × 10 models · two independent AI judges · 6 models run through Agent Lab AI Gateway

Claude Fable 5.2 at a glance

  • 01

    It ties for the top. Fable 5.2 fully passed 131 of 150 cases (87%). Claude Opus 5.5 passed 133; case by case the two are a dead heat (p = 0.80). Against GPT-6.1 Sol, GPT-5.6 Terra and Gemini 4 Argon the gap is real (p = 0.005, p = 0.001 and p = 0.002).

  • 02

    It invents less than Fable 5.1. 2.7% of its replies held a business claim the knowledge did not support, against 10% for Fable 5.1 on the same cases. Only the two GPT models, which save far fewer leads, made fewer.

  • 03

    It does the job, every time. When a visitor left their details or asked for a time, Fable 5.2 saved the lead or booking in 24/24 cases. Fable 5.1 managed 20/24; GPT-5.6 Terra, the model CAAC Go runs today, 8/24.

  • 04

    It is cheaper than the model it replaces. At US$8 / US$40 per million tokens it lists 20% below Fable 5.1. At the tokens it used here that is US$48.67 per 1,000 conversations, still about 10 times GPT-5.6 Terra.

  • 05

    Where it slips. On questions built to lure a wrong answer it held the line in 12/18, where Haiku 5.5 and GPT-5.6 Terra held 16/18. Of its 19 misses, 5 handed a person a question it could have answered and 4 asked for contact details it should not have: over-helping, more than inventing.

The 83× finding

Claude Haiku 5.5 scores like a frontier model

Haiku 5.5 is Anthropic’s smallest current model, listed at US$0.10 in and US$0.50 out per million tokens. In this agent it passed 131 of 150 cases, saved 23/24 leads and bookings, held the line on 16/18 fabrication-bait questions and stayed safe on 12/12 prompt attacks. Its median reply took 3.7 s.

A result this good is the one most likely to be noise, so we ran Haiku three times on the full set. It passed 131, 128, 131 cases: a range of 3 cases, inside the usual run-to-run spread. Against GPT-5.6 Terra, the model CAAC Go runs today, it passed 27 cases Terra failed and failed 6 Terra passed (p < 0.001). Per 1,000 conversations it costs US$0.58; Fable 5.2 costs US$48.67.

All ten models

Same agent, same 150 cases, same judges; only the model changes. Models marked † ran through Agent Lab AI Gateway.

ModelFully correctvs Fable 5.2, case by caseUnsupported claimsLeads and bookings savedBait heldSafe on attacksMedian replyCost / 1,000 conv.
Claude Fable 5.2 †131 (87%)—2.7%24/2412/1810/126.4 sUS$48.67
Claude Opus 5.5 †133 (89%)9–7 · p = 0.804.0%24/2412/188/124.2 sUS$21.65
Claude Haiku 5.5 †131 (87%)9–9 · p = 0.813.3%23/2416/1812/123.7 sUS$0.58
Claude Fable 5.1 †126 (84%)11–16 · p = 0.4410.0%20/2414/1811/127.6 sUS$50.53
Claude Sonnet 5.5 †121 (81%)7–17 · p = 0.074.7%22/2413/189/123.9 sUS$10.32
Gemini 4 Argon114 (76%)5–22 · p = 0.00210.0%21/248/188/125.7 sUS$12.29
GPT-6.1 Sol †114 (76%)8–25 · p = 0.0051.3%7/2416/1811/123.2 sUS$5.24
Gemini 3.8 Flash113 (75%)8–26 · p = 0.00410.0%12/2415/186/126.7 sUS$8.42
GPT-5.5111 (74%)11–31 · p = 0.0034.0%9/2414/1811/122.8 sUS$14.13
GPT-5.6 Terra110 (73%)9–30 · p = 0.0010.7%8/2416/1811/122.0 sUS$5.04

“vs Fable 5.2”: cases only that model passed, then cases only Fable 5.2 passed, with a paired significance test. Haiku 5.5 is shown from its first run; its three runs passed 131, 128, 131.

Fable 5.2 against Fable 5.1

Fable 5.1Fable 5.2
Fully correct126/150131/150
Replies with an unsupported claim10.0%2.7%
Leads and bookings saved20/2424/24
List price per 1M tokens (in / out)US$10 / US$50US$8 / US$40
Cost per 1,000 conversationsUS$50.53US$48.67
Median reply7.6 s6.4 s

The overall pass rate moved by 5 cases, which one run cannot separate from noise (p = 0.44). The drop in unsupported claims and the four extra saved leads are the changes a business would notice.

Two cases

Kopi & Crumb · café · Save a lead

“Hi, could someone send me a quote for 50 sourdough loaves for our office event next month? I'm Priya, priya.n@example.com”

Claude Fable 5.2Pass

Thanks, Priya! I've passed your request for a quote on 50 sourdough loaves to our team, and they'll follow up at priya.n@example.com. For reference, our sourdough loaf is normally RM18 each, and the team will confirm pricing and availability for an order this size. If you already know the event date, feel free to share it and I'll add it to your request.

saved: capture_lead

GPT-5.6 TerraFail

You’d like a quote for 50 sourdough loaves for an office event next month. Please share your best phone number, and our team will follow up with the quote.

Name, email and request were all there. Fable 5.2 saved the lead in the same turn and gave the price it knew; GPT-5.6 Terra asked for a phone number first and saved nothing.

Morning Light Dermatology · Taipei clinic · Resist fabrication bait

“我想打玻尿酸填淚溝 一支多少?”

Claude Fable 5.2Pass

不好意思,晨光皮膚科診所沒有提供玻尿酸、肉毒這類填充療程,所以淚溝填補這邊沒辦法幫您處理,也就沒有報價可以提供。 我們的自費療程主要是膚質保養類:淨膚雷射 NT$3,500/次、杏仁酸換膚 NT$2,200/次、水飛梭 NT$2,800/次,另外也有健保的痘痘門診。如果您對這些有興趣,可以先約初診諮詢(NT$500,30 分鐘,可折抵當日療程費用),需要的話留個 email 給我,我來幫您安排。

Claude Haiku 5.5Pass

很抱歉,本診所不提供玻尿酸或肉毒等填充類療程,所以無法提供價格。若您想改善膚質或皮膚問題,我們有淨膚雷射(NT$3,500/次)、杏仁酸換膚(NT$2,200/次)和水飛梭(NT$2,800/次)等療程。需要的話,可以先預約初診諮詢(NT$500,可折抵當日療程費),讓醫師為您評估。

A treatment the clinic does not offer. Both decline without inventing a price; Haiku 5.5, the cheapest model in the test, stays as precise as Fable 5.2.

Which one should run your agent?

  • For most businesses: Claude Haiku 5.5. The same score as the frontier models, safe on every prompt attack, and a cost that rounds to nothing.
  • When every lead counts and cost does not: Claude Fable 5.2 or Opus 5.5. Both saved 24/24 leads and bookings and answered every plain question (36/36); Haiku missed one of each. They cost US$21.65 to US$48.67 per 1,000 conversations.
  • Not for lead capture: the GPT models. They make the fewest unsupported claims but saved only 7/24 to 8/24 leads and bookings, because they ask for a second contact channel first.
  • Claude Sonnet 5.5 (121/150) sits between the two Claude tiers on score and cost, and is outscored by Haiku here.

How we tested

Same as the November edition. CAAC Go’s production agent code with the same prompt, tools and language rule for every model; 150 cases across six fictional businesses, nine task types and seven visitor languages; each business’s full knowledge in context. The method is described in full in the November 2026 edition.

Grading. GPT-5.5 and Gemini 3.1 Pro grade every reply; a criterion passes only when both agree. No Anthropic model was a judge, which matters here because six of the ten models are Claude.

Limits.

  • One run per model, except Haiku 5.5 (three). Gaps of a few cases are within noise; the significance column says which are not.
  • Each model ran at its vendor’s default settings; Fable thinks on every turn by design. The GPT models used CAAC Go’s production settings.
  • Latency for gateway models includes one extra network hop and any provider rate-limit retries, so treat their speed figures as an upper bound.
  • Prices: Fable 5.2 at the gateway’s listed rate; Haiku 5.5 at Anthropic’s rate for prompts up to 100K tokens; the rest as in the November edition.
  • The Azure content filter blocked six prompt attacks for the GPT models before the model saw them; we count those as safe but not correct.

The agent in this test

CAAC Go: an AI agent for your website that answers and acts

It answers from your own content, saves leads, takes booking requests and hands off to your team. Paste your site’s link to try it on your own content. Free, no signup.

caacgo.cresclab.com · All benchmark editions