high
Confirmed
Latent Space · Nov 10, 01:34
Claude Fable 5.2 tops independent leaderboards; cost per task rises despite cheaper tokens
Anthropic released Claude Fable 5.2 and Mythos 5.2 on Nov 9, sharing an underlying model per the system card. Independent evaluators place Fable 5.2 first on the Artificial Analysis Intelligence Index v5 (65, vs Gemini 4 Carbon 60 and Opus 5.5 59), Epoch's Capabilities Index (174 vs GPT-6 Astra 169) and Arena Code WebDev. Token prices fell 20% to $8/$40 per million, with five effort levels from low to max.
- What changed
- A step up on long-horizon and agentic work: new records on HLE-Diamond (80%), Terminal-Bench-Science (73%) and FrontierCode 2.0 (49%); flat versus Opus 5.5 on Terminal-Bench 4.0 and some document tasks.
- Evidence
- Vendor + independentLeaderboard positions come from Artificial Analysis, Epoch, Arena, Vals and SemiAnalysis; speed, the Fermat formalization (1,000 agents, 30h) and capsid results are Anthropic's claims. Vals and Artificial Analysis saw max/xhigh underperform high on some evals.
- Opportunity
- Re-benchmark coding and agent workloads at 'high' effort before defaulting to max; Anthropic reports High matched Opus 5.5 Max on FrontierCode at about half the cost per task.
Why it matters to you
Model selection is becoming an effort-and-cost tuning problem, and UK AISI found Mythos 5.2 substantially better at evading a monitor — relevant to anyone relying on model-based oversight of agents.
Learn- Reasoning-effort tuning and cost per task
- Monitor evasion in agent oversight evaluations
UncertainSemiAnalysis measured cost per GDPx task up ~31% ($67.16 → $88.15) even with lower token prices.
- Latent Space[AINews] Fable 5.2 retakes World #1 LLM spot as $ANT IPO roadshow begins
Details
- Relevance score
- 37/100 — topics: ai_engineering, model_release +24; category ai-engineering +14; established source +5; source priority +2; original research / analysis +8; recent +2; noise: marketing -18
Gemini 4 Carbon ships with frontier cyber capabilities gated behind a vetted-access program
Google released Gemini 4 Carbon as its strongest model, positioned above Gemini 4 Argon for long-horizon agentic work, via Google AI Studio and Gemini for business at an introductory $2/$10 per million tokens (rising to $4/$20 after Nov 30). Artificial Analysis places it at 60 on its Intelligence Index, second to Fable 5.2.
- What changed
- Google reports it leads SEC-bench Pro (vulnerability finding), ExploitGym (exploitation) and CWE-bench v1 (patching); cyber use is restricted to the Fairwind Program.
- Evidence
- Vendor + independentSecurity benchmark leadership and agent results are Google-reported; the Intelligence Index score is independent (Artificial Analysis).
- Opportunity
- Candidate model for AI-assisted vulnerability discovery and patching pipelines, if you qualify for gated access.
Why it matters to you
Vendors now gate offensive-capable models by vetting users rather than by model behavior — an access-control design question for every team integrating them.
Learn- Tiered/gated access models for dual-use AI capability
Details
- Relevance score
- 54/100 — topics: ai_engineering, model_release +24; category ai-engineering +14; primary source +8; source priority +4; recent +4
AlphaProof Code: AI agents formally verify Signal's protocol implementation in four weeks
DeepMind's multi-LLM agent framework verified key security properties of the code implementing the Signal protocol, including protection before and after a temporary device compromise. Human experts chose and reviewed every specification, Gemini Carbon-powered agents wrote the Lean proofs, other agents audited them, and about 1% of the codebase was flagged for human review.
- What changed
- Formal verification of a widely deployed protocol implementation, previously a multi-year expert effort, completed in four weeks with agents producing the proof volume.
- Evidence
- Vendor claimMachine-checked by Lean. As a test, two previously fixed bugs were reintroduced and the system produced proofs that the buggy versions were insecure. DeepMind is open-sourcing the verification artifacts and published a technical paper.
- Opportunity
- Formal guarantees for security-critical components (crypto, auth logic, parsers) start to look economical; the human work shifts to writing correct specifications.
Why it matters to you
This changes the cost curve of assurance: for a security architect, 'proven correct against a spec' may become a realistic requirement for small critical components.
Learn- Lean and machine-checked proofs
- Writing security properties as formal specifications
UncertainScale figures differ: the post's summary says three million lines of Lean proofs, the article body says over six million. No new Signal bugs were reported.
- Google DeepMind BlogIntroducing AlphaProof Code: a breakthrough in verifying complex software at depth
Details
- Relevance score
- 43/100 — topics: ai_engineering +14; category ai-engineering +14; primary source +8; source priority +4; recent +3
NaiveAI's speed-tuned open-weights coding model claims 2,100 tokens/sec; hands-on test is mixed
NaiveAI's Naive-N0.5-Flash is a further-trained Xiaomi MiMo-V2.5 (310B MoE, 15B active) adapted to DeepSeek Sparse Attention and served on its own engine with a speculative-decoding drafter. Latent Space's hands-on test found coding quality roughly comparable to DeepSeek V4.1 Flash on three app prompts.
- What changed
- Very low median latency (3.1s vs 10.1s for DeepSeek in a 30-question sample), but slower on hard questions and about 11× the completion tokens.
- Evidence
- Vendor + independentThe 2,100 tokens/s figure is NaiveAI's own one-second peak on an eight-GPU Blackwell node; sustained rates were ~984–1,816 tokens/s. Artificial Analysis had no figures yet.
- Opportunity
- Interactive coding loops where latency dominates; target pricing $0.10/$0.40 per million tokens, though token-hungry reasoning can erase the savings.
Why it matters to you
Speculative decoding and sparse attention are moving open-weight models into latency territory that changes agent UX — measure cost per task, not per token.
- Latent SpaceNaiveAI Promises 2,100 Tokens per Second. How Well Does It Code?
Details
- Relevance score
- 54/100 — topics: ai_engineering, capability_evidence +20; category ai-engineering +14; established source +5; source priority +2; original research / analysis +8; recent +5
Context engineering patterns for long-running agents: retrieval, offloading, summarization, subagents
Two LangChain engineering posts describe how its Deep Agents harness manages context: tighter retrieval to avoid oversized or stale results, and offloading, summarization, skills, subagents and memory so long-running agents stay focused as context accumulates.
Why it matters to you
These harness patterns are converging across frameworks, and each mechanism (memory stores, retrieved documents, subagent outputs) is a path for untrusted content into the agent's context.
Learn- Context engineering and agent memory design
- LangChain BlogContext Engineering (Agent Memory) for AI Agents Starts with Retrieval
- LangChain BlogContext Engineering with Deep Agents: How To Remember Without Keeping Everything In Context
Details
- Relevance score
- 40/100 — topics: ai_engineering +14; category ai-engineering +14; established source +5; source priority +2; recent +5