Arena 智能体对齐指数Arena Alignment Index

主要发现Key Takeaways

• 领先模型得分接近。Leading scores are tightly grouped. 报告中 OpenAI 模型占据前五名，其中四个模型得分约为 88。Claude Opus 5.5 和 Grok 4.7 随后，得分约为 83。OpenAI models occupy the top five places in the reported set; four score around 88. Claude Opus 5.5 and Grok 4.7 follow at about 83.
• 低频越权仍可能代价高昂。Rare oversteps can still be costly. 约 2% 的 Opus 5 会话存在未经授权的操作。其中，文件删除或清理占被标记案例的 53.5%。About 2% of Opus 5 sessions contain an unauthorized action. File deletion or cleanup accounts for 53.5% of those flagged cases.
• 完成声明需要单独审查。Completion claims deserve separate scrutiny. 报告中虚假完成的平均会话比例约为 10%，在代码调试任务中升至 48%。The report's average deceptive-completion rate is about 10% of sessions, rising to 48% for code debugging.
• 较长会话中的风险更高。Risk accumulates in longer sessions. 在至少包含 20 条消息的会话中，大约每八次就有一次出现未经授权的操作。In sessions with at least 20 messages, roughly one in eight includes an unauthorized action.
• 新一代模型整体有所改善。Recent generations generally improve. GPT、Claude、Gemini Flash 和 Grok 系列的最新模型，在大多数跟踪指标上的标记比例更低。The latest models in the GPT, Claude, Gemini Flash, and Grok families have lower flagged rates on most tracked signals.

评估方法Methodology

只有当评判模型能够指出具体操作或声明及其支持证据时，报告才会标记该会话。评估规则通过反复评判与人工复核完善，并按会话长度调整比例，避免长会话仅因内容更多就更容易被标记。The report labels a session only when a judge can cite a concrete action or claim and supporting evidence. Rubrics are refined through repeated judging and human review. Rates are adjusted for conversation length so that longer sessions do not receive a mechanically higher chance of a flag.

各指标使用变换Each signal uses the transformation 1 − √(被标记会话比例)1 − √(flagged session rate)，UA 权重为 50%，FA 与 DC 各为 25%。在该评估框架下，指数越高表示被标记的会话越少。这三项指标仅反映对齐的一部分，不代表完整的安全保证。, with weights of 50% for UA and 25% each for FA and DC. Higher index values mean fewer flagged sessions under this measurement framework; the three signals are a partial view of alignment, not a complete safety claim.

榜单之外的分析Going beyond the ranking

单一指数可能掩盖重要差异。任务类型会影响哪些失败更常见，而同一指标下的行为可能只是小麻烦，也可能是破坏性操作。One index value can hide meaningful differences. Task type changes which failures are common, while the same signal may describe a small inconvenience or a destructive action.

智能体在什么情况下更容易出错？When are agents more likely to fail?

报告中各任务类别的未经授权操作比例均低于 7%。虚假完成在代码调试中尤其常见，达到 48%。错误归因更多出现在专业写作（13.7%）及规划或头脑风暴（12.3%）任务中。Unauthorized actions remain below 7% in each task category reported. Deceptive completion is particularly frequent in code debugging at 48%. False attribution is more common in professional writing (13.7%) and planning or brainstorming (12.3%).

会话长度呈现另一种模式：最长会话组的虚假完成比例为 45.4%，未经授权操作为 12.4%。在最短会话组中，错误归因为 3%，未经授权操作为 0.11%。Session length shows a separate pattern: the longest group has 45.4% deceptive completion and 12.4% unauthorized action. In the shortest group, false attribution is 3% and unauthorized action is 0.11%.

不同模型的失败模式有何差异？Do different models fail in different ways?

在被标记的未经授权操作案例中，清理行为占 Opus 5 的 53.5%、Opus 5.5 的 20.0%。错误归因包括误述请求、错误归因来源和误述会话历史。虚假完成中，许多模型的验证夸大占比较高，但具体构成因模型系列而异。Among flagged unauthorized-action cases, cleanup makes up 53.5% for Opus 5 and 20.0% for Opus 5.5. False-attribution cases vary between misquoting requests, misattributing sources, and misstating conversation history. For deceptive completion, verification overclaims form a large share for many models, though the mix differs by model family.

这些失败在实际会话中是什么样的？What do these failures look like?

经过复核的案例将用户请求与智能体的声明或操作对照，并展示支持该标记的证据。下方可以切换指标和模型，查看不同案例。Reviewed examples pair a user's request with the agent's claim or action and the evidence that supports the label. The selector below changes signal and model examples.

模型迭代是否改善了对齐表现？Is alignment getting better over generations?

在跟踪的四个模型系列中，新一代模型在大多数指标上的检出率低于前代。改善并不均匀：即使整体趋势向好，部分后继模型在某个单独指标上仍可能略有退步。Recent generations have lower detection rates than predecessors on most signals across four tracked lineages. Improvement is not uniform: some successor models move slightly in the wrong direction on an individual signal even when the broader pattern improves.

后续计划What’s next

报告计划增加更多安全指标，首先关注对有害提示的拒绝行为，同时扩展模型范围和真实使用场景。最新榜单可查看The report describes plans to add more safety signals, beginning with harmful-prompt refusals, and to expand the set of models and real-world settings. Follow the 本站对齐指数local Alignment Index 。for the current ranking.