iCreatAI News
Skip to content
Hacker News AI· pantaleone·· 19 days agoAI score60

Show HN: AgenticBench, 9 security tests run on 25 AI coding agents

Show HN: AgenticBench, 9 security tests run on 25 AI coding agents

AI summary

AgenticBench publishes a security leaderboard running 9 defined tests on 25 AI coding agent harnesses, each in a fresh container with the harness's default settings, scoring results as pass, fail, missing control (MC) or N/A, with N/A and MC never counted as passes. Results include Codex 0.163.0 at 10/11, GitHub Copilot CLI 1.0.103 at 25/11 and Cline 3.0.73 at 11/11, while Auggie 0.36.0 scored 5/11; failures are disclosed privately to makers first, with 14 disclosures raised so far (6 published, 8 held). The board states it is not a penetration test or certification and measures nine tests using planted, fake secrets, with full materials available to tested harness makers under an NDA.

Source: Hacker News AI · agenticbench.org