






Proven on real-world systems, we publish the leading benchmarks for AI coding agents. That same expertise drives our internal evaluations, so LogicStar keeps getting better as models evolve.
validating tests generated
LogicStar reproduces every bug with a failing test that proves it's real and validates fixes actually resolve them. State-of-the-art performance on SWT-Bench Verified.
overestimation of success rate in SWE-Bench Verified
Many AI coding agents overfit to a single benchmark. We automatically create new benchmarks for every use-case and show popular software engineering agents lose up to 60% of performance on an application focused benchmark of 366 diverse codebases.
of working AI-generated code is exploitable
Even frontier models produce exploitable backends. Across 392 tasks, one in three working solutions contains SQL injection, path traversal, or code injection vulnerabilities.
cost increase, zero performance gain
Over 60,000 repos include AGENTS.md files to guide AI agents. Our evaluation shows these files reduce success rates by up to 3% while adding 20% to inference costs.
of AI refactoring attempts break code
AI agents solve only 22% of multi-file refactoring tasks and introduce breakage in 63% of attempts. CodeTaste measures whether AI restructures code the way a senior engineer would.