Agent Evaluation and Benchmarks

Comprehensive benchmark for evaluating LLMs as agents across 8 distinct environments.

Frontier benchmark for measuring general intelligence capabilities in AI agents beyond pattern matching.

Evaluates web agents on 283 real-world tasks across 163 live websites with interception and trace-based scoring.

Benchmark for General AI Assistants measuring real-world reasoning and tool use.

Framework for evaluating large language models with composable tasks and scoring.

Takes consumer personal AI agents apart hands-on and scores them on real errands, reporting each result with a confidence interval and no composite score.

Benchmark for evaluating LLMs on real-world software engineering tasks from GitHub issues.

Benchmark for web agent evaluation using real websites with realistic task completion metrics.

Benchmark suite that executes task prompts and Workflow Runner YAML in private fresh-repository workspaces with blinded per-step judging.