
Polymath offers Horizon-SWE
A benchmark that evaluates AI agents on end-to-end, long-horizon software engineering workflows in production-grade systems.
- Currently Offered

A benchmark that evaluates AI agents on end-to-end, long-horizon software engineering workflows in production-grade systems.
Horizon-SWE is a benchmark that drops frontier AI agents into a simulated software company with a running application, real tools, and long-horizon tasks covering the full software development lifecycle.
It measures feature correctness, deployment and DevOps reliability, and engineering quality across 50 diverse tasks, with reported pass rates and partial-credit scores for leading models.


LongHorizonSWE is an evaluation dataset for frontier models on 4–12 hour engineering tasks as autonomous engineers, not coding assistants.
LongHorizonSWE is an evaluation dataset for frontier models on 4–12 hour engineering tasks as autonomous engineers, not coding assistants.



Agent Arena is a public leaderboard ranking developer tools by autonomous AI agent adoption.


Hyperbench provides AI agents that understand a system, work across repositories, and verify code changes on real hardware.


Simulated applications with expert-curated artifacts and verifiers for training AI agents on long-horizon professional workflows.


StableBrowse Agent Experience Platform helps devtool, infrastructure, and SaaS companies make their products usable by AI agents.


AI agent effectiveness layer that injects project context and verifies agent-written code in real time.


Orca is an open-source, terminal-native agent development environment from Stably AI that orchestrates multiple coding agents in parallel.



Open-source agentic IDE for orchestrating multiple AI coding agents in parallel.


Eval Engineering Platform turns AI evaluation results into automated production guardrails.


Observability and evaluation platform for shipping quality AI agents at scale.



Allstacks uses AI agents to help product and engineering teams optimize software creation.



OpenHands' canvas for running, orchestrating, and automating multiple AI coding agents.


Zenflow Code is Zencoder's spec-driven orchestration platform that coordinates parallel AI coding agents with built-in verification.


AI-native platform for end-to-end control of the software development lifecycle.


Monitoring platform that detects silent failures in production AI agents.


Operating system for agentic software development orchestrating specialized AI agents across the SDLC.


Evaluations compares models on standard benchmarks or custom tasks with repeatable runs.
Agent Arena is a public leaderboard ranking developer tools by autonomous AI agent adoption.

Hyperbench provides AI agents that understand a system, work across repositories, and verify code changes on real hardware.

Simulated applications with expert-curated artifacts and verifiers for training AI agents on long-horizon professional workflows.

StableBrowse Agent Experience Platform helps devtool, infrastructure, and SaaS companies make their products usable by AI agents.

AI agent effectiveness layer that injects project context and verifies agent-written code in real time.

Orca is an open-source, terminal-native agent development environment from Stably AI that orchestrates multiple coding agents in parallel.

Factory AI's autonomous coding agent that embeds into developer workflows to handle complete engineering tasks.

Open-source agentic IDE for orchestrating multiple AI coding agents in parallel.

Eval Engineering Platform turns AI evaluation results into automated production guardrails.

Observability and evaluation platform for shipping quality AI agents at scale.

Evals is an agent-experience evaluation platform scoring how product flows work for AI agents.

Allstacks uses AI agents to help product and engineering teams optimize software creation.

Platform for building, testing, and monitoring AI agents in production.

OpenHands' canvas for running, orchestrating, and automating multiple AI coding agents.

Zenflow Code is Zencoder's spec-driven orchestration platform that coordinates parallel AI coding agents with built-in verification.

AI-native platform for end-to-end control of the software development lifecycle.

Monitoring platform that detects silent failures in production AI agents.

Operating system for agentic software development orchestrating specialized AI agents across the SDLC.

Evaluations compares models on standard benchmarks or custom tasks with repeatable runs.
