Find the right AI model for the job

Frequently asked questions

How do you pick the best model for each use case?

Each card ranks models on the one benchmark that best matches the job: DeepSWE for coding agents, Terminal-Bench 4.0 for terminal and agent tasks, Humanity's Last Exam for hard reasoning, and the Artificial Analysis Intelligence Index for overall quality, open-weight models and models in FlowHunt. If fewer than three models have a DeepSWE score, the coding card falls back to SWE-bench Pro and names it.

What do Verified and Self-reported mean?

Verified scores were measured by an independent evaluator, such as Artificial Analysis, the ARC Prize Foundation or the DeepSWE leaderboard, with one fixed setup for every model. Self-reported scores come from the model's maker, in its model card or launch post. Only verified scores count toward rankings. Self-reported ones are shown and marked, but never ranked. When both exist, the model page shows both.

Why can the same model have two different scores on one benchmark?

Labs often test their own models with a custom setup: a different agent harness, more thinking time or a larger token budget. Independent evaluators use one fixed setup for every model, so their numbers are easier to compare and often lower. We rank on the independent number and keep the lab's number on the model page.

What is the Artificial Analysis Intelligence Index?

It is an independent composite score from Artificial Analysis that combines agent, coding, knowledge and science tests into one number. We use version 4 of the index. Its scores are not comparable with earlier versions, so older numbers you may have seen elsewhere will look higher.

What is DeepSWE?

DeepSWE is a set of 113 long software tasks written from scratch, so they are unlikely to appear in training data. A model has to build or change real code, and hidden tests decide whether it worked. It separates today's coding agents more clearly than SWE-bench Verified, which most frontier models have nearly maxed out.

Why is SWE-bench Verified only kept for history?

Frontier models now solve almost all of its 500 GitHub issues, so the gaps between them are within noise. We still show it in the full table so you can compare with older reports, but it no longer decides any ranking.

What is the difference between open-weight and closed models?

Open-weight models publish their trained weights, so you can run them on your own hardware, keep data in-house and fine-tune them. Closed models are only available through their maker's API or apps. Use the Open weight filter to see only open-weight models.

How often is this leaderboard updated?

We update it when major models are released or when independent evaluators publish new results. The date at the top of the page shows the last update, and every model page lists the release date and the source of each score where one is published.