When will SWE-bench be solved?
12
100Ṁ892
2030
0.9%
2024
1.8%
2025
36%
2026
58%
2027
1.4%
2028
1.4%
2029
0.7%
later

Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.

Get
Ṁ1,000
to start trading!
Ordenar por:
comprou Ṁ5 YES

Relatively significant chance of never (which resolves at later). Frontier LLMs are just going to get more and more contaminated with this benchmark's test data - at some point authors might just stop tracking

© Predita Markets, Inc.•Termos de Uso•Privacidade