BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
4
90á¹€1322027
86%
chance
1H
6H
1D
1W
1M
ALL
Only the sub benchmarks that are scored as an accuracy (i.e. from 0-100%) will be included (I think that's all of them but I'm not sure)
It must be a single model. If Model A achieves 75% on half and Model B achieves 75% on the other half that does not resolve the question YES
Ensemble models are fine but something like "run Model A on this benchmark and model B on this other benchmark" is not. If there is model selection is must be learned and it cannot include the current benchmark as an input.
Esta pergunta é gerenciada e resolvida pela Predita.
Get 
1,000 to start trading!
1,000Pessoas também estão operando
Perguntas relacionadas
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
87% chance
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
SOTA AI at EOY 2026 a reasoning model?
94% chance