The AI predictor gave a confident conclusion: American oil production is expanding, with new wells opening in Texas; consumer trends are in line with seasonal expectations. Ukraine hasn't hit Russian refineries in a while, and the US is fairly insulated from that anyway. Prices will be flat-to-down with tight confidence intervals.
Five models, over five prediction techniques, all gave the same kind of answer. The prediction was uploaded to Metaculus - and it turned out to be completely wrong.
At the time of that Metaculus question, the US Air Force had a 24/7 airlift from the US to RAF Fairford, and from Fairford to Jordan, with widespread anticipation of a series of strikes on Iran. The whole world knew it, but the predictors didn't. How did we get here?
It's not about prediction
I'm Alex; I compete in AI forecasting in the Metaculus future evaluation tournaments. People's first instinct when they think about AI forecasting is that it's about how well you predict - doing RL on open models, clever explicit modelling techniques, sampling parameters. The stuff of research papers and architectural breakthroughs.
It's not. On the questions where the research phase is straightforward, a better predictor gets you the edge. But most of the capability is born of frontier model progress[1], and the questions where people have big wins or losses come from research differentiation: whether you got the dataset, whether you got the subtly wrong dataset, or whether your picture of the world was too narrow.
Boring iteration
A lot of the practical progress is boring. You reduce costs, which lets you do more; you use a variety of models, and test which work well, and which work well together - Mantic found that a fine-tuned oss-120b and Grok 4 were the least replaceable, despite Grok's middling individual performance[0]. It's interesting but unglamorous iteration.
A thousand under-resourced footnotes
Everything depends on everything else, and you often don't even know what something is dependent upon. The aforementioned oil prediction question didn't think to check if there was a war on; an agent predicting air passenger numbers might not think to Google whether the US government is shutting down tomorrow. There's a crude, mechanical fix: At the start of your prediction process, you give a model some examples of this happening, have it generate dozens of search queries, and check if there's anything to dig into. It works ok, but it's just probes into the dark, and it misses the complex interconnections of the modern world. You quickly discover that if you seriously want to consider every possible factor, your effort will grow exponentially; every factor has its factors, and this matters more as we move towards AI superforecasting, and the ambition to consider the butterfly effect rather than blindly widening our confidence intervals to account for it.
As you iterate on a production system, you quickly find yourself reaching for structure to try to mitigate your failure cases. You make a subagent that explicitly focuses on the resolution criteria, then another to make sure you didn't misinterpret. You tell it to look for the long shots, then learn what it can about them - within a limited budget. A frontier model rigged up to a search tool and an unlimited thinking budget can work this stuff out eventually, but it can't do so economically. And a model can know about something, but still not properly account for it, because it's one of a thousand under-resourced footnotes.
I've used political examples, but many areas prove worse. For example, a simple question about quarterly performance at an internationally-exposed business must track not just the business itself, but its competitors, economic climate, regulatory environment, domestic and international politics, and whether tariffs were announced a few hours ago - and how credible those tariffs are, and why they were imposed.
Humans get continuous learning for free; where we stop actively considering, we fall back to our priors about the world - priors that are continuously recalibrated as we absorb information throughout our daily life. LLMs have sensible priors, too - until they turn out to be dangerously out of date.
What about Bayes nets?
What if we borrow from Bayes nets, belief propagation, and the truth-maintenance systems of the 1980s? What if we could express the world as a beautiful, albeit probably-not-acyclic-anymore, flow of probabilities, events, relationships, and analyses? The political future of the vice president could affect the situation in Iran, which could affect the next month's oil prices, which could affect the President's approval rating - and, awkwardly, the vice president's political future.
The complexity explodes massively. Even when you try to model within a constrained question - as I once tried to do, on a question about the Fed's rates - you start wondering exactly how causation flows, what primitives best describe a relationship, or how you model a threshold versus an influence. But you can do it, and you can test it. Empirically: it doesn't work.
Explicit modelling hurts
While explicit modelling and cognitive structure improve research systems, they actively sabotage the prediction stage. The SOTA predictors used by Metaculus competitors typically aren't single-shot prompts, but they're not prescriptive reasoning frameworks either. The best systems are those that optimise for maximally comprehensive yet in-context analysis of what matters, and maximal diversity of reasoning and models while minimising the prescriptiveness of the cognitive framework, particularly where that framework is imposed upon the question rather than emergent from it. Prompting to "think about base rates" is a standard part of a good prompt. Prompting to "make a base rate, then identify key factors and work through each one, quantifying their impact and updating your base rate" is worse - and, in fact, is worse than "predict this please".
Everything from an explicit Bayesian analysis to Monte Carlo sims to graph-based modelling approaches ruin calibration. While these systems often do a good job of reasoning about factors, and can provide good ideas to more robust systems, they struggle to put it to a number without going wildly and confidently off. The more numbers the LLM conjures out of its latent space, the worse it gets.
My untested hypothesis here, based on my personal experience attempting to make these systems work, is that the analysis of the direction of their relationships, and their abstractions, is fine. It is putting numbers on lots of little things that goes wrong - because naming-the-number from a qualitative situation is a difficult task, and the relationships involved often amplify small mistakes.
There is a narrow exception here: When it's predicting something that particularly suits quantitative reasoning, like a predictable time series, modelling can work well.
Structure in research, not prediction
Research benefits from structure, scaffolding, and explicit cognitive frameworks; meanwhile, prediction benefits from breathing room, and is hurt by imposed frameworks and constant quantification. They need not necessarily be in conflict.
Compression, not analysis
This is conjecture; I am proposing a system I have not built. As you will see, it has a high unit cost but is subject to economies of scale; I am a student without the budget to test it.
We can build out the graph structure at the research layer, then distill it down to a report that compresses what matters, rather than constraining the predictor to the whims of the research system. Specifically, we can associate artifacts - a blog post, data series, or rich object, in the terminology of Mercury, my internal research system - with nodes and edges on a graph structure that is continuously maintained as new information comes in. Whether the information is continuously updated or updated as needed is irrelevant; either way, artifacts, and the graph structure they live on top of, would be preserved beyond a single run.
In addition to the advantages of the graph structure, building a continuous knowledge base comes with another advantage: Cost-effective access to historical information. The primary constraint when building a forecaster is cost, and one of the first decisions you make is to stop caring so much about the past. It reduces the workload substantially, but in doing so, it loses access to historical information: Has this happened before? How did we get here? What was the actual sequence of events that got us here, over the course of several years? A continuous knowledge base gets to spread this cost over multiple runs - one prediction's present is another prediction's past.
And so you ask a question, consult the graph, and let it ripple. The system need not model causation, thresholds, or strengths, merely relationships - "what might matter". At each interconnected topic, you compress the past and present into the core information a predictor needs to include in its worldview - not analysing, not giving an opinion, but compressing the core picture into what we know, how abnormal this is, and what matters right now. That report then becomes an artifact that can be consulted in the future.
What would break this
Right now, AI forecasting performs worse than human superforecasters[1]. Part of that gap could be the continually-updated-priors gap I described earlier. It clearly causes catastrophic issues sometimes, and it's a reasonable assumption that as the models improve, their reasoning becomes increasingly bottlenecked by what they know vs how they think. But that's an assumption, and an untested one; perhaps the vast majority of situations don't require special world context? This may be possible to test empirically: Compare a narrow research report with a comprehensive one across progressively more capable models, and see whether the gap between the prediction quality of narrow vs wide research is getting bigger or smaller. Once again, I don't have the budget for this; inference is expensive.
There might also be implementation issues. Using past reports to guide future ones could help push the cost-quality Pareto frontier by "reusing cognition", but what if that cognition was wrong? A single mistake could poison the whole world model, rather than a single prediction.
Furthermore, my experience with research systems is that every time you see your system work in practice, you begin to see flaws you could never have thought of from the start - and you start planning the next one. My proposed system may be a good first iteration - something to learn from - but the optimal approach is likely to look very different.
The bitter lesson is expensive
The closing gap between man and machine is largely due to frontier model progress[1], and research improvements hit diminishing returns. So why not just put a frontier model in a sensible harness and let it work this out itself, rather than building an elaborate structure maintained by inexpensive models?[2]
Frontier models are expensive, and I assume that that will continue. An end-to-end system may well prove the best predictor (and, indeed, according to Metaculus[1] some successful systems have been end-to-end), but it is unlikely to be the most cost-effective at a given quality. A mature, cost-effective superforecaster is likely to look like a well-designed structured research system linked to a frontier model (or straightforward ensemble) as a predictor, with the predictor having an escape hatch to research anything the earlier system missed.