Event Forecasting: Three Natures of Uncertainty

“I never think about the future. It comes soon enough.” Albert Einstein

Introduction: An Event Is Not a Number

Imagine two questions:

1. “What will the dollar exchange rate be in a month?”
2. “Will company X declare bankruptcy by the end of the year?”

The first one is about a number. The second one is about a fact. And these are fundamentally different tasks.

In Article 2 we learned to forecast numbers: from the naive “tomorrow will be like today” to neural-network transformers. But numbers are, oddly enough, the simple case. You have a long history, you extend it into the future, and everything is measured in familiar units: degrees, hryvnias, units of goods sold.

Events are a more complex world. Here:

• There may be no history. Presidential elections happen once every 4–5 years. That is only a handful of data points. And the decision of a specific leader about an invasion is an altogether unique event.
• The forecast influences reality. If you forecast the bankruptcy of a bank, people withdraw their money, and the bank goes bankrupt. The forecast created reality.
• Accuracy is measured differently. “60% that candidate X will win.” Candidate X won. Was the “60%” forecast good or bad? It is not obvious.

Why this matters. Most decisions in life, business, and politics are decisions about events. To invest or not? To hire this person or not? To impose sanctions or not? To enter this country’s market or not? All these decisions are bets on events.

The main thesis of this article: different events “live” according to different laws, and each one needs its own instrument. But there are universal thinking techniques that work for events of any type.

Roadmap:

Three types of events → Methods for each → Universal techniques
→ Collective intelligence → Measuring accuracy
→ Cognitive traps → Rare events → How to choose an approach

Three Types of Events — Three Sources of Uncertainty

All the events we want to predict fall into three large classes. The criterion is simple: who or what has the greatest influence on the outcome?

EVENTS
├── Type 1: Dependent on nature
│     Source: physical/biological laws
│     Key question: "How does nature work?"
│     Examples: earthquake, hurricane, harvest, epidemic
│
├── Type 2: Dependent on many agents
│     Source: mass behavior, markets, society
│     Key question: "How will thousands/millions behave?"
│     Examples: stock price, elections, fashion, trends
│
└── Type 3: Dependent on specific individuals
      Source: will and strategic calculation
      Key question: "What will this specific person do?"
      Examples: invasion, company merger, deals

Why is it important to distinguish these? Because for each type you need its own toolkit. A mistake in choosing the type, and your forecast is doomed. You cannot forecast an earthquake using game theory methods, just as you cannot forecast a political leader’s decision using meteorology methods.

Let us take apart each type separately.

Type 1: Events Dependent on Nature

Why This Is the Simplest Type

Nature is the ideal “client” for a forecaster. It has no intentions, it does not bluff, it does not change its strategy, and it does not read your forecasts or react to them.

A hurricane does not care what you think about it. Tectonic plates do not care whether you announced a “high probability of an earthquake.” A virus does not change its behavior because you published an epidemiological forecast.

This makes Type 1 the most reliable for forecasting, provided that you understand the laws that govern the phenomenon.

How to Forecast: Models Based on Laws

Physical/biological models. You know the laws → you build a mathematical model → you run a simulation → you get a forecast.

• Weather forecast: hydrodynamics equations + data from thousands of weather stations and satellites + supercomputer = a forecast of a hurricane’s trajectory 5 days before landfall. Thirty years ago this was impossible — today it is routine.
• Epidemic forecast: the SIR model (Susceptible → Infected → Recovered). You know how the virus is transmitted, what the incubation period is, and how many people are already sick → you can calculate how many will get sick in a week. This is exactly how forecasts worked during the pandemic.

A curious fact. In 1854, long before viruses were discovered, the London physician John Snow predicted the source of a cholera outbreak — a contaminated water pump on Broad Street — purely by the geography of deaths. He plotted the cases on a map, saw a cluster around the pump, and persuaded the authorities to remove its handle. The outbreak stopped. This is, perhaps, the first example in history of epidemiological forecasting based on data.

Statistics of Rare Natural Events

When the laws are not fully known, a statistical approach is used.

Earthquakes. We still cannot predict a specific earthquake (the day, the magnitude, the epicenter). But we can estimate long-term seismic hazard: “In region X, the probability of an earthquake of magnitude 7+ over the next 30 years is 60%.” This is not “when” and not “where exactly,” but this is enough for building codes and evacuation planning.

Strengths and Weaknesses

• Strengths: where the laws are well known, the forecast is almost guaranteed. Besides, the forecast does not influence the system (a hurricane does not react to your forecast).
• Weaknesses: events in complex nonlinear systems. In such systems, the smallest error at the input can grow avalanche-like (the so-called “butterfly effect”). Most of the events interesting to us belong precisely to this type.

Type 2: Events Dependent on Many Agents

Why This Is Harder

Here the outcome is determined by the actions of many people: hundreds, thousands, millions. Each has their own goals, their own information, their own strategy. And, a key point, no one “controls” the system. The outcome emerges from interaction.

The stock market is an ideal example. Millions of traders buy and sell. Each has their own strategy. The market’s overall movement is not planned by anyone; it emerges emergently.

An additional complication: the forecast influences the system. If a Goldman Sachs analyst forecasts a 10% fall in the S&P 500 index, investors start selling, and the index really falls. The forecast changes the reality it predicts.

Bayesian Networks (Bayesian Networks or Bayesian Belief Networks — BBN)

Intuition. Draw a “map” of causes and consequences. Each arrow is a probabilistic link. Not “A always causes B,” but “if A has happened, then the probability that B will happen increases by 30%.”

An everyday example:

• In the morning there is a downpour → with probability (x%) there will be traffic jams on the roads → with probability (y%) you will be late for work.
• But you can also go in the reverse direction: your boss lives close to work and knows nothing about the traffic jams. He sees that you and several other subordinates with cars are late. He can draw the conclusion that today there were huge traffic jams.

It is exactly this ability — to draw conclusions in both directions — that makes Bayesian networks a powerful tool. BBNs are used in medical diagnosis (symptoms → probable disease), technical diagnostics (a sign → probable failure), and risk assessment.

How a Bayesian Network Is Built: A Graph Without Loops

Take a look at any Bayesian network, and you will see circles and arrows. Mathematicians call this a directed acyclic graph (Directed Acyclic Graph, or DAG for short). The scary name hides three simple things:

• Circles (nodes) are variables: “rain,” “traffic jams,” “being late for work.” Each variable has a set of states, and there can be as many states as you like: rain is simply “falling / not falling,” but traffic jams can have a whole gradation — “none / weak / medium / strong.” Binary variables (yes/no) are only the simplest case.
• Arrows (edges) are a direct influence: rain → traffic jams. Read it simply: “rain influences traffic jams.”
• “Acyclic” means: the arrows do not form loops. Following the arrows, you will never return to the point where you started.

Why are loops forbidden? Imagine a family tree: the arrow “parent → child” points in the direction of birth. You cannot be your own great-grandfather, and in a Bayesian network an event cannot be its own cause. This is not a mathematician’s whim: it is precisely acyclicity that allows you to compute all the probabilities in a finite number of steps. If A influenced B, B influenced C, and C again influenced A, the computations would “loop” and never give an answer.

But the real world is full of loops! Panic → selling → falling prices → even more panic. How to model them? About this a little later — this is a separate interesting story.

A historical fact. Bayesian networks in their modern form were invented in the 1980s by Judea Pearl, an Israeli-American computer scientist. In 2011 he received the Turing Award — the “Nobel Prize” of computer science — for this work. Later Pearl went further and wrote the book The Book of Why about how to teach machines to reason about causes, not just about correlations — the very mantra we discussed in Article 1.

Conditional Probability: The Heart of the Network

The arrows show what influences what. But how strongly? This is written down in the tables of conditional probabilities — the heart of each node.

The notation P(wet asphalt | rain) is read like this: “the probability that the asphalt is wet, given that it is raining.” The vertical bar means “given that.”

Each node stores a small table. If the variable has several states, the table simply gets longer. For example, for the node “traffic jams” with four states (no traffic jams / weak / medium / strong):

• If it is raining: no traffic jams — 10%, weak — 20%, medium — 40%, strong — 30%.
• If it is not raining: no traffic jams — 60%, weak — 25%, medium — 10%, strong — 5%.

In each row, the probabilities sum to 100% — because one of the four states will necessarily happen. That is the whole construction. No magic: circles, arrows, tables. A network of a hundred nodes is simply a hundred such tables connected by arrows.

Why This Works at All: The Magic of Factorization

A natural question: why all these circles? Why not make one big table of the “probability of any combination of events”?

Let us calculate. Imagine 20 variables, each “yes/no.” The full table of all combinations contains 2 to the power of 20 — about a million rows. For 50 variables — already a quadrillion. No computer can hold such a table, and no expert can fill it in.

A Bayesian network does a clever trick: it factorizes one gigantic table into dozens of small ones. The key assumption: each node depends only on its immediate parents, not on the whole network. “Wet asphalt” depends directly on rain, but not on what was said on television yesterday. Instead of a million rows — twenty tables of 4–8 rows each. This is exactly what turns an impossible problem into a perfectly practical one.

Where do the arrows and the numbers come from? Two ways. The first — experts draw the map manually and fill in the tables from experience (this is how medical diagnostics work). The second — an algorithm learns from data: you give it thousands of past examples, and it computes all the percentages itself. This is called “Causal Discovery.” In practice the two are often mixed: people draw the structure, the machine learns the numbers.

Loops Are Forbidden. But Reality Is Loops!

We said: cycles in a graph are forbidden. But look around:

• Positive (reinforcing) feedback: panic on the exchange → traders sell → prices fall → panic grows → they sell even more actively. A snowball rolling down a mountain and growing. This is exactly how stock market crashes and bank panics are born — remember the self-fulfilling prophecy from Article 1.
• Negative (stabilizing) feedback: the thermostat in your apartment. It got hot → cooling switched on → it got cool → cooling switched off. These are the brakes of the system: they do not let an indicator escape either up or down. The economy has such loops too: inflation rose → the central bank raises the rate → loans become more expensive → demand falls → inflation slows down.

How does a network that is forbidden to have loops model a world that rests on them?

The solution is unfolding in time. The key observation: a loop exists not at a single moment but in time. Panic today causes selling tomorrow, and the falling prices the day after tomorrow reinforce the panic the day after tomorrow. We draw not one column of nodes but three: “today,” “tomorrow,” “the day after tomorrow.” Now all the arrows point only forward in time: panic today → selling tomorrow → prices the day after tomorrow → panic the day after tomorrow. The cycle disappears! The graph is acyclic again, and the mathematics works again.

This is like a subway map: the lines themselves form rings, but if you lay out the journey by hours — “at 9:00 I am at station A, at 9:15 at station B” — all the arrows point into the future.

And this is exactly what dynamic Bayesian networks, which we will talk about next, do: they are the legitimate way to introduce feedback loops into a world where loops are forbidden.

Limitations: Where Bayesian Networks Are Not Enough

To be honest, there are also weak points:

1. The tables balloon. If a node has two parents and all variables are binary, the table has 4 rows. If there are ten parents, it is over a thousand. And if each one also has four states, as in our traffic jam example, the table becomes altogether indecent. Therefore in practice Bayesian networks are kept “sparse”: not many arrows for each node.
2. Continuous quantities have to be chopped up. A temperature of 22.4°C does not fit into a table with a few states. It is cut into categories “cold / normal / hot” — and accuracy is lost at the boundaries: 22.4° and 22.6° may fall into different categories, even though the difference is negligible.
3. Garbage in — garbage out. If the map is drawn incorrectly (an important arrow is missing or a nonexistent one is added), the network will honestly compute all the probabilities, but for a world that does not exist.

Dynamic Bayesian Networks (Dynamic Bayesian Networks — DBN)

The same thing, but in time. The state of the system today influences the state tomorrow and so on.

Example — a video game publisher:

• Step 1 (today): due to minor technical bugs, the game receives the first restrained reviews from bloggers. The probability that the rating will drop is growing.
• Step 2 (tomorrow): the lower rating today lowers the probability that Steam or YouTube algorithms will recommend the game to new users tomorrow. The number of downloads falls.
• Step 3 (the day after tomorrow): the drop in downloads tomorrow means that the day after tomorrow the game’s online player count will be low, and this launches a new wave of probability: “the game is dying,” and even loyal fans turn away from it.
• DBN calculates this step-by-step dynamic process, helping the publisher understand on exactly which day and with what probability it is critically important to release a patch in order to break this cycle.

DBNs are used for modeling financial markets, supply chains, the spread of information on social networks, and the modeling of feedback loops.

Agent-Based Modeling (Agent-Based Modeling)

And now a fundamentally different approach. Instead of deriving equations for the whole system, we create a digital crowd and watch what it does.

How it works:

1. We create thousands (or millions) of virtual “agents.”
2. We give each one simple rules of behavior: “If there is free space in front of you, go forward. If there is a wall in front of you, turn right. If you sense panic, run.”
3. We run the simulation.

And at the macro level a complex behavior emerges that no one programmed: traffic jams, panicked evacuation through bottlenecks, “waves” of shoppers in a supermarket.

A curious fact. In 1986, Craig Reynolds created a simulation of a flock of birds — Boids. Each virtual “bird” had only three rules: (1) stay close to your neighbors, (2) do not collide, (3) fly in the same direction as your neighbors. And the flock maneuvered realistically without any “leader” or central planner. This very idea — complex behavior from simple rules — lies at the core of agent-based modeling.

Where it is applied: stadium evacuation, panic spread, stock market dynamics, rumor spread, urban planning.

Strength: it allows you to see unexpected effects that cannot be derived analytically. You simply run the simulation and observe.

Weakness: the results are very sensitive to the very rules you “sewed into” the agents. If the rules are not realistic, the simulation will show a beautiful but false picture. Garbage in — garbage out, as programmers say.

Type 3: Events Dependent on Specific Individuals

Why This Is the Hardest Type

Here the outcome depends on the decision of one person or a small group. Not a crowd, not nature — the will and desires of specific individuals.

Particular difficulties:

• Hidden information: you do not know the true intentions, resources, and constraints of the opponent.
• Strategic thinking: the opponent is thinking about what you think about him. “I know that you know that I know that you know…” — a potentially infinite recursion.
• Bluffing and signals: a person can deliberately deceive. Nature or a crowd — no.

Examples: will the leader of a country start reforms? Will the board of directors agree to a merger? Will the president appoint this minister?

Game Theory with Incomplete Information (Bayesian Games)

The basic idea. We model the situation as a “game”: there are players, each has a set of possible actions (strategies), and each receives some “payoff” depending on what actions everyone chose.

Incomplete information. In the real world you do not know the opponent’s true “payoffs.” Is the economic outcome more important to him, or political survival? Is he aggressive or cautious?

In Bayesian games each player has a hidden “type”, and you have only a probabilistic assumption about it: “With probability 70% the opponent is the aggressive type, with 30% — the cautious one.”

Bayesian updating in action. You observe the opponent’s actions and update your estimate of his type:

• You expected the aggressive type to start with an ultimatum → the opponent started with an ultimatum → the estimate “he is aggressive” rises from 70% to 85%.
• You expected the aggressive type to respond to sanctions with escalation → the opponent did not respond → the estimate “he is aggressive” falls.

This is a continuous process: each action is a new piece of information.

A real example. During the Cuban Missile Crisis of 1962, the Kennedy administration continuously updated its estimate of Khrushchev’s intentions based on his actions: the deployment of missiles → public statements → the tone of diplomatic notes → the movement of ships. Each signal changed the estimate of the probability of a nuclear conflict.

Signals: Expensive vs Cheap

The opponent does not merely act; he sends signals. And the forecaster’s main question is: whom to believe?

• A costly signal (costly signal): one that is hard or expensive to fake. Military exercises involving the mobilization of reservists near the border are expensive (both financially and politically). Withdrawal from an international treaty is expensive (reputational losses). Such signals carry more information.
• A cheap signal (cheap talk): a statement on Twitter, diplomatic rhetoric, “we are not planning to…”. It costs zero. It may be a bluff, it may be the truth; without additional evidence you cannot tell.

A rule for the forecaster: weigh signals by their cost. A costly signal → greater weight. A cheap signal → lesser weight. And always ask the question: “Could the opponent afford to send this signal if he were bluffing?”

Red Teaming: Play the Opponent

The idea. Assemble a team, give it the role of the opponent — with his goals, constraints, resources, worldview — and ask it to come up with the worst scenario for you.

• In business: “You are the board of directors of our main competitor. Your budget is X. Your pain points are Y, Z. What would you do to push us out of the market?”
• In military affairs: “You are the adversary’s command. Your goals are A, B, C. Your constraints are D, E. What operation would you plan?”

Why this works. Red teaming overcomes “mirror imaging” (mirror imaging), one of the most common mistakes in analysis: attributing your own logic, values, and risk tolerance to the opponent.

A classic example: American analysts for years underestimated North Korea’s willingness to endure economic sanctions, because they measured it by their own standards: “who would suffer like this for the sake of a nuclear program?” But the values and tolerance for suffering in Pyongyang are different. Red teaming forces you to temporarily “step into the shoes” of the opponent.

Structured Expert Judgment

When the event is unique and there is almost no data, all that remains is to rely on experts. But how exactly?

Simply surveying experts is a bad method. You get “an expert said 70%,” but you do not know:

• Is this expert calibrated at all? (Perhaps when he says 70%, he is right in only 40% of cases.)
• Has he not fallen under groupthink?
• Is the discussion dominated by the most influential, not the most accurate?

Structured expert judgment (Structured Expert Judgment) is a formal protocol developed for nuclear safety and volcanology:

1. First, experts are given calibration questions with known answers (“what was the temperature in London on July 15, 2019?”). This allows you to measure how accurate and calibrated each expert is.
2. Then — the target questions (the ones that interest us).
3. The experts’ answers are weighted by their calibration. More accurate experts receive greater weight in the final forecast.

This is not ideal (an expert who is accurate on calibration questions may be wrong on target ones). But it is significantly better than simply “asking an expert.”

Bueno de Mesquita: Game Theory in Real Forecasting

Bruce Bueno de Mesquita is a political scientist, a professor at New York University and Stanford, a consultant to the CIA and the US State Department. But most importantly — he created a computer model that predicts the outcomes of political negotiations, conflicts, and decisions with an accuracy that he himself estimates as “significantly better than the average expert.”

How his model works:

Imagine negotiations on a nuclear deal with Iran. In the room there are dozens of players: the president of Iran, the supreme leader, European diplomats, the American president, Congress, the Israeli lobby, Russian and Chinese representatives. Each has their own interests, their own level of influence, and their own persistence.

Bueno de Mesquita’s model formalizes this into four parameters for each player:

1. Position: what does the player want? For example, on a scale from 0 (full nuclear disarmament of Iran) to 100 (Iran with nuclear weapons without restrictions).
2. Power: to what extent can the player change the outcome? The US president has more leverage than, say, the Austrian foreign minister.
3. Salience: how much does the player care? For Iran this is a question of national security — salience 90%. For Brazil — zero.
4. Flexibility (resolve/flexibility): is the player ready for a compromise, or will he fight to the death?

The model simulates the negotiations: the players take turns making proposals, react to others’ moves, and the algorithm looks for the point where further bargaining no longer changes the outcome. This is not the classical Nash equilibrium; this is a heuristic simulation of the negotiation process. This point is the forecast: “the negotiations will end with a compromise at the mark of 42 — Iran will keep a limited nuclear program under international control.”

Is it really 90% accurate? Bueno de Mesquita has documented hundreds of forecasts: from peace negotiations in Ireland to political reforms in China. Independent verification (in particular, research published in PS: Political Science & Politics) confirmed that his model is indeed significantly more accurate than the average expert.

Why this matters to us:

Bueno de Mesquita showed that game theory is not abstract mathematics but an engineering tool that can be applied to real political events. His book The Predictioneer’s Game (2009) is a step-by-step description of how the model works, with real cases.

And he also wrote The Dictator’s Handbook (together with Alastair Smith) — a book about how leaders make decisions when their main goal is to stay in power. This is a brilliant complement to game theory: what will a leader do if his priority is political (and physical) survival.

Where Superforecasters Came From: Philip Tetlock, Foxes, Hedgehogs, and the IARPA Tournament

If Bruce Bueno de Mesquita (Bruce Bueno de Mesquita) is “how to apply game theory to predict a specific political decision,” then Tetlock is “why experts are wrong and how to fix it.”

2005. The psychologist Philip Tetlock (Philip Tetlock) from the University of Pennsylvania published the book Expert Political Judgment. He and his team collected 28,000 forecasts from 284 political experts over 20 years and compared them with reality.

The result was shocking. The average expert turned out to be no more accurate than a chimpanzee throwing darts at a board. Or, to be completely precise, no more accurate than random guessing taking base rates into account.

But Tetlock discovered something more important. Not all experts were equally bad. Some were systematically better than others. And this difference was described by an ancient metaphor.

Foxes and Hedgehogs

“The fox knows many little things, but the hedgehog knows one big thing.” Archilochus, ancient Greek poet (7th century BCE)

This metaphor was revived by the philosopher Isaiah Berlin in a 1953 essay, dividing thinkers into two types. Tetlock applied it to forecasters, and it turned out that it perfectly describes who forecasts well and who badly.

Hedgehogs:

• Style of thinking: one big idea. Everything is explained through it.
• Attitude to contradictions: they avoid them. “This does not fit my theory — which means it is a throwaway.”
• Confidence: high. They love the word “therefore” and “without a doubt.”
• They are often invited on TV: they are confident, they give clear answers, they look good on camera.
• Accuracy of forecasts: low. And the more confident the hedgehog, the worse the forecast.

Foxes:

• Style of thinking: many small ideas. Eclecticism.
• Attitude to contradictions — they accept them. “Perhaps I am wrong; let’s look at this from the other side.”
• Confidence: low. They love the words “perhaps,” “on the one hand… on the other hand…”
• They are almost never invited on TV. “Well, it’s complicated…” — a bad format for television.
• Accuracy of forecasts: high. The more a forecaster is like a fox, the more accurate his forecast.

Examples of hedgehogs: Marxists (everything is explained by class struggle), Friedmanites (everything is explained by monetary policy), techno-optimists (progress will solve everything), collapseologists (everything is flying into the abyss). Each of them is sometimes right, but they do not know how to be right systematically, because the world does not fit into one theory.

Examples of foxes: people who say “it depends on…”, who read different sources, who change their opinion when new data appears. They rarely get on television, but it is they who win forecasting tournaments.

Tetlock’s discovery was at once pessimistic and optimistic. Pessimistic — because most “experts” are no better than chance. Optimistic — because you can be better. You just need to think like a fox.

The IARPA Tournament and the Birth of Superforecasters

In 2011, the American intelligence agency IARPA (Intelligence Advanced Research Projects Activity — the research wing of US intelligence) decided to check: can people be taught to be foxes?

They announced a tournament. Five teams, thousands of volunteers, hundreds of questions about real geopolitical events: “Will Greece leave the eurozone?”, “Will North Korea conduct a nuclear test this year?”, “Will the price of oil cross $100 by the end of the quarter?”

Tetlock led one of the teams — Good Judgment Project. And he did something no one had done before him: he started training the forecasters. Not giving them secret information, but teaching them to think.

The result surpassed everything. Tetlock’s team did not simply win; it crushed the rivals. A small group inside the team, about 2% of participants, showed results so impressive that they were called superforecasters (superforecasters).

They:

• Outperformed all other participants in the tournament.
• Outperformed professional CIA analysts who had access to classified data.
• Showed a Brier Score of about 0.15–0.20 — significantly better than random guessing (0.25), but far from ideal (0.0). This means: even the best forecasters are wrong often; they just honestly acknowledge their uncertainty.

And the main thing: they were not geniuses. These were ordinary people: engineers, pensioners, housewives, lawyers. What united them was not IQ, not education, not access to information. What united them were specific thinking techniques.

In 2015, Tetlock and co-author Dan Gardner published the book Superforecasting (Superforecasting), where they described these techniques. Here they are.

Outside View: Start with the Base Rate

“This project is unique. No one has ever done anything like this before.” The phrase that has killed more forecasts than any other.

Inside View vs Outside View. In the 1970s, the psychologists Daniel Kahneman and Amos Tversky (a future Nobel laureate) noticed a strange thing. When people forecast, they almost always use the inside view: they focus on the unique details of this particular case.

But there is an alternative — the outside view: first look at the statistics of the class to which the event belongs.

Example — a forecast of a startup’s success:

• Inside view: “What a cool team! What an innovative product! The market is enormous!” → forecast: a 30% chance of success.
• Outside view: “99.5% of startups do not become unicorns. This is the base rate — 0.5%.” → then we look: is this startup unique enough to raise the rate? Perhaps to 2%? To 5%? But certainly not to 30%.

Tetlock’s superforecasters always start with the outside view. This is discipline: first the class statistics, then the individual details. Not the other way around.

Kahneman and Tversky themselves became victims. When they were writing a psychology textbook, they estimated the probability of finishing it in a year as “very high.” Outside view: most textbooks take more than a year to write. Inside view: “but we are Kahneman and Tversky!” They finished the textbook only after 8 years.

Fermi Estimation: Break the Complex into the Simple

Enrico Fermi is an Italian physicist, a Nobel laureate, the person who launched the first nuclear reactor. And he was also famous for his ability to estimate the impossible through simple components.

Fermi’s classic question: “How many piano tuners are there in Chicago?”

You do not have this number. But you can approximately compute it:

• Population of Chicago: ~3 million.
• Average family size: ~3 people → ~1 million households.
• Percentage of households with a piano: ~2% (approximately) → ~20,000 pianos.
• Plus schools, concert halls, studios: another ~5,000 → a total of ~25,000.
• A piano is tuned ~1 time per year → 25,000 tunings.
• One tuner does ~2 tunings per day × 250 working days → 500 tunings per year.
• 25,000 ÷ 500 = ~50 tuners.

This is called Fermi estimation. Each individual component is approximate. But errors tend to cancel each other out (one overestimated, another underestimated), and the final answer comes out surprisingly close to reality. By the way, in Chicago there really are about 50–60 piano tuners.

Application to event forecasting:

“What is the probability of a military conflict in region X over 5 years?”

• Base rate (outside view): how many conflicts have there been over the last 50 years in similar regions? → ~15%.
• Next, a structured factor analysis: enumerate all relevant factors (an economic crisis, domestic political instability, territorial disputes, external players), estimate their influence on the event, and then integrate into a Bayesian model.
• Important: it is precisely the breaking down into factors that is useful — it forces you to think systematically.

Bayesian Updating: Change Your Mind Often and Little by Little

Most people treat their forecasts like a verdict: “I thought for a long time and decided — the probability of the event is 60%!” And then they defend this number, ignoring new information.

Superforecasters do it differently. For them a forecast is a living estimate that they constantly update.

• Not this: “I said 60% — and this is my iron position.”
• But this: “New information: a hardline defense minister has been appointed. This raises the probability of conflict. My forecast: it was 60% → it became 65%.”

Many small updates, not one big “I have fundamentally changed my mind.” This is Bayesian thinking in practice: you constantly, little by little, correct your estimate based on new data.

An everyday analogy. You wake up and want to predict whether there will be rain today or not. The base probability of rain is 20% (the climatic norm). You look out the window: cloudy → 45%. You get dressed, go outside: something is dripping → 80%. You hear thunder → 95%. This is Bayesian updating in its pure form — you have been doing it all your life without knowing the name.

Keeping Score: Keep the Score

Without feedback you will not learn. This is an axiom.

Imagine a basketball player who practices shooting with his eyes closed. He does not see whether he hit or not. Will he shoot better after a year of training? No. Because there is no feedback.

It is the same with forecasts. If you do not write down your forecasts and do not check them after the fact, you do not know how good you are. And, what is worse, your brain rewrites history (hindsight bias — “I knew it all along!”), creating the illusion that you guessed more than you really did.

Superforecasters keep their own score. They write down the forecast, the date, the probability, and then, when the event is resolved, they compare. This is painful. Numbers do not lie. But it is the only way to improve.

How exactly to measure accuracy — about that next. For now, just a fact: keeping score by itself raises the accuracy of forecasts. When you know that your predictions will be checked, you think more carefully.

Collective Intelligence: When a Crowd Is Smarter Than an Expert

Up to this point we have talked about how one person can forecast events. But there is a separate, extraordinarily powerful class of tools: collective forecasting.

The central idea: do not try to be smarter than everyone else — aggregate the intelligence of many.

The Wisdom of the Crowd

1906, Plymouth, England. At an agricultural fair, Sir Francis Galton, Darwin’s second cousin, a statistician, an anthropologist, and by avocation a eugenicist (this is a separate, sad story), decided to conduct an experiment.

At the fair, 800 people tried to guess the weight of an ox. Galton, as an aristocrat of the Victorian era, was convinced that the “crowd” was ignorance, and that the average guess would be horribly far from the truth.

He collected all 800 slips of paper and calculated the median. The result: 1207 pounds. The actual weight of the ox: 1198 pounds. The error — less than 1%.

Galton was shocked. The “crowd” turned out to be almost perfectly accurate, provided that its estimates were independent.

Conditions under which the wisdom of the crowd works:

1. Independence of estimates. People must not consult with one another. If everyone watches the same television show, the wisdom of the crowd turns into the madness of the crowd.
2. Diversity of participants. Different experience, different knowledge, different viewpoints = a wider coverage of the space of possibilities.
3. A method of aggregation. The arithmetic mean or the median. Not “choose the most authoritative one.”

Confirmation from Tetlock. In the IARPA tournament, teams that aggregated the forecasts of several participants with a simple mean consistently outperformed any individual participant, even the best one. Ensemble > individual. Always.

Prediction Markets (Prediction Markets)

If the wisdom of the crowd is a theoretical idea, then prediction markets are its practical embodiment. And with money involved.

How it works. You create a contract: “This contract pays $1 if Candidate X wins the 2028 election, and $0 if he loses.” People trade this contract on an open market. If the contract costs $0.60, the market estimates the probability of winning at 60%.

Why this works better than surveys:

• Money on the line. People think honestly, not beautifully. If you are convinced that Candidate X will lose, you will earn by betting against him. If you simply “want” him to lose, you will lose money if you are wrong. The market disciplines.
• Aggregation of dispersed knowledge. No expert knows everything. But someone knows about the economy, someone about demographics, someone about the mood in a key state. The market collects these knowledge into one through the price mechanism.

Iowa Electronic Markets (IEM) is the oldest and most famous prediction market, launched by the University of Iowa in 1988. In the presidential elections of 1988–2012, IEM was more accurate than polls in 5 out of 6 cases. And there is no magic — only collective wisdom backed by money.

Modern platforms: Polymarket, Metaculus, PredictIt. Metaculus, by the way, does not use money, only reputation points. But even this works: people want to be accurate when accuracy is publicly visible.

Delphi Method (Delphi Method)

History. At the beginning of the Cold War, the American military faced a problem: how to estimate how many atomic bombs would be needed to destroy the military-industrial potential of the USSR? The experts’ estimates differed by orders of magnitude. How to combine them so that the loudest voice does not drown out the most intelligent one?

The RAND Corporation (the very “think tank” that gave the world game theory, scenario planning, and in general a significant part of this article’s toolkit) developed a method that was ironically named Delphi — in honor of the Delphic oracle, who foretold the future by inhaling vapors from a cleft.

How Delphi works:

1. Round 1. Each expert anonymously gives their forecast with a justification.
2. Statistics + arguments. Everyone sees the anonymized statistics of the answers (median, range) and the best arguments for different positions. Without names. Without status.
3. Round 2. The experts can correct their forecast under the influence of arguments, not authority.
4. We repeat, until the estimates stabilize.

What Delphi removes:

• The effect of “the loudest voice in the room” (HiPPO — Highest Paid Person’s Opinion).
• Groupthink (groupthink) — when everyone agrees because they are afraid to contradict.
• The status effect — when the opinion of a Nobel laureate weighs more than the arguments of a PhD student, even if the PhD student is right.

Modern application: technological forecasting, risk assessment of new drugs, healthcare planning, military analysis.

Ensembles of Forecasts

This is the simplest (and at the same time most powerful) conclusion from the research on collective intelligence:

A simple mean of several independent forecasts is almost always better than any individual forecast.

Even if all the forecasters are mediocre. Even if none of them is an expert.

Why this works. The errors of independent forecasters cancel each other out. One systematically overestimates the risks (a pessimist), another underestimates them (an optimist). The mean is closer to the truth than each one individually.

A practical conclusion: if you take event forecasting seriously, do not rely on one person. Not even on yourself. Collect 3–5 independent estimates and take the mean. This takes 10 minutes and improves accuracy more than a week of additional analysis “by one person.”

How to Measure the Accuracy of Forecasts

In time series this is simple: you compare the forecast with the fact — you get an error in degrees Celsius, hryvnias, or units of goods sold.

In events everything is more complex.

The forecast: “60% that Candidate X will win.” Candidate X won.

Is this good for the “60%” forecast? Or bad? And what if the forecast was “90% that Candidate X will lose,” and he lost? And if the forecast was “99% that Candidate X will win,” and he lost — how much worse is this than a forecast of “51% that he will win”?

We need a numerical measure that takes into account not just “guessed / did not guess,” but also the degree of confidence.

Brier Score

In 1950, the meteorologist Glenn Brier (Glenn Brier) proposed a simple and elegant formula. It measures the mean squared error between the forecast (a probability) and the fact (0 or 1).

Intuitively:

• Forecast 100% → fact 1 (it happened) → error 0 (an ideal forecast).
• Forecast 100% → fact 0 (it did not happen) → error 1 (a catastrophe — was certain and wrong).
• Forecast 50% → any fact → error 0.25 (acknowledged full uncertainty — not good, but not terrible either).
• Forecast 70% → fact 1 → error 0.09 (good — was cautiously confident and guessed right).

The lower the Brier Score, the more accurate the forecaster. Random guessing (50/50) gives a Brier Score ≈ 0.25. Professional superforecasters have a Brier Score of about 0.10–0.15.

Why the Brier Score is better than “% of correct guesses”:

It punishes excessive confidence. If you say “99%” and you are wrong, you get a huge penalty (0.98). If you say “51%” and you are wrong, the penalty is much smaller (0.26). This forces the forecaster to honestly report their uncertainty, rather than “play to the public” with confident statements.

Calibration (Calibration)

The Brier Score is a general measure. But there is also a deeper diagnosis: calibration.

Definition. A forecaster is calibrated if, in all cases where he said “X%,” the event really happened in X% of the cases.

• Says “70%” → it comes true in ~70% of such forecasts → calibrated.
• Says “90%” → it comes true in ~50% of cases → excessively confident (the most common problem).

A stunning fact. Research shows that when ordinary people say “I am 90% sure,” the correct answer occurs in only ~50% of cases. When people say “I am 100% sure,” they are right in ~70–80% of cases.

Even experts: doctors making a diagnosis, engineers assessing risks, financial analysts — are systematically excessively confident.

Tetlock on experts. In his famous book Expert Political Judgment (2005), Tetlock showed that political experts who appear on television forecast events no better than a chimpanzee throwing darts at a board. But there is a nuance: the more confident the expert, the worse his forecast. Self-confidence is inversely proportional to accuracy.

A good forecaster is not the one who is confident. It is the one who is calibrated. And calibration is a skill that can be trained. This is exactly what the Brier Score is for.

Cognitive Traps: Why Our Brain Is Systematically Wrong

Event forecasting is not only about methods. It is about psychology. Our brain is a product of evolution, optimized for survival in the savanna, not for estimating the probability of a default on Argentine bonds. And it makes systematic errors.

Excessive Confidence (Overconfidence)

The most common — and the most dangerous — cognitive trap.

An experiment. Ask 100 people: “Estimate the length of the Nile River with 90% confidence, that is, give a range in which the correct answer should fall with probability 90%.”

In theory, the correct answer should fall outside the range in only 10% of cases. In practice, in 40–50%. People give too narrow ranges. They are excessively confident in their knowledge.

It is the same with event forecasts. “I am 90% sure that this will happen.” In reality, this happens in 50–60% of cases.

How to fight it. There is only one way — the Brier Score. To see the number of your excessive confidence. This is painful, but it cures. Superforecasters who receive feedback through the Brier Score for months gradually calibrate themselves and start giving more honest estimates.

Conjunction Fallacy (Conjunction Fallacy)

The classic experiment of Kahneman and Tversky — the “Linda Problem” (1983):

Linda is 31 years old. She is intelligent, straightforward, and graduated from the philosophy faculty. During her student years she was actively interested in issues of social justice and took part in protests.

What is more probable? (A) Linda works as a bank employee. (B) Linda works as a bank employee and is an activist of the feminist movement.

85% of people choose (B). But (B) is a subset of (A). Feminist bank employees are a part of all bank employees. A part cannot be larger than the whole. Mathematically, the probability of (B) is always less than or equal to the probability of (A).

Why do people make a mistake? Because (B) is more detailed, more expressive, more like a story. And our brain confuses “similarity to the truth” with “probability.”

A consequence for forecasting. A detailed scenario sounds more convincing, but it is always less probable than a broader, less detailed one. Each added “and…” lowers the probability:

• “Company A will acquire company B” → probability P.
• “Company A will acquire company B and this will happen in March” → probability P×(something<1) = less than P.

But people intuitively give the highest estimate to the most detailed scenario, because it “sounds good.” This is a direct road to bad forecasts.

Inside View vs Outside View

We have already talked about this, but it is so important that it is worth repeating as a trap.

Each situation seems unique. “This time is special.” “This candidate is not like the others.” “This crisis is unprecedented.”

Statistically, almost always no. Most situations belong to some class, and the base rate for that class is your best friend.

Superforecasters disciplinedly start with the outside view, even when they “really want to” dive into the unique details.

Hindsight Bias (hindsight bias)

“I knew all along that this would happen!”

No, you did not know. If you had written down your forecast before the event, you would have seen that you gave this 20%, not “I knew it all along.”

Hindsight bias is the brain’s tendency to rewrite the past as if it had been predictable. After the event has happened, it seems “obvious.”

This is not just a funny quirk of the psyche. This is the main obstacle to learning. If past events seem “obvious,” then you do not see your mistakes. You do not calibrate yourself. You do not improve. You simply live in the illusion that “next time I will guess it.”

The only defense: write down forecasts before the event is resolved. Paper does not lie. The Brier Score number does not lie. This is your only weapon against self-deception.

Rare Events and Early Warnings

Everything we have talked about so far works for events that happen quite often: elections, economic crises, bankruptcies, mergers. But there is a separate class — rare, catastrophic events. The very “black swans” of Nassim Taleb.

Why Rare Events Are a Separate Problem

Standard methods work poorly when the event happens once in 50 years. You simply do not have enough data. How to estimate the probability of a global pandemic if the last one (before COVID-19) was in 1918?

Another problem: in rare events the stakes are asymmetric. The mistake “did not predict a flood” costs much more than the mistake “predicted a flood, and there was none.” In the first case, victims and destruction. In the second, a needless evacuation. This changes the very logic of forecasting.

Early Warning Signals (Early Warning Signals)

In complex systems — ecosystems, financial markets, political regimes — before a “phase transition” (a collapse, a crisis, a revolution), characteristic signs often appear:

• Critical slowing down (critical slowing down). Imagine a ball in a hollow. You push it — it wobbles and returns to its place. But the closer the system is to the tipping point, the longer it returns to equilibrium after a shock. This can be measured, and it works for ecosystems (before the collapse of a fish population), financial markets (before a crash), and even for the human psyche (before a depressive episode).
• Growing dispersion. The indicators start to “jump” more strongly — the amplitude of oscillations grows. The system becomes less stable.
• Flickering (flickering). The system starts switching between states: now normal, now crisis, now normal again. This is like a light bulb that flickers before burning out.

An example from the real world. Researchers discovered that before the “Arab Spring” of 2011, exactly such “flickering” was observed in the moods of social networks in the relevant countries: periods of calm alternated with bursts of protest activity, which became more and more frequent. This was not a forecast of the date, but it was an early warning of the approach of a phase transition.

Extreme Value Theory

A separate branch of statistics that deals not with the “average,” but with the “tails” of a distribution — rare, extreme events.

• Not “what will be the average temperature in July,” but “what is the probability of anomalous heat of +45°C, which happens once in 100 years.”
• Not “what will be the average return of a portfolio,” but “what is the probability of losing 40% of capital in a week.”

Where it is used: flood insurance, the design of dams (what water level to design for: a 10-year, a 100-year, or a 1000-year flood?), financial risk management, seismic hazard assessment.

For an event forecaster this means: do not ignore the “tails.” The biggest consequences (and the biggest opportunities) often lie exactly where “normal” statistics do not even look.

A Comparative Overview of the Three Types of Events

Type 1: Nature

• What drives it: physical/biological laws
• Is there “will”: no
• Stability of the “rules”: high (the laws do not change)
• Main methods: physical models, SIR, statistics of rare events
• Does the forecast influence the event it predicts: no (a hurricane does not care)
• Example: forecasting weather, epidemics

Type 2: Many Agents

• What drives it: mass behavior
• Is there “will”: yes, in individual agents, but it is diluted in the mass
• Stability of the “rules”: medium (they change over time)
• Main methods: Bayesian networks, DBN, agent-based modeling
• Does the forecast influence the event it predicts: yes! (markets react to forecasts)
• Example: stock price, elections, fashion

Type 3: Specific Individuals

• What drives it: strategic choice
• Is there “will”: yes, a central role
• Stability of the “rules”: low (a person can change their mind)
• Main methods: game theory, red teaming, expert judgment
• Does the forecast influence the event it predicts: yes! (a leader reacts to forecasts)
• Example: invasion, merger, deals

How to Choose an Approach, a Practical Algorithm

Three Questions Before the Event:

1. Who or what determines the outcome?

• Nature → Type 1. Physical/biological models.
• People → go to question 2.

2. How many people influence the outcome?

• Millions/thousands → Type 2. Bayesian networks, agent-based modeling.
• Units/tens → go to question 3.

3. Do the key individuals think strategically?

• Yes (they think about each other’s actions) → Type 3. Game theory, red teaming.
• No (they simply follow a procedure) → perhaps Type 2 (bureaucratic systems).

4. Is the uncertainty too high for a point estimate?

• Yes → scenario planning (from Article 1): instead of one probability — several coherent “worlds” in which you test your strategy.

Universal Rules for Any Event:

1. Start with the base rate (outside view). “How often does something like this happen?”
2. Break the complex into the simple (Fermi estimation). Estimate the components and their influence on the base probability.
3. Update the estimate often and little by little (Bayesian updating). Not a “verdict,” but a “living estimate.”
4. Keep score (Brier Score). Without numbers you do not know how good you are.
5. Aggregate opinions (ensemble). The mean of 3–5 independent estimates > any single one.
6. Do not be excessively confident. If you say “probability = 90%,” check whether you are really right in 90% of similar cases.

Try It Yourself: Your First Step into Forecasting

You can read ten books about riding a bicycle and still fall at the first attempt. Forecasting is the same: it is a skill, not knowledge. And the only road to it is practice.

You can start today, free of charge and without risk.

1. Register at Metaculus — a platform where thousands of people forecast real events every day: from elections to scientific discoveries. There is no money there, only reputation and an honest score of your accuracy.
2. Make 10 forecasts this week. Choose questions that will be resolved within 1–3 months. Use the tools from this article: start with the base rate, break the complex question into simple ones, and write down your probability.
3. Update. Once a week, return to your forecasts: has new information appeared? Change your estimate little by little, as superforecasters do.
4. Come back in a year and look at your Brier Score. The platform will calculate it for you. You will be impressed first by how poorly calibrated you are (we all are), and then by how quickly you learn.

Congratulations — you are no longer a reader about forecasting. You are a forecaster.

Tools for Event Forecasting

Python Libraries

• Bayesian networks and causal analysis: pgmpy, pymc, DoWhy, CausalML, EconML
• Game theory: Nashpy, Gambit, Axelrod, OpenSpiel, PyCID (combines causal graphs with game theory).
• Agent-based modeling: Mesa

Platforms and Applications

Platforms for practice:

• Good Judgment Open (gjopen.com) — a platform from Tetlock’s team. The best start for a newcomer to event forecasting.
• Metaculus (metaculus.com) — forecasting real events. You choose a question, give a probabilistic forecast, and get a Brier Score. Without money.
• Polymarket, Kalshi — prediction markets with real money. The price of a contract = the collective estimate of probability.

Building Bayesian networks and causal models:

• TETRAD (cmu.edu/dietrich/philosophy/tetrad) — a free tool from Carnegie Mellon University for finding causal links in data.
• GeNIe (bayesfusion.com) — a graphical editor of Bayesian networks. You create a graph of causal links with a mouse. There is a free academic version.
• Bayes Server (bayesserver.com) — a more powerful tool for Bayesian networks. It has dynamic Bayesian networks. There is a free trial version.

What to Read Next

• Philip E. Tetlock, Dan Gardner — Superforecasting: The Art and Science of Prediction (2015). The practical techniques of superforecasters. If you can choose only one book about forecasting — choose this one.
• Philip E. Tetlock — Expert Political Judgment: How Good Is It? How Can We Know? (2005). The very book with the 28,000 forecasts, foxes, hedgehogs, and “no better than a chimpanzee with darts.” More academic, denser than Superforecasting — for those who want the evidence base.
• Bruce Bueno de Mesquita — The Predictioneer’s Game: Using the Logic of Brazen Self-Interest to See and Shape the Future (2009). Game theory in real forecasting of political events. With real cases and an explanation of the model.
• Daniel Kahneman, Olivier Sibony, Cass R. Sunstein — Noise: A Flaw in Human Judgment (2021). About the random variability of judgments. Why two experts give different forecasts on the same data and what to do about it.
• Judea Pearl, Dana Mackenzie — The Book of Why: The New Science of Cause and Effect (2018). From the creator of Bayesian networks — about why correlation is not causation and how to teach machines (and yourself) to reason about causes. Written for a broad audience, without formulas.

Next: Article 4 — “Event Stream Forecasting: When Events Come One After Another.” What to do when events are not isolated but come in a stream and influence one another: queues in a supermarket, calls to a call center, aftershocks after an earthquake, viral posts. From classical queueing theory to neural networks that learn to predict cascades of events.