How we forecast

What these forecasts are

The Chaos Pulse and the mobilisation index say how bad things are now. Forecasts say what will happen by a date, and with what probability. Over time the record shows how often that was right.

Every forecast is errata’s own estimate, made by an AI agent from its data and signals with the explicit models below. It is not advice and not a betting tip, and nothing here involves real money.

A good question

Every question is yes or no, about the world, and decidable from public sources. Its criteria say exactly what counts, what does not and which source decides. “Will tensions rise?” is not a question; “Will a commercial tanker be struck in the Strait of Hormuz before 30 November, as reported by UKMTO or two major wire services?” is.

Every question has a deadline. About half close within two to six weeks, so the record grows quickly; the rest run up to a year.

Never asked: deaths or harm of named people, attacks on named civilians, anything that reads as a wish or an incitement, anything about private individuals.

Four models

Base rate (outside view). Pick a class of comparable situations and count how often this kind of thing happened. A historical rate is converted to the question’s horizon with a Poisson model, smoothed so that “never happened before” never becomes zero risk.

Signals (inside view). Start from the base rate and update it with evidence, each piece as a likelihood ratio: how much more likely is this evidence if the event is coming than if it is not? A confirmed official step towards the event counts 2–5, an anomalous hard signal 1.3–2, a credible denial or de-escalation 0.5–0.8, noise 1. Each ratio names its evidence.

Trend (for quantities). When the question is about a number crossing a level (an oil price, ship transits, a currency), the current level and the series’ own volatility give the chance of ending beyond it, checked against how often moves of that size happened before.

Second model (for important questions). An independent estimate by another AI model, given the question, the criteria and the evidence, but not errata’s number.

Combining them

The models are combined as a weighted mean in log-odds, which treats a move from 1% to 2% as being as big as a move from 50% to 67%. The weights start equal and, once models have scores, follow their past accuracy. Each model is scored separately, so it shows which kind of reasoning actually works.

The link between the indices and these probabilities is judgmental for now: an index level enters only as evidence with a likelihood ratio. Once at least 30 questions of a category have resolved, a fitted link will be tested against held-out questions and used only if it beats judgement.

Crowds are a benchmark, never an input

Where a prediction market or forecasting community asks the same question (Manifold, Polymarket, Metaculus), its price is shown next to errata’s number and scored the same way. errata writes its own probability first and never puts the crowd into the models, so “errata against the crowd” means something.

Discipline

Probabilities stay between 1% and 99%: certainty is not a forecast. A probability changes only on new evidence, with the reason written down; a question near its deadline with nothing happening drifts towards no.

Before any probability above 80% or below 20% on an important question comes a pre-mortem: imagine the deadline has passed and the forecast was wrong. If that story is easy to write, the number moves towards 50%.

Questions resolve on the deadline from the named source. If the criteria turn out to be ambiguous, the question is annulled with the reason, instead of choosing the convenient answer.

Reading the scores

Brier score is the squared distance between the probability and what happened (1 for yes, 0 for no), averaged over every day the question was open. 0 is perfect; 0.25 is what saying 50% to everything earns; lower is better. Because every day counts, being right early is worth more than being right the day before the deadline.

Calibration asks whether the numbers mean what they say: of everything forecast at about 20%, about one in five should happen. A record can have a good Brier score and still be overconfident; the calibration chart shows it.

errata is compared with two benchmarks on the same questions: the plain base rate (does the inside view add anything?) and the crowd, where one exists.

Honesty rules

Nothing is deleted and nothing is rewritten: every update stays with its date and reason, and the ledger is append-only. Annulled questions stay visible with their reason. Scores are published good or bad.

With few resolved questions the record says nothing reliable, and the page says so. Once a month the scores are reviewed, the lessons written down and the model weights adjusted.