Any model can look clever on the data it was built from. That is not a sign of intelligence, it is a sign of memory. The test is whether it holds up on data it has never seen.
Two checks do most of the work.
When the model says 30%, do those things happen about 30% of the time?
Sort every prediction into buckets — everything it called 20-25%, everything it called 25-30%, and so on — then check what actually happened in each bucket. A well-calibrated model produces a straight line. A badly calibrated one is systematically confident where it has no right to be, which is the failure mode that quietly empties bank accounts.
Ours is well calibrated on match results, and was not always. An earlier version was over-confident on goal totals; the fix was to pull team ratings partway back toward average before predicting, which compresses the spread and stops the model believing its own strongest opinions too readily.
This is the check that catches self-deception, and it is the one most people skip.
Take the same machinery and feed it scrambled data — the structure preserved, the signal destroyed. Any "edge" that survives that was never an edge. It was the testing procedure finding patterns in noise, which testing procedures will do all day if you let them.
We run a placebo alongside every result we publish. If the real version does not clearly beat the scrambled version, the result does not get claimed.
Across three separate seasons, including one the model had never been built on, it predicts measurably better than chance — roughly 0.6 percentage points of skill above the placebo, replicated each time. Adding richer inputs beyond xG — league position, recent form, rest days between matches, head-to-head history, 45 features across 11,436 matches — made it a sharper predictor again, and that improvement also held up out of sample.
So: a genuinely better predictor, confirmed by the two checks that are designed to catch people fooling themselves.
Which is not the same thing as a way to make money, and we will keep saying so. Better prediction and profitable betting are different claims with different evidence bars, and ours clears the first and not the second.