What Is the CRPS? Grading a Whole Probability Distribution

If you forecast whether it will rain tomorrow, a Brier score can grade you: you gave a probability to a yes/no event, and after the fact the Brier score measures how well that probability matched reality. But what if the thing you are forecasting is not yes/no — what if it is a number, like tomorrow's closing price of Bitcoin, and you predict not a single value but a whole distribution of where it might land? For that, you need the Continuous Ranked Probability Score, or CRPS.

It is the tool that lets us grade a genuinely probabilistic forecast of a continuous quantity, and it is the natural extension of the Brier score we already use.

From a Yes/No to a Whole Distribution

A good forecast of a number is not a point. If someone asks where an asset will trade tomorrow and you answer "62,300 exactly," you are almost certainly wrong and you have said nothing about your uncertainty. An honest forecast is a distribution: most of the probability mass around a central estimate, with tails covering the surprises. The better calibrated that distribution, the more useful it is.

The problem is grading it. Once the real number arrives — say the asset closes at 61,800 — how do you score a whole curve against a single realized point? That is exactly the question the CRPS answers.

What the CRPS Actually Measures

The CRPS compares your forecast's cumulative distribution — the curve that says "the probability the outcome is at most x" — against the outcome that actually happened. The realized value can itself be written as a step: below it, the "true" cumulative probability is 0; at and above it, it is 1. The CRPS measures the squared area between your predicted cumulative curve and that step, across the whole number line.

In plain terms: it rewards you for putting probability mass close to where the outcome landed, and penalizes mass you placed far away. A confident, well-aimed forecast scores low. A vague forecast that spreads mass everywhere scores worse, and a confident forecast aimed at the wrong place scores worst of all. Lower is better, and a perfect forecast — all your mass exactly on the outcome — scores zero.

CRPS and Brier Are the Same Idea

Here is the connection that makes the CRPS feel familiar. The Brier score is the CRPS for a binary event. When the outcome can only be 0 or 1, the "distribution" you forecast is just a single probability, the cumulative curve has one step, and the squared-area calculation collapses into the ordinary Brier formula. CRPS is Brier, generalized from a coin flip to a full number line.

That is why the two live so comfortably together. If you have read our posts on the Brier score, you already understand the CRPS: it is the same honesty test, applied to richer forecasts.

Why It Is a Proper Scoring Rule

Like the Brier score, the CRPS is a proper scoring rule. That means the only way to minimize your expected CRPS over time is to report the distribution you genuinely believe. You cannot game a better grade by pretending to be more certain than you are, or by hedging wider than your real belief. Overconfidence gets punished when the outcome lands in a tail you dismissed; underconfidence gets punished by the steady cost of vagueness. Honesty is, mathematically, the optimal strategy — which is exactly the property that makes a score trustworthy for comparing forecasters.

Where We Use It

We built NeuPortal on proper scoring rules because they are the only honest way to grade a probability. On our own public scoreboard we use the Brier score to grade discrete calls. In the public forecasting arenas we compete in, crypto price forecasts are graded with the CRPS — the same philosophy, applied to full distributions of where a price might go. Different score, identical principle: state your real uncertainty, then let the number, not the narrative, decide whether you were any good. You can follow our graded record at neuportal.ai/experiment.