Talk is cheap in forecasting. Anyone can say their model is good; the hard part is letting someone else grade it, on a neutral field, in public. That is exactly why NeuPortal has entered two open AI forecasting competitions — and why we are writing this before we know how we place.
Our whole experiment is built on one idea: a forecast only means something if it is locked before the event, timestamped so it cannot be backdated, and scored honestly afterward — losses included. A public competition is the strongest possible version of that idea. It is someone else's scoreboard, someone else's data, and hundreds of other competitors. There is nowhere to hide.
Why Compete in Public Arenas
A live scoreboard on our own site is good evidence. A ranking in an independent, third-party arena is better. Nobody can accuse us of drawing our own chart, choosing our own benchmark, or quietly dropping the calls that went wrong. The competition fixes the data, the metric, and the timing, and then ranks everyone by the same rule.
That is why we treat these arenas as an extension of the experiment rather than a distraction from it. They test the same claim — that our probabilistic models are well calibrated — against a much harder, external standard.
CrunchDAO Synth: Probability Distributions, Scored by CRPS
The first arena is CrunchDAO's Synth challenge. It is a natural fit, because it asks for exactly what NeuPortal produces: not a single predicted price, but a full probability distribution over how a crypto asset's price will change over the next 24 hours, at multiple time resolutions.
Those distributions are graded with the Continuous Ranked Probability Score, or CRPS. If you have read our other posts, this will feel familiar: CRPS is the continuous cousin of the Brier score we already use to grade ourselves. Both are proper scoring rules, which means the only way to get a good grade over time is to report what you genuinely believe. The philosophy is identical; only the setting is new. Entry is free and there is no stake to put down.
Liquidity Arena: AI Agents on a Live Stage
The second is Liquidity Arena, a global championship for AI agents that opens on 20 July. We have registered for its Track A (Logic Frontier), the developer and research track, where systems are judged on the quality of their reasoning and forecasts rather than on deploying large amounts of capital.
Different arena, same principle: put the model on a public stage and let the results speak.
Our Honest Starting Line
Here is the part we care about most. We began with a deliberately plain baseline — the competition's own documented benchmark model — and we shipped it as our version one. We did that on purpose. Before claiming any edge, we tested a set of improvements, and we insisted on measuring them rather than assuming them.
The lesson was instructive. On clean, synthetic data our improvements made no difference at all, and we said so plainly. But on a backtest over seven days of real crypto prices, a better-calibrated model — one that tracks changing volatility and allows for the fat tails that real markets actually have — cut the CRPS by roughly four and a half percent versus the benchmark. That is a small, honest, measured number. It is not a promise of winning; it is a starting point we can stand behind, because we can show the measurement.
What We Will Publish
We will report how our models actually do on these leaderboards — the good rounds and the bad ones — the same way we publish every graded call on our own scoreboard. Across the calls we have scored so far in our own experiment, the market is still ahead of our model 11 to 4, and we have never hidden it. Competition results will get the same treatment.
If our approach is any good, an independent arena is where it should show. And if it is not, that will show too. Either way, you will be able to read the number. Follow along at neuportal.ai/experiment.
Educational content — not financial advice, and not a betting tip.