Ask whether artificial intelligence can trade financial markets and you'll get two confident, opposite answers. One camp sells you a bot that supposedly turns a few hundred dollars into a fortune while you sleep. The other insists markets are efficient and any edge is luck dressed up as skill. Both answers skip the only question that matters: if an AI *were* trading well, how would an outsider ever verify it?
That question — verification, not capability — is where almost every "AI trading" claim quietly falls apart.
The Claim That Can't Be Wrong
Here's a thought experiment. Someone shows you a chart: a smooth equity curve climbing up and to the right, labeled "our AI's live performance." What can you actually check?
Usually, nothing. The screenshot could be a backtest relabeled as live. The start date could have been chosen after the fact. Losing periods could be cropped out. The strategy could have been one of fifty, and you're seeing only the survivor. Because the record was assembled *after* the results were known, it can be edited to say whatever the seller needs it to say.
A claim you cannot disprove isn't a strong claim — it's a marketing asset. The scientific word for a result that no possible evidence could contradict is *unfalsifiable*, and unfalsifiable is exactly what most AI-trading pitches are. The model might be brilliant. You have no way to know, and neither does the person selling it, because they never committed to anything before the outcome landed.
Why Backtests Flatter the Machine
Most impressive AI-trading numbers come from backtests — running a model over historical data. We've written elsewhere about survivorship bias and overfitting (more on the [NeuPortal blog](https://neuportal.ai/blog)), so we'll keep this brief: a backtest is a hypothesis, not a track record.
The core problem is that history is fixed and the model gets to peek. Tune enough parameters against the same dataset and you can fit almost any curve — you've memorized the past, not learned the future. Then add the frictions a backtest usually ignores: slippage, fees, the market impact of your own orders, data that arrived late in real life. The beautiful curve often flattens or inverts. A backtest tells you a strategy *would have* worked on the data it was built from. It says remarkably little about tomorrow.
Win-Rate Is the Wrong Scoreboard
When a live number does get shown, it's almost always win-rate: "correct 80% of the time." It sounds like skill. It's close to meaningless.
Win-rate ignores the *size* of wins and losses. A strategy can be right 90% of the time and still go broke if that other 10% is large enough — picking up pennies in front of a steamroller. Conversely, a trader right only 40% of the time can compound steadily if the winners dwarf the losers. Win-rate also says nothing about *risk*: how much capital was exposed, how deep the losing streaks ran, whether the returns came from genuine edge or from quietly taking on tail risk that simply hadn't blown up yet.
Selling win-rate is easy precisely because it hides the two things that actually determine whether a strategy survives: payoff asymmetry and risk taken.
The Metric That Survives Scrutiny: Risk-Adjusted Returns
Serious evaluation asks a different question — not "how often was it right?" but "how much return did it earn per unit of risk?" That's what risk-adjusted metrics like the Sharpe ratio try to capture: return above a baseline, divided by the volatility endured to get it. Two strategies can post the same total gain while one delivered it smoothly and the other via a stomach-churning ride that no real capital could have held through. Sharpe — and cousins like Sortino — separate them.
Just as important is drawdown: the worst peak-to-trough loss along the way. A strategy's maximum drawdown is the real test of whether anyone could have actually stayed invested. Discipline around drawdown — position sizing, exposure limits, stop rules — is often the difference between a model that compounds and one that detonates. None of this shows up in a win-rate or a cherry-picked highlight reel. It only shows up when you watch the *whole* record, losses included.
What a Trustworthy AI-Trading Claim Would Look Like
Put the pieces together and the shape of a credible claim becomes clear. It has three properties:
1. **Pre-commitment.** The forecast or position is locked *before* the event, in a way that can't be quietly revised afterward. If you can't prove when a prediction was made, it doesn't count. 2. **Proper scoring.** Predictions are probabilities, and they're graded with a proper scoring rule — Brier score, log loss — that rewards calibration and punishes confident wrongness. "Right or wrong" becomes "how well-calibrated, measured continuously." 3. **Risk-adjusted, warts-and-all reporting.** Performance is judged on risk-adjusted returns and drawdown across the *entire* history, with losing periods on the board next to the wins.
Notice what this framework does: it makes the claim *falsifiable*. If the AI is bad, the public record will say so. That's the entire point. Accountability isn't a marketing garnish here — it's the only thing that separates a track record from a screenshot.
How We Run It in Public
This is the standard we hold ourselves to, and we want to be precise about what we are and aren't claiming.
We build agents that forecast and trade across live markets — crypto, sports exchanges, prediction markets like Polymarket and Kalshi. Before an event resolves, the relevant forecast is locked, hashed with SHA-256, and timestamped against the Bitcoin blockchain via OpenTimestamps, so the prediction's *existence and timing* can be verified independently later. Then it's scored in public using proper scoring rules — and both wins and losses go on the board at [neuportal.ai/experiment](https://neuportal.ai/experiment).
What we are *not* doing is telling you we made a specific amount of money, or posting an accuracy percentage stripped of context. Those are exactly the unfalsifiable claims this whole piece is about. Our strategies run on live data in paper and live-verify modes; the aim is a record outsiders can check, calibration you can watch accumulate over time, and risk-adjusted results reported honestly — not a highlight reel. If the approach doesn't hold up, the public scoreboard is where that will be visible first.
A Reader's Checklist
Next time you meet an AI-trading claim, ask:
- **Was the prediction locked before the outcome?** If it can't be timestamped, treat it as a story. - **Is it live, or a relabeled backtest?** Ask for the methodology, not the curve. - **What's the metric?** Win-rate and total return alone are red flags; ask for risk-adjusted returns and maximum drawdown. - **Are the losses shown?** A record without losing periods has been edited, not lived. - **Can you independently verify any of it?** If verification depends entirely on trust, trust is what you're being sold.
Can AI trade? Possibly — some of it, sometimes, within limits. But "can it trade" and "can you prove it traded well" are different questions, and only the second one protects you. The honest version of AI trading isn't a promise of profits. It's a record you're allowed to check.
Educational content — not financial advice.