Cryptocurrency

Why a Strong AI Trading Backtest Proves Less Than it Seems, According to NAGA's Experts

Written By : Market Trends

Anyone who has trained a machine learning model knows the first trap. A model can score almost perfectly on the data it was trained on and then fall apart on data it has never seen, because it has learned the noise along with the signal. Data scientists call this overfitting, and they guard against it almost by reflex, with held-out test sets and cross-validation.

Trading strategies suffer from exactly the same problem, and AI tools have made it far easier to fall into. A language model or an automated optimiser can generate and test hundreds of variations of a strategy in a fraction of the time a single test once took. Every extra variation is another chance to find a backtest that looks excellent by accident.

Regulators are watching how these tools are presented. In October 2026, the UK Financial Conduct Authority called for clearer disclaimers when chatbots give financial information, and its examples of poor practice included firms overstating what their AI tools can do.

For the experts at NAGA, the social trading and investing platform whose European crypto entity was authorised under the EU's MiCA regulation in June 2026, the assessment of any AI-built strategy starts somewhere most marketing never goes. Before looking at the chart, they ask how many backtests it took to find the one on display.

The question most results leave out 

The problem is not new. As early as 2014, a paper in the Notices of the American Mathematical Society by David Bailey, Jonathan Borwein, Marcos López de Prado and Qiji Jim Zhu showed that, for a model built on five years of data, looking at as few as 45 configurations of a strategy is enough to be misled. The more versions are tried, the better the best one looks, predictive value or not.

AI has made that number trivially easy to exceed. A review of AI in equity and crypto markets published in September 2026 makes selection one of its central tests and frames it as a single question: "How many features, architectures, prompts, assets, periods and hyperparameters were tried before the reported result?" Its overall verdict is sober. Across the evidence it reviewed, "no general AI architecture is shown to deliver persistent, cross-regime, capacity-aware net alpha."

For anyone used to machine learning, the mechanism is familiar. Choosing the best of many backtests is a form of training, and the backtest that wins has been fitted to that particular stretch of history.

Signals fade once they meet new data

The same review documents a second problem, predictor decay. Signals that looked strong in their original samples tend to weaken afterwards, partly because some of that strength was luck and partly because other investors start acting on them. It is already visible in AI tools: research on language-model readings of financial news finds that predictability "weakens as use of the technology spreads".

Even a genuine edge has a shelf life, and one that many people can download at once is likely to have a short one.

The three questions NAGA's experts ask

From this research, NAGA's experts draw three questions to put to any backtest, whoever or whatever produced it.

The first is how many versions were tested. If nobody can say, the safe assumption is many.

The second is whether any data was held back. A result checked only against the data used to build it cannot tell a real pattern from a coincidence that happened to fit.

AI tools add a subtler version of this problem, and researchers have now measured it. In a recent study, Zhenyu Gao, Wenxi Jiang and Yutong Yan of CUHK Business School gave a language model only Kodak's name, ticker and a date in July 2020. It put the probability of the share price rising at about 0.9999, because the event sat inside its training data. For dates after the training cut-off, the effect all but vanished. A backtest that asks a model to "decide" on past dates can quietly benefit from knowing the answer.

The third is whether the strategy has run forward, on prices that did not exist when it was built. NAGA's backtesting guide names the trap directly as optimisation bias, "also popular as curve-fitting", warns about look-ahead bias in the data, and says traders also need to forward test a strategy in the real-time market.

On NAGA, that step can be taken on a demo account with virtual funds, letting a strategy run on live prices for weeks before any real money is involved. For a retail trader, it is the closest equivalent to a held-out test set.

What a forward test cannot settle

NAGA's experts are just as clear about the limits of their own advice. A few weeks of forward testing is a small sample, and a strategy can still pass it by luck. Demo execution does not reproduce every condition of a live account, or the pressure of real money. Markets also change, which NAGA's guide itself notes: a strategy that suited one period may behave differently in the next.

Leverage raises the stakes of getting this wrong. With leveraged products, losses can mount quickly, and an edge that existed only inside a backtest is no defence once real positions are open.

In machine learning, nobody would accept a model that had only ever been scored on its training data. Applied to an AI-built trading strategy, that same standard comes down to one request before trusting any chart: show how it did on data it never saw.

Trading leveraged products involves a significant risk of loss and is not suitable for all investors.

Join our WhatsApp Channel to get the latest news, exclusives and videos on WhatsApp

                                                                                                       _____________                                             

Disclaimer: Analytics Insight does not provide financial advice or guidance on cryptocurrencies and stocks. Also note that the cryptocurrencies mentioned/listed on the website could potentially be risky, i.e. designed to induce you to invest financial resources that may be lost forever and not be recoverable once investments are made. This article is provided for informational purposes and does not constitute investment advice. You are responsible for conducting your own research (DYOR) before making any investments. Read more about the financial risks involved here.

Top 10 Ethereum-Based Tokens to Know in 2026

Arthur Hayes Targets ETH, ZEC, NEAR & VVV Before AI Bubble Bursts

Robinhood vs eToro: Crypto Trading Features, Fees and Options Compared

Cardano Price Nears USD 0.14 as Leios Upgrade Offers Hope for ADA Recovery

Wells Fargo's BlackRock Bitcoin ETF Position Doubles to USD 2.35 Billion