Overfitting (Curve Fitting)

Overfitting, or curve fitting, happens when a strategy is selected because it matches quirks of the development sample that do not persist. The rules then describe historical noise more closely than a repeatable market effect.

The risk grows with the number of choices. If a researcher tests 72 parameter sets, some will look better by chance even when none has a durable edge. In AlphaProve's August 2026 demonstration, the in-sample winner made 17.96% with a 0.88 Sharpe ratio, then lost 14.32% with a −1.21 Sharpe in the following period. All ten configurations selected by in-sample performance lost out of sample. That is a worked example of selection risk, not a universal estimate of how often strategies fail.

The warning signs

Warning signs include a lone winning parameter surrounded by losing neighbors, many researcher choices relative to the available observations, performance concentrated in one period, and a large gap between development and later results. None is a mechanical verdict. A real effect can be regime-dependent, and even a simple strategy can fail its next sample by chance.

A common misconception is that a high Sharpe or a fixed number of parameters proves overfitting. There is no universal cutoff. The relevant questions are how many alternatives were tried, how the winner was selected, whether failed tests were retained, and how uncertain the estimates are.

The defenses

Define the search and selection rule before looking at evaluation data. Keep neighboring parameter values, failed trials, and the size of the search visible. Use chronological holdouts and forward-walk analysis, then reserve a final untouched period when the claim requires one. Bailey and coauthors' paper on the probability of backtest overfitting explains why ordinary holdout testing can still disappoint after repeated strategy searches. Monte Carlo can test path risk, but reshuffling a biased set of trades does not remove selection bias.

On AlphaProve

AlphaProve's parameter sweeps make every tested combination explicit, while its forward-walk run type separates chronological selection and evaluation windows. Those tools help expose the search process; they do not decide that an edge is real. We overfit a strategy on purpose publishes the 72-result experiment, its selection rule, and the negative result that followed.