Backtesting & validation

Deflated Sharpe ratio

Adjusts a reported Sharpe ratio for the number of trials, non-normality of returns, and sample length. A high in-sample Sharpe often collapses once you account for how many variants were tested.

Bailey and López de Prado treat it as a false-discovery guardrail, not a performance boast. See our backtesting pitfalls reading list.

Purged cross-validation

A time-series CV scheme that removes training observations whose labels overlap the test window in time. Standard k-fold CV leaks information when labels span multiple bars or days.

López de Prado pairs purging with an embargo gap before each test fold. Covered in our ML signals reading list and backtesting pitfalls list.

Combinatorial purged CV

Extends purged CV by averaging performance across many train/test splits rather than a single chronological partition. Reduces variance in estimated out-of-sample metrics for financial series.

Useful when walk-forward alone gives unstable Sharpe estimates. See ML signals for the primary reference and implementation notes.

Backtest overfitting

Fitting noise in historical data so a strategy looks profitable in-sample but fails live. Grows with parameter count, feature mining, and repeated re-tuning on the same dataset.

Harvey, Liu, and Zhu frame it as a multiple-testing problem across published anomalies. Start with our backtesting pitfalls reading list.

Probability of backtest overfitting (PBO)

Estimates how often the best in-sample strategy underperforms median out-of-sample among many candidates generated from the same history. High PBO means your winner was likely luck.

Bailey et al. use combinatorially symmetric splits to compute it. Linked from backtesting pitfalls.

Walk-forward analysis

Retrain a model on a rolling past window and test on the next unseen segment, repeating through history. Mimics how a live system would have been updated over time.

Still vulnerable to global hyperparameter tuning unless each fold is truly blind. Discussed alongside purged CV in backtesting pitfalls.

Look-ahead bias

Using information in a backtest that would not have been available at the simulated trade time — revised fundamentals, full-universe survivorship sets, or same-bar close prices.

Point-in-time databases exist to prevent it. See alternative data pitfalls and backtesting pitfalls.

Survivorship bias

Backtesting only on stocks or funds that still exist today, dropping delisted names and dead strategies from the sample. Inflates average returns and factor premia.

Hou, Xue, and Zhang show many anomalies shrink under proper replication. Covered in factor investing and alt data pitfalls.

Multiple testing correction

Statistical adjustments when many hypotheses are tested at once — Bonferroni, Benjamini-Hochberg FDR, or Bayesian model averaging. Without correction, some "significant" factors are false positives.

Central to the factor zoo debate and backtest mining. See factor investing and backtesting pitfalls.

Out-of-sample decay

The drop in Sharpe, alpha, or hit rate when a strategy moves from development data to fresh periods. Often 30–70% for mined signals even before costs.

Mulligan et al. quantify Sharpe decay after publication and re-optimization. Read more in backtesting pitfalls.

Net-of-cost backtest

A simulation that subtracts explicit transaction costs, spread, borrow fees, and market impact from gross returns. Gross-only backtests routinely overstate capacity and Sharpe.

Stat arb and execution lists stress net P&L as the only honest metric. See statistical arbitrage and market impact models.

Execution & market impact

Square-root law

Empirical rule that price impact scales with the square root of traded volume relative to average daily volume — not linearly. Holds across many asset classes in meta-studies.

Gatheral and Bouchaud derive conditions under which it emerges from order-flow persistence. See market impact models for Sato-Kanazawa universality evidence.

Temporary vs permanent impact

Temporary impact is the transient price move from your order that mean-reverts; permanent impact is the lasting shift in fair value from information revealed by trading. Optimal execution models split costs this way.

Almgren-Chriss and Gatheral treat the decomposition differently; both frameworks are in our market impact reading list.

Almgren-Chriss optimal execution

A framework that minimizes expected execution cost plus a risk penalty on remaining inventory, producing a trade schedule between immediate liquidation and slow TWAP-like execution.

Still the baseline for institutional schedule algorithms. Primary paper linked from market impact models.

Implementation shortfall

Total slippage versus a decision price: delay cost, spread, fees, and market impact combined. Perloff and Kennedy popularized it as the right benchmark for execution quality.

Any alpha estimate must beat implementation shortfall at intended size. Context in market impact models and statistical arbitrage.

Kyle's lambda

A measure of price impact per unit of order flow in Kyle's (1985) informed-trading model. Higher lambda means the market is less liquid or more sensitive to informed flow.

Connects microstructure theory to empirical impact coefficients. Background in market impact models.

VWAP execution

Executing in proportion to market volume so your average fill tracks the volume-weighted average price. Common benchmark for institutional orders in liquid equities.

Optimal when you have no strong short-term alpha and want minimal footprint. Compared to Almgren-Chriss schedules in market impact models.

Capacity constraint

The maximum capital a strategy can deploy before returns degrade from market impact, borrow limits, or signal crowding. Often the binding constraint for stat arb and factor tilts.

Li-Papanicolaou and stat arb papers optimize net-of-cost weights under capacity. See statistical arbitrage and market impact.

Factor investing & anomalies

Factor zoo

The explosion of hundreds of proposed return predictors — value, momentum, investment, accruals, and endless variants. Many fail replication once multiple testing and microcap biases are fixed.

Feng, Giglio, and Xiu formalize testing in a crowded factor space. Start with our factor investing reading list.

Anomaly decay

The post-publication decline in a factor's premium as arbitrageurs learn it, data vendors commoditize it, or the original sample was overfit. Momentum and some value spreads show slower decay than micro anomalies.

McLean and Pontiff document average decay; Hou-Xue-Zhang revisit replication. See factor investing.

False discovery rate (FDR)

The expected fraction of rejected null hypotheses that are actually true — i.e., fake factors. Benjamini-Hochberg controls FDR when scanning many signals at once.

Harvey and Liu argue published t-stats in finance imply rampant false discoveries. Applied to factors in factor investing.

Fama-French factors

Long-short portfolios (HML value, SMB size, RMW profitability, CMA investment, etc.) that explain cross-sectional return variation beyond market beta. The workhorse baseline in asset pricing.

Kenneth French publishes free daily and monthly factor returns. Data and extensions linked from factor investing.

Smart beta

Rules-based portfolios that tilt toward academic factors — often cap-weighted index alternatives with explicit factor exposures. Implementation details (rebalance frequency, sector neutrality) dominate realized tracking error.

Distinguish backtested factor portfolios from live ETF friction. Recent replication work in factor investing.

Cross-sectional predictability

The ability to rank stocks by expected return at a point in time using characteristics — book-to-market, momentum, ML scores. Distinct from forecasting the market index (time-series predictability).

Gu-Kelly-Xiu and Kelly-Malamud-Zhou show ML raises cross-sectional R² but with complexity costs. See return prediction and ML signals.

Turnover

How much of the portfolio is replaced per rebalance, often annualized. High turnover amplifies transaction costs and can erase paper alpha in factor and ML strategies.

Novy-Marx and Velikov stress turnover-adjusted performance in factor work. Discussed across factor investing and stat arb.

Machine learning signals

Triple-barrier labeling

Labels each trade by whichever barrier hits first: profit target, stop loss, or vertical time limit. Better matches real trading exits than fixed-horizon return labels.

Core to López de Prado's financial ML workflow. Explained in our ML signals reading list.

Meta-labeling

A secondary model that predicts whether a primary signal's bet should be taken and at what size — not the direction itself. Separates alpha discovery from bet sizing and filtering.

Reduces false positives when the primary model has decent recall but poor precision. See ML signals.

Fractional differentiation

Transforms price series to stationarity while preserving more memory than integer differencing. Lets ML models use informative lags without full non-stationarity.

López de Prado introduces a tunable differentiation order d. Covered in ML signals and return prediction.

Information coefficient (IC)

Cross-sectional correlation between predicted and realized returns at each rebalance. IC and breadth (number of independent bets) drive the fundamental law of active management.

Practitioners track IC decay out-of-sample more than in-sample R². ML signal papers in ML signals report IC alongside Sharpe.

Feature importance (MDA / MDI)

Methods to rank input variables in tree ensembles: mean decrease accuracy (out-of-bag permutations) or mean decrease impurity (in-sample splits). MDA is generally more reliable for finance features.

López de Prado warns MDI favors high-cardinality noise. See ML signals.

Stacking ensemble

Trains a meta-model on out-of-fold predictions from base learners, combining diverse models without averaging weights blindly. Can improve robustness if base models make uncorrelated errors.

Kelly et al. use neural architectures for SDF estimation; simpler stacks appear in practitioner ML pipelines. Context in return prediction and ML signals.

Embargo period

A gap after each training set and before the test fold in purged CV, absorbing serial correlation and reaction lags. Prevents label leakage when features react slowly to past returns.

Set embargo length from the autocorrelation structure of your features. Defined alongside purged CV in ML signals.

Statistical arbitrage

Statistical arbitrage

A basket of mean-reverting or relative-value bets — pairs, sectors, or factor-neutral spreads — scaled by estimated edge and hedged against broad market moves. Profitability depends on speed, costs, and crowding.

Modern work uses ML and attention over rank spaces. Our statistical arbitrage reading list covers three recent papers.

Cointegration

A long-run equilibrium relationship between non-stationary price series — their linear combination is stationary. Classic foundation for pairs trading and spread construction.

Engle-Granger and Johansen tests remain standard; ML stat arb often relaxes linear cointegration. Background in statistical arbitrage.

Pairs trading

Long one asset and short a correlated peer when their spread deviates from history, betting on convergence. Simplest stat arb template; fragile when correlation breaks structurally.

Gatev-Goetzmann-Rouwenhorst is the classic reference; newer rank-space methods in statistical arbitrage.

Mean reversion

The tendency of a price, spread, or factor z-score to return toward a long-run average after a shock. Half-life and transaction costs determine whether the edge is tradable.

Distinct from momentum (trend continuation). Both appear in stat arb and factor investing.

Half-life (mean reversion)

Expected time for a spread to revert halfway back to its mean under an Ornstein-Uhlenbeck model. Governs holding period and capital velocity in stat arb books.

Estimated via AR(1) or OU regression on the spread. Practical notes in statistical arbitrage.

Alternative data

Point-in-time data

Records that reflect what was knowable on each historical date — accounting restatements, index membership, and fundamentals as they were filed, not as revised today.

Without PIT data, backtests of value and quality factors are systematically optimistic. Chen-Zimmermann replication discusses fixes; see alternative data pitfalls.

Vendor due diligence

Evaluating an alt-data provider's collection method, coverage bias, revision policy, and legal rights before buying. Bad vendor metadata kills more signals than weak ML.

AQR and Chen-Ferson-Peters outline checklists for systematic teams. Our alternative data reading list includes a practitioner checklist.

Signal decay (alt data)

The shortening half-life of an alternative signal as more funds subscribe, APIs proliferate, or the underlying behavior adapts. Satellite and web-scrape alphas often decay fastest.

Ke-Kelly-Xiu show text signals need careful NLP pipelines to persist. Discussed in alternative data pitfalls and ML signals.

Alternative data

Non-traditional datasets — satellite imagery, credit card flows, app usage, patent filings, NLP on filings — used to predict fundamentals or sentiment before they appear in price.

Edge comes from cleaning and mapping, not raw access. Start with alternative data pitfalls for signal-vs-noise framing.