A free, browser-based toolkit that catches overfit backtests before they cost you real money — with Monte Carlo, walk-forward, and permutation testing. Your data never leaves your machine.
Each stage solves a specific problem in going from trading idea to deployed algo. Together they form the pipeline: spec → validate → portfolio-fit → prop firm.
Turn your trading idea into a written, testable specification.
Prove the results aren't overfit before you risk real money.
See how your new strategy interacts with your existing ones. Correlation, drawdown stacking, and allocation review across multiple systems.
Simulate whether your strategy would pass a Topstep, Apex, or Lucid evaluation. Monte Carlo runs your backtest against real prop firm rule sets across every account SKU.
A 6-step pipeline from strategy idea to validated portfolio, with pass/fail gates at every stage. The full guide walks through each stage, the exact thresholds, and the decision logic.
Most retail backtests are overfit. A strategy that looks profitable across years of historical data often performs that way for one reason: it was tuned, by hand or by optimizer, to match the noise in the sample. When deployed forward, the edge disappears.
The statistical methods used by institutional quants to detect this — permutation testing, Monte Carlo resampling, walk-forward validation — are well-documented in the academic literature but rarely applied in retail workflows. The math is straightforward. The tooling was missing.
Plumbline Trading Suite is an attempt to close that gap. It is not a trading platform. It is not a signal service. It is a set of small, focused tools that answer one question: given the trades I observed, how confident should I be that the underlying strategy has a real edge?
Plumbline Stage II: Backtest Verification applies three independent statistical tests. Passing one is not enough. A strategy that fails any of them is flagged for further investigation. The full mathematical treatment — formulas, worked examples, and edge cases — lives on the methodology page.
Randomly flips the sign of each trade's P&L thousands of times to estimate the distribution of outcomes under a null hypothesis of no edge. If the observed result is not significantly above the null distribution, the apparent edge may be noise.
Resamples the trade sequence with replacement to estimate the range of plausible equity curves and worst-case drawdowns. Reveals sequence risk and stress-tests the strategy against orderings that did not appear in the original backtest.
Splits the data into rolling in-sample and out-of-sample windows. A strategy that performs well in-sample but poorly out-of-sample is almost always overfit. The efficiency ratio between the two windows is the headline metric.
Stage II does not stop at headline performance. These full three-page reports show the same validation pipeline applied to two strategies: one that survives every gate and one that posts positive expectancy but fails statistical significance, ruin-risk, and out-of-sample testing.
Positive expectancy is backed by significant permutation and t-test results, zero modeled ruin probability, and profitable out-of-sample windows.
Positive expectancy alone is not enough. The result fails both significance tests, exceeds the ruin-risk gate, and does not retain its edge out of sample.
Example reports are provided to demonstrate the output format and validation logic. A passing historical test does not guarantee future profitability.
Clarity matters more than marketing. These tools are useful for one specific thing. They are not a substitute for the other parts of a trading practice.
Honest accounting of the current state. The shipped tools are usable today. Everything else is work in progress, no guarantees on timing.