An end-to-end walkthrough of the six-stage validation pipeline — from vague trading idea to a statistically-vetted, portfolio-fit strategy. Each stage has a specific job, a clear deliverable, and a hard pass/fail criterion before advancing.
Six stages. One decision gate. Every strategy that reaches your portfolio has survived statistical scrutiny.
No strategy enters your portfolio without passing statistical validation first. The Decision Gate after Backtest Verification is the single most important checkpoint in this pipeline — it prevents curve-fit, fragile, or random strategies from contaminating your portfolio. Only validated, evidence-backed strategies proceed to multi-strategy analysis. Most strategies will not pass. That does not mean the software is broken. Most strategies simply do not have edge.
Specification, code generation, and backtesting.
The Prompt Builder enforces NinjaScript development discipline by generating structured, scope-locked prompts for AI coding tools. Each prompt includes workflow presets, core rules, scale profile, domain modules, and explicit deliverables.
Key ActionsA complete, structured NinjaScript development prompt ready for AI tool input.
The AI tool receives the structured prompt and generates NinjaScript (C#) code. Paste it into the NT8 NinjaScript Editor, compile, and resolve errors. The structured prompt dramatically reduces compile failures, hallucinated methods, and scope drift.
Key ActionsA compiled, error-free NinjaScript strategy loaded in NinjaTrader 8.
NinjaTrader's built-in Strategy Analyzer runs the compiled strategy against historical market data. Use a minimum of 8 years of data to capture multiple market regimes. After the backtest completes, export the trade list as a CSV file for external validation.
Key ActionsA CSV file containing all trade data (entry/exit times, profit, MAE, MFE, instrument).
Statistical validation, the decision gate, and portfolio analysis.
Backtest Verification performs rigorous statistical analysis on exported trade data to determine whether a strategy's edge is real or the result of randomness and curve-fitting. It applies permutation tests, Monte Carlo simulation, and walk-forward analysis to stress-test the strategy.
Key ActionsA comprehensive statistical validation report: pass or fail on 5 core tests.
The Decision Gate is the most critical checkpoint in the pipeline. It sits between individual strategy validation and portfolio-level analysis. A strategy failing any of the five tests is rejected before it can enter your portfolio.
Key ActionsA clear pass/fail decision. Passing strategies advance. Failing strategies iterate.
Portfolio Fit receives only validated, passing strategies. It combines them into a portfolio view and evaluates diversification by analyzing daily P&L correlation, portfolio-level drawdown, allocation weighting, and blended equity curves.
Key ActionsPortfolio analysis: correlation, drawdown, allocation, diversification, blended performance.
The exact thresholds that decide whether a strategy advances, iterates, or is discarded.
| Validation Test | Pass Threshold | What It Measures |
|---|---|---|
| Positive Expectancy | > $0 per trade | Average profitability per trade. |
| Sign-Flip Permutation P-Value | < 0.05 | Edge is not due to random win/loss order. |
| T-Test P-Value | < 0.05 | Statistical significance of mean trade P&L. |
| Monte Carlo Prob. of Ruin | < 5% | Likelihood of account blowout under stress. |
| Walk-Forward OOS Edge | Positive OOS profit | Strategy holds up on unseen data periods. |
| Portfolio Metric | Pass Threshold | What It Measures |
|---|---|---|
| Portfolio Max Drawdown | < 25% of capital | Worst peak-to-trough decline of combined equity. |
| Inter-Strategy Correlation | < 0.40 average | Strategies should not move in lockstep. |
| Portfolio Expectancy | > $0 per trade | Blended portfolio must be net profitable. |
| Allocation Balance | No single strategy > 40% | Avoid concentration risk in one system. |
| Diversification Benefit Score | > 0 | Portfolio better than best single strategy risk-adjusted. |
The Decision Gate between Backtest Verification and Portfolio Fit is the single most important architectural decision in this pipeline. A strategy that fails statistical validation has no business in your portfolio — adding it would contaminate correlation analysis, inflate the combined equity curve with unreliable returns, and mask real risk.
When a strategy fails, the specific failure tells you what to fix: low expectancy points to weak entry/exit logic; a high p-value means the edge may be random noise; high ruin probability indicates inadequate risk management; a failed walk-forward signals overfitting to in-sample data. Use the Prompt Builder's diagnostic mode to generate a targeted debugging prompt, then restart at Stage 1. Each iteration produces a more robust candidate. The goal is a portfolio of independently validated, low-correlation strategies that survive every statistical test.
One-line summary of every stage. Print this, tape it to your monitor.
| # | Name | Category | Key Action & Output |
|---|---|---|---|
| 1 | Prompt Builder | Plumbline | Define requirements → generate structured NinjaScript prompt. |
| 2 | AI + NT8 Editor | NT8 + AI Tool | Paste prompt → AI generates code → compile in NT8 editor. |
| 3 | Strategy Analyzer | NinjaTrader 8 | Backtest 8+ years → Trades tab → Export CSV. |
| 4 | Backtest Verification | Plumbline | Drop CSV → check expectancy, p-values, Monte Carlo, WFA. |
| 5 | Decision Gate | Pipeline Control | PASS → Portfolio Fit | FAIL → loop to Stage 1. |
| 6 | Portfolio Fit | Plumbline | Import validated CSVs → correlation, drawdown, allocation. |
The Plumbline Trading Suite is provided for educational and informational purposes only. It does not constitute financial advice, trading recommendations, or guarantees of profitability. Past performance is not indicative of future results. Futures and derivatives trading involves substantial risk of loss and is not suitable for all investors. All trading decisions are the sole responsibility of the user. Your data is processed 100% locally in your browser — no information is sent to any server, API, or cloud service.