Teaching myself quant

I saved a thread on X a while ago. One of those quant self-study roadmaps that float past every few months: here is the math you actually need, in order, no shortcuts. I had bookmarked maybe five of them over the years and read none of them properly. This one I copied into a repo, restructured into folders, and started actually doing.

So this is not my curriculum. I want to be clear about that up front, because there is a version of this post where I pretend I designed a five-level path from probability to stochastic calculus and you should be impressed. I didn't. Someone on X laid out the path, I adapted it, and I am a few weeks in. What follows is what it feels like from the bottom of the ladder.

The shape of it is simple. Five levels, and the claim is you cannot skip any of them:

1. Probability. Conditional thinking, Bayes, expected value, variance.
2. Statistics. Hypothesis testing, p-values, the multiple-comparisons trap.
3. Linear algebra. Covariance matrices, eigendecomposition, PCA, Markowitz.
4. Calculus and optimization. Gradient descent, convex optimization.
5. Stochastic calculus. Brownian motion, Ito, Black-Scholes.

I am in level two. I have not touched four or five except to read the table of contents and feel slightly nauseous. If you came here to learn stochastic calculus from someone who knows it, wrong post.

The first level was friendlier than I expected. Probability is the one piece of this I had real exposure to before, so most of it was rehearsal. Conditional probability, P(A | B), the thing where a stock goes up 60% of all days but 75% of high-volume days, and the 75% is the interesting number because it is conditioned on something you can observe. Bayes as a belief updater rather than a formula to memorize. Expected value as your conviction, variance as your risk, and the quiet point underneath both: a bet that wins a dollar 51% of the time and a bet that wins a hundred dollars 51% of the time have the same edge but completely different odds of bankrupting you before the edge shows up. I knew all of that in a loose way. Writing the simulations made it less loose.

Then level two punched me.

The chapter opens with a sentence I have not been able to shake. Most of what looks like edge is actually noise. And the way it demonstrates this is so cheap and so devastating that I want to walk through it, because it changed how I read every backtest result I have ever seen on the internet.

Here is the setup. You build a trading strategy. You backtest it. It returns 15% a year. Good, right? You found something. The honest question is: could a strategy with zero real edge have produced a result this good by luck? That is a hypothesis test. Null hypothesis, the strategy has no expected return. The p-value is the probability of seeing results this good if the null is true. p below 0.05, you reject the null, you declare victory.

The trap is what happens when you do this more than once. If you test a thousand random strategies, strategies that are literally pure noise, about fifty of them clear p < 0.05 by chance. That is what 0.05 means. One in twenty.

The notebook makes you watch it happen. Roughly this, simplified from what is actually in the lesson:

# illustrative: test 100 strategies that are all pure noise
import numpy as np
from scipy import stats

p_values = []
for _ in range(100):
    returns = np.random.normal(0, 0.02, 252)   # zero mean by construction
    _, p = stats.ttest_1samp(returns, 0)
    p_values.append(p)

p_values = np.array(p_values)
print((p_values < 0.05).sum(), "of 100 'strategies' look significant")

Every strategy in that loop is noise. There is no signal anywhere. The mean is zero because I set it to zero. And about five of them come back looking significant, with a straight face, with a p-value you would have published.

The fix the chapter teaches is Bonferroni, which is almost insultingly simple: if you ran a hundred tests, divide your threshold by a hundred. Demand p < 0.0005 instead of p < 0.05. Run that on the same array and the false positives mostly vanish. There are smarter corrections, but the point is not the specific arithmetic. The point is that the number of things you tried is part of the evidence, and if you do not write that number down, you are lying to yourself for free.

What got me is that this is not an exotic failure mode. This is the default behavior of anyone enthusiastic with a backtester. You try ideas until one works. That is the whole problem. The trying is the multiple comparison, and nobody counts it. I have absolutely done a version of this in regular software, staring at a dashboard until I find the cut of the data that confirms what I already believed. I just never had the word for why it was wrong.

The rest of level two builds on the same suspicion. Regression to ask whether your returns are just the market in a costume, decomposing a strategy into beta times the index plus whatever is left. Maximum likelihood as the real meaning of the word "calibrate," which everyone says and few define. A normality test on actual returns that comes back screaming NO, because returns have fat tails and the tidy bell curve everyone draws is a convenient fiction. The chapter pulls real data for this, stock returns through yfinance, Fama-French factors from Kenneth French's site, so you are not testing your noise detector against more of your own synthetic noise.

I keep the synthetic stuff close though, because the synthetic experiments are where the lesson is cleanest. When you generate the data yourself you know the ground truth. You know there is no edge in that loop. So when the test tells you there is, you cannot argue with it or rationalize it. The math caught you, and you watched it catch you.

There is a line in the roadmap material about estimation error being the real enemy, that perfect math with imperfect parameters still fails. I do not have the grounding yet to feel the full weight of that. Ask me at level three when I am inverting a covariance matrix estimated from not enough data and watching it produce a portfolio that bets the farm on whichever asset got the luckiest sample. I can see the shape of where it is going. I cannot yet feel it in my hands.

That is roughly the honest status. One level that was review, one level that rearranged how I look at any claim of edge, and three levels I am not going to pretend I have opened. I do not know if I will finish. The roadmap says it takes the better part of two years to go from nothing to something, and I believe that part more than any other claim in it.

What I did not expect was that the most useful idea so far has nothing to do with finance. Count your comparisons. Most of what looks like a signal is the residue of how many times you looked.