Feet tap. Hands clap. Heads shake. Music affects us. It moves us. By contrast, noise annoys us. Car horns beeping. Trains braking. Rain falling. It’s all sound. Yet it is obvious to discern the difference between music and noise. Music has intent, it has rhythm, it has pitch. Noise is an unwanted byproduct of our environment. Of course, we might have the opinion that certain music is little better than noise.
Markets are full of noise. In practice, in financial markets the noise overwhelms the signal, the music of the markets, which traders are seeking to hear. It’s all too easy to overfit a model that is simply taking an imprint of the noise to calibrate it, rather than the signal. The result is a model which works well on in-sample data, essentially, the training set you’ve used to fit it. However, it performs poorly on out-of-sample data.
In a sense, it’s like trying to compose music, without using a scale, and instead combining random frequencies, in the hope of producing a masterpiece. Yes, it may well end up sounding like music, but more often than not, such a composition will seem atonal. If we go to the other extreme and underfit our market model, it’s the equivalent of composing a song with a single monotonous tone.
It’s essentially a matter bias-variance trade off, which is pretty much part of any introductory machine learning course. When you overfit a model, typically you are using a model which is far too complex. As it result it more easily manages to fit to random noise, which is precisely the thing we do not wish to model. Such a model is very sensitive to changes in the training set. These type of models exhibit high variance in their estimates. Non-linear models tend to have more variance than linear models. If we have a model which is too simple to capture the properties of our dataset. This has a high amount of bias. Linear models tend to exhibit more bias than non-linear models.
We can see that ultimately, these objectives seem to conflict, decreasing variance, increases bias and vice-versa. As the name, bias-variance trade off, suggests, it’s about finding a compromise between bias and variance, when we create a model for a dataset. We want a model which is complex enough so it doesn’t exhibit high bias. At the same time, we want a model which is simple enough, so it doesn’t end up having high variance. The analogy is that we want to compose a melody which isn’t so complicated that it sounds like noise, yet not so simple, that it seems boring.
With financial markets, we have numerical quantities which allow us to compare how good one trading model is from another (the one which makes the most money out-of-sample). Well, with music, we’ll all have a different model for deciding which music is best!