Abrazolica

What's going to happen tomorrow? The future is sending us signals about what is going to happen. The only problem is the information is being transmitted over an extremely noisy channel in a very intricately encoded form. So how do you pick up these signals and how do you decode the information contained in them. The signals are actually all around you in the form of evidence or data that you can collect right now. The big problem is that there is just too much of it, so you have to be selective. When you're selective then the process of decoding the signal becomes more error prone. Maybe with enough data you could decode the signal with no error, but probably not.

Even the laws of physics have moved beyond the idea that the future is perfectly predictable. Only classical mechanics has perfect predictability. The future of a particle in classical mechanics is completely encoded by a function called a Hamiltonian whose inputs are values that you can measure right now. But nature is fundamentally quantum mechanical. The future of a particle in quantum mechanics is also determined by something called a Hamiltonian but in this case it's an operator and not a function. The operator typically has many solutions called states and the particle's future, unless it is disturbed or measured, exists as a superposition of these states. In the future, when you look at the particle in some way, it will magically collapse into one of these states. It is tempting to say that the particle must have been in that state all along and if only you had some missing piece of information you could have predicted it. The problem is that experiments have shown that the particle really is in a superposition of states until you measure it.

So in physics there is no such thing as perfect predictability. A clockwork universe does not exist. Does this apply to what you're trying to predict? Maybe or maybe not. When we launch a spacecraft to the outer planets, its path can be predicted very accurately. Most prediction methods are based on the premise that the future is encoded by some function of things we can measure right now. Even something like a neural network is based on the idea that there is a function that determines the future. You can even define the equivalent of a Hamiltonian for some types of neural networks (see Hopfield Network for example).

It reminds me of Borges' The Library of Babel where he describes a fictional library that contains every possible book that could ever be written. This means that some book in the library contains a perfect description of what will happen tomorrow. The problem is the number of books is almost infinite and the vast majority are complete nonsense so finding the right book is virtually impossible. Some prediction methods implicitly assume there is a functional equivalent of The Library of Babel. This is a library containing every possible function. If you could just find the right function then the future would be perfectly predictable. The approach works surprisingly well for a lot of things.

But you should never be satisfied with a prediction method that says "this is what's going to happen". You want a prediction system that says "this, that, or the other" is going to happen and with these probabilities. In other words you should assume that the future is in a superposition of states. Some of those states are more probable than others but all are possible.