The Signal and the Noise: Why So Many Predictions Fail- But Some Don’t
by Nate Silver
Nate Silver seems to be the “It” guy for statistics. In 2009, Time Magazine listed him as one of the most 100 influential people in the world. He’s know for forecasting baseball and elections with good accuracy and consistency.
I picked up this book in the hope of better understanding statistics, particularly hoping it would strengthen my understanding what is noise and what is signal in business.
The book is not only a good journey through statistics covering such topics as global warming, baseball, the market, hurricanes, and terrorism, but also takes some side routes into psychology and sociology.
Here are my biggest takeaways:
- People love to predict things, but we are not very good at it.
- We have evolved to recognize patterns. The problem is our world has become so inundated with information, we believe we can see patterns in randomness when there isn’t any.
- Aggregate forecasting is typically more accurate than an individual forecaster—up to 20% more accurate.
- It is always easy to sort out the relevant signals from irrelevant ones after the fact. Case in point: 9/11 terror attacks.
- Math classes need to teach statistics and probability instead of geometry and calculus. This isn’t the first time a very wicked smart person said this (Deming, Neil de Grasse Tyson, Arthur Benjamin come to mind).
- Silver warns we need to move away from the spectrum that things are 0 or 100% certain. They usually aren’t. We need to take a more probabilistic approach.
- We are naturally drawn to people who make the big/bold predictions. Silver said this is because they sound persuasive.. They are usually wrong more than they are right, though. Silver calls these types of people hedgehogs. Those who are better at predicting Silver calls foxes, These people take a more complicated approach to predicting and are more probabilistic. They are more likely to be correct. You typically don’t hear from the foxes in our society.
- When the facts change, foxes will change their forecasts. This may make them appear to be weak to others. Hedgehogs typically double down.
- Americans believe we can control our fates (called determinism). This makes it hard for us to swallow the concept of probability.
- Because Americans are a deterministic people, it is difficult for forecasters, who deal in probabilities, to turn their messages into deterministic ones.
- One of the biggest things Silver talks about when making predictions is to understand Baye’s Theorem. When I read it, it was over my head, and much of it sounded subjective, but after awhile, it was starting to make more sense. I want to review and study this in more depth. It might help me.
- We can never make perfectly objective predictions, they will always be tainted by a subjective POV.
- In order to accelerate our learning process, we need to test ourselves by making predictions in the real world and see how they pan out instead of relying on a statistical model.
- Heuristic strategies (or rules of thumb) are good to use when predicting, but we need to have the wisdom to know when to discard them. He used an example of chess when Bobby Fisher sacrificed higher value pieces in order to gain strategic advantage.
- He introduced me to the concept of Complex Systems. This sounds like it needs to be investigated more. I’ve already ID’d a book about it.
- Silver said he believed skilled poker players are better than 99% of the population at making good probabilistic judgments. He said playing the game will refine these skills.
- Its fine to move away from consensus, but the further you do, the stronger your evidence must be in order for you to believe you are right and everyone else is wrong.
- He says anyone who is interested in forecasting must read Principles of Forecasting by Scott Armstrong. It should be considered canon.
- Advice from Michael Mann, a global warming advocate on dealing with naysayers and persuading a public that does not deal with uncertainty and is used to overconfident forecasters: “…be very clear about where the uncertainties are . . . but [do not] have our statements be so laden in uncertainty that no one even listens to what we’re saying.”
- Its important to not pretend that you don’t have prior beliefs. Work to reduce your biases. State your beliefs up front so people know that you have a subjective filter.
- Be willing to test your ideas. Don’t wait for a flash of insight. Progress usually comes from small incremental and sometimes accidental steps.
Admittedly, A lot of what Silver wrote went over my head (I had to skip some sections when my head started to hurt), but I came away with a much better appreciation about deciphering signal and noise– it is hard for everyone, even the experts.
The book can be bought here.