Variation

BOOK REVIEW: THE SIGNAL AND THE NOISE

The-Signal-and-the-Noise-Why-So-Many-Predictions-Fail-but-Some-Dont-4356DAN’S SCORE: Stars 4
The Signal and the Noise: Why So Many Predictions Fail- But Some Don’t
by Nate Silver


Nate Silver seems to be the “It” guy for statistics. In 2009, Time Magazine listed him as one of the most 100 influential people in the world. He’s know for forecasting baseball and elections with good accuracy and consistency.

I picked up this book in the hope of better understanding statistics, particularly hoping it would strengthen my understanding what is noise and what is signal in business.

The book is not only a good journey through statistics covering such topics as global warming, baseball, the market, hurricanes, and terrorism, but also takes some side routes into psychology and sociology.

Here are my biggest takeaways:

  • People love to predict things, but we are not very good at it.
  • We have evolved to recognize patterns. The problem is our world has become so inundated with information, we believe we can see patterns in randomness when there isn’t any.
  • Aggregate forecasting is typically more accurate than an individual forecaster—up to 20% more accurate.
  • It is always easy to sort out the relevant signals from irrelevant ones after the fact. Case in point: 9/11 terror attacks.
  • Math classes need to teach statistics and probability instead of geometry and calculus. This isn’t the first time a very wicked smart person said this (Deming, Neil de Grasse Tyson, Arthur Benjamin come to mind).
  • Silver warns we need to move away from the spectrum that things are 0 or 100% certain. They usually aren’t. We need to take a more probabilistic approach.
  • We are naturally drawn to people who make the big/bold predictions. Silver said this is because they sound persuasive.. They are usually wrong more than they are right, though. Silver calls these types of people hedgehogs. Those who are better at predicting Silver calls foxes, These people take a more complicated approach to predicting and are more probabilistic. They are more likely to be correct. You typically don’t hear from the foxes in our society.
  • When the facts change, foxes will change their forecasts. This may make them appear to be weak to others. Hedgehogs typically double down.
  • Americans believe we can control our fates (called determinism). This makes it hard for us to swallow the concept of probability.
  • Because Americans are a deterministic people, it is difficult for forecasters, who deal in probabilities, to turn their messages into deterministic ones.
  • One of the biggest things Silver talks about when making predictions is to understand Baye’s Theorem. When I read it, it was over my head, and much of it sounded subjective, but after awhile, it was starting to make more sense. I want to review and study this in more depth. It might help me.
  • We can never make perfectly objective predictions, they will always be tainted by a subjective POV.
  • In order to accelerate our learning process, we need to test ourselves by making predictions in the real world and see how they pan out instead of relying on a statistical model.
  • Heuristic strategies (or rules of thumb) are good to use when predicting, but we need to have the wisdom to know when to discard them. He used an example of chess when Bobby Fisher sacrificed higher value pieces in order to gain strategic advantage.
  • He introduced me to the concept of Complex Systems. This sounds like it needs to be investigated more. I’ve already ID’d a book about it.
  • Silver said he believed skilled poker players are better than 99% of the population at making good probabilistic judgments. He said playing the game will refine these skills.
  • Its fine to move away from consensus, but the further you do, the stronger your evidence must be in order for you to believe you are right and everyone else is wrong.
  • He says anyone who is interested in forecasting must read Principles of Forecasting by Scott Armstrong. It should be considered canon.
  • Advice from Michael Mann, a global warming advocate on dealing with naysayers and persuading a public that does not deal with uncertainty and is used to overconfident forecasters: “…be very clear about where the uncertainties are . . . but [do not] have our statements be so laden in uncertainty that no one even listens to what we’re saying.”
  • Its important to not pretend that you don’t have prior beliefs. Work to reduce your biases. State your beliefs up front so people know that you have a subjective filter.
  • Be willing to test your ideas. Don’t wait for a flash of insight. Progress usually comes from small incremental and sometimes accidental steps.

Admittedly, A lot of what Silver wrote went over my head (I had to skip some sections when my head started to hurt), but I came away with a much better appreciation about deciphering signal and noise– it is hard for everyone, even the experts.

The book can be bought here.

Advertisements

Personal Kanban: Bringing Focus to Chaos- Part 2

As a project manager, I’ve got a ton of things I have to worry about. The devil is often in the details and there are a LOT of details. How do I keep track of them all? How do I know which to tackle first? How do I know how much I can do in a day? Thank the project management gods for the personal Kanban board. This is my go-to tool for personal time, priority, and energy management. In my last post I described the basics of the personal Kanban board and how it works. In this post I’ll describe how I determine what I can do in a day.

20161202_161055

Notice that each of the post-its have a number in the top right corner. This is an idea I got from Scrum. These are velocity points.

In a nutshell—velocity works like this:

Each item I have to do is given a certain number of points. I use the Fibonacci sequence. For me, the numbers are a combination of time, effort, complexity, and uncertainty. They are not precise time estimates, such as “this task will take me 15 minutes.” They are relative. To give you some type of idea of time, though—a 1 point item takes about 5-10 minutes. A 2 point story is about 10-25 minutes. A 3 is about 30-45 minutes. I may assign a higher point for an item if its hard or exhausting or if I’m not certain how long it will take. Like I said—its relative, but if I keep to my rules, I find it to be pretty darn accurate.

Over time, I keep track of how many points I can accomplish in a day. This will tell me how much I can get done in a day. My average is 36.5. So, when I plan my day, I put up @ 37 points worth of work.

 

20161202_161716

38.5 points ready to go. I have one in WIP–ready to go when I walk in to start work in the morning.

IMPORTANT: 37 points is NOT my target. My target is to do as much quality work I can in a day by staying focused. Time and energy is limited. I know with all things being equal, I can produce in the 37 point range. At the end of the day, I don’t fret if I don’t hit 37 points, and I don’t celebrate if I go over. The reason? Variation.

personal

Here’s a graph of about two months of work on a control chart. You can see this is a stable system with an average of 36.5 points with an upper control limit of @ 50 and a lower limit of around 24. If I go outside those limits, I will look at what went wrong. If I see a downward trend, I will investigate; if I see an upward trend I will investigate. None of these have ever happened.

If you think about it, this is actually a pretty amazing amount of predictability when you consider how crazy work can get. THAT is what brings order to the chaos.

Tips

  • Break down big items. I won’t go beyond an 8 (that’s about an hour and a half worth of work). Anything larger needs to be broken down. Small tasks keep you focused and keeps things progressing. If I have a large item to work on, I write “Epic” in the corner and put it in the “To Do” box. When its time to work on it, I break it down into smaller items.
  • When you complete an item, re-evaluate its points. I find that sometimes I was wrong in my initial estimate. It may be higher or it might be lower. I cross out the old number and write down the new. This gives me a more accurate total when I count up total points.
  • If something unexpected comes up during the day, I create a new post-it, stick it in the “To do” box and remove equivalent points from the “To do” box. Example– My supervisor unexpectedly comes by to get some information about the project. In the end, this was the equivalent of about 2 points of work. To stay within my daily velocity, I add 2 points for my supervisor visit and remove 2 points from my “To do” box (preferably, the lowest priority should be removed).
  • I use different colored post-it notes to represent my different projects. This helps me understand where I’m putting my efforts and helps me to better plan out my days.
  • Related: This is a great way of collecting personal data. For example, I can track how often I am interrupted or how much time I spend on reporting or in e-mail. I’ve also used it to track how much of what I do is adding to customer value. This helps me understand what I’m actually doing and what I can do to improve.
  • Some items have a specific time associated with them. For example—meetings. I put the time associated with this task in the top center and place it approximately where it will fall during the day (not necessarily by its priority). For example, an 8:30 phone call will be toward the top and a 4:00 meeting will be toward the bottom.
  • Related: Some items may be high priority, but take place in another time zone. For example, I may need to schedule a high priority appointment with one of our California sites, but because they are three hours behind my time zone, I won’t be able to reach them until mid day. I place them in the center of the day’s priorities (about the time they will be arriving to work so I can hit them up first thing).
  • Create a post-it for stopping to review what you’ve accomplished, plan for the next day, think about what you can do differently, and review and prioritize what all you have coming up in the “To Do” box (they call this a retrospective and backlog grooming in the Scrum world. This is represented by my “Wrap-Up” post-it). This is VERY important for this process. Its the last item on my list of things to do, but always takes priority over anything if I hit the 3:45 pm range.
  • As the day winds down, I will re-evaluate what all I have left to do in the “Today” box. I may add or remove items depending on how much time is left in the day.
  • I mentioned Jim Benson in my last post and I’m going to mention him again. While I came up with this idea independently, Jim is the trailblazer for Personal Kanban. I look forward to reading his book to get some ideas on what I can do better. I recently discovered a podcast interview he conducted with Mark Graban over at LeanBlog. I highly recommend it.

 

My Journey to Understanding Variation- Part II

processUnderstanding variation has not been easy for me. I think many have the same issues and its why one doesn’t see many people talking about it. During my last post, I talked about my first reactions to it, but committing myself to it because Dr. Deming said it was important.

I knew I needed to learn it, but boy, I was nervous about it. All those math equations were hurting my head just thinking about it. But I wanted to figure it out.

At this time, I was having a really rough go at work. We were all struggling: exhaustion, inability to innovate, frustration, low or poor quality, slow delivery times. No one seemed to know what to do about it. There was resistance to the Agile movement. Some thought it should be done away with.

My supervisor implored us to be change agents and recommended we find hard data to help management understand what was going on and to help them understand how we can change. Learning variation seemed like a good place to start.

I found an article about Deming and Six Sigma (I highly recommend the read). The book Understanding Variation by Donald Wheeler was suggested for someone trying to learn variation.

It was a pretty easy read and I thought I had a pretty good grasp on variation after I completed it. When I finished, the time had come to start plotting some points on a control chart. But what to measure?

This was one of the most puzzling parts for me in the beginning. Measure what exactly? A team member who had six sigma experience said it would be difficult to measure anything at the company. Others told me it was possible, but even they said they weren’t exactly sure what to do (they were pretty new to Six Sigma and still learning as well).

We were using Scrum as our methodology (well, a . . . um . . . version of it anyway). I never saw hard numbers coming out of it, though. The only thing I saw was velocity. Perhaps this was the best candidate. I often bemoaned how erratic our velocity was and that it wasn’t even close to consistent. I thought perhaps a control chart would help show how unstable it really was and help us understand what to do next.

I found a free template on line for Excel and got my numbers ready.

This was our velocity for six sprints:

Sprint 1: 52
Sprint 2: 35
Sprint 3: 72
Sprint 4: 65
Sprint 5: 60
Sprint 6: 39

This is what it looked like after plotting:

behavior-chart

This graph indicates a system in statistical control with a variation between about 5 and 100. In more simple terms, it meant our team could predictably produce a velocity between 5 and 100 points and be considered stable.

To say the least, I was not happy.

“You mean its ok for our velocity to be between 5 and 100???? That’s not acceptable!! How is this supposed to help us improve? This shows me nothing!”

I quickly closed the control chart, wondering if I was wasting my time and Deming didn’t know what he was talking about. After I calmed down a little, I started thinking that velocity didn’t work with control charts or perhaps we were just doing velocity wrong. In hindsight, perhaps I should have spoken to one of our employees who had a six sigma belt and she could have explained it.

Overall, I was a little disillusioned and disappointed.

But something told me I just wasn’t understanding this just yet and I needed to be patient.

It turns out, Deming fully expected the situation like mine. He said “the transformation” was discontinuous. The individual will learn a little here, learn a little there, and start making connections. In other words, it doesn’t happen all at once. I didn’t know he said this at the time, though, and if I didn’t have faith in Deming, I probably would have abandoned the idea right there.

I can certainly understand why some would decide not to continue with understanding variation. Our western minds are geared for results NOW and we don’t want to wait around for something to make sense. Time is too short and precious for us. We’d rather move on and looking for lower hanging fruit. Deming warned of this type of short-term thinking.

The next post in this series—the light bulbs start coming on.

BOOK REVIEW- Understanding Variation: The Key to Managing Chaos

understanding-choasDAN’S SCORE: Stars 4
Understanding Variation: The Key to Managing Chaos
by Donald J. Wheeler


Why is a variation book on an Agile blog? Well, I did say this was Evolving Agile. I’ve come to understand the concepts Agile is teaching is only part of the puzzle. I believe W. Edwards Deming to be the grandfather of Agile. Understand Deming—better understand Agile. And Deming emphasized understanding variation above everything else.

I read this book at a time when several things were happening in my life that were pointing towards understanding variation. One was I was discovering Deming and this was the one concept I really struggled with. The second was my company was investing in people learning Six Sigma. I was unable to attend the training, but was certainly interested (ironically, my company was also trying Agile, but were having an awful time implementing it—I wonder how the Six Sigma experiment is going). All signs seemed to be pointing me in learning it.

I first saw this book listed in an article written by Davis Balestracci, “Deming is Dead . . . Long Live Deming.” (btw, this is one of the first online articles I read about Deming and is an EXCELLENT read. I highly recommend it. Its also where I got the idea to read Deming Dimension and Fourth Generation Management).

Balestracci recommends this book, among others, to read instead of spending a ton of money getting a Six Sigma belt. This book was recommended by others as a good starting point for beginners.

Its a good book and I learned a lot. Having a fear of math, I was leery about reading it, but Wheeler is a good writer and breaks things down in an easy-to-understand way for us who are math challenged. There’s lot of pictures and graphs. Its broken down into small segments so easily digested. Its also short—about 121 pages without the appendix. I finished it in less than two weeks (and I’m a slow reader). It teaches the concept of variation, explains the jargon, and walks one through examples and what to look for. Some of the bigger things I learned about was specifications (this is the voice of the customer) and that the actual process—represented by the control charts is the voice of the process. Its important to understand the difference between the two. It also goes over special and common cause variation which is key to understanding variation. In the end, it got my feet wet and I tried my hand at making control charts (which I will write about in a future post).

For better understanding agile– the immediate effect was it helped me better understand the concept of velocity. For example–if your team has a velocity of 50, 47, 52, 41, 37 there is no reason to panic that your team’s performance is getting worse (or worse yet-get mad at them for slacking). Its just the natural variation in your team’s system. The key will be figuring out how to reduce the variation. Simply understanding this concept helped me tremendously as a scrum master and agilest.

Ultimately, though, I couldn’t make the leap from the book’s examples (which were primarily from the manufacturing and financial sector) into my own IT world. In other words, I didn’t quite understand how it could help me with what I was doing specifically. Still, it showed me this stuff made sense after all–I just needed to now figure out how I could apply it.

Bottom line—this is a great book to start to understanding variation.You may not come away with how exactly it can help you, though, like me. I would recommend Fourth Generation Management as a follow up. Joiner goes into more detail about how to reduce the different types of variation and is more nuts and bolts.

Buy Understanding Variation here.

My Journey to Understanding Variation- Part I

reduce-variation

I know for a lot of folks, this quote doesn’t make much sense. Truth be told, it didn’t make much sense to me either until just recently. I am now just beginning to understand how profound it is.

Of the four principals of Deming’s System of Profound Knowledge, Understanding Variation has been the most difficult for me to learn, despite studying it the most. I don’t think I’m alone in this. Deming Disciples often bemoan the fact that the companies they consult for don’t use control charts. Joseph Juran lamented how the Hawthrone plant, the cradle of the quality revolution and here he and Deming learned about quality control, had stopped using control charts. I can’t help but think the reason for this is because its such a darn hard thing to understand!

I thought I would record my own journey of trying to understand variation. I’m hoping it will help me sort out my own thoughts, but also help others to understand as well.

When I first heard about how variation could help with management, I had a pretty negative reaction. These are the thoughts that went through my head:

  1. mathJust looking at the charts causes a phobia I have had since my school days. I failed algebra three times. In middle and high school, I had to keep taking basic skills math classes every year because I was so poor in the advanced classes. Math makes me feel very, very, very dumb.
  2. In my experience, only basic mathematics is ever needed in real life. Fancy equations and charts are for engineers trying to design the space shuttle.
  3. People are not sets of data. What do you think we are, robots pumping out numbers? This stuff dehumanizes us. Dr. Deming–you are a mathematical egghead who doesn’t get people.
  4. This stuff is going to take a long time to learn and time is not something I have a lot of. I need results now. I’ll go look for something else.
  5. It doesn’t seem obvious to me how plots on a graph is in anyway going to help me do my job better. At first gloss, this looks like a waste of my time.
  6. From what I have heard and read so far, it doesn’t make much sense. There is a lot of jargon I don’t understand. Its confusing.
  7. It seems like a lot of work for nothing.

Am I alone in these thoughts and feelings?

So, why did I continue?

control-chart-subgroup

  1. I trust Deming. He has a track record of great success. His other advice seems to be spot on. Why not take him up on what he believes is the most important element of all?
  2. I know from past experiences that at first things may not make sense to me, but after awhile it begins to click and I have a lot of ‘oh-my-god’ moments.
  3. I’ve heard from others that at first this stuff didn’t make sense to them either. Perhaps the same thing will happen to me.

During my next post, I will talk about my first experience with control charts. It didn’t go too well.