Link This | Email this | Blog This | Comments (1)

Fun With Statistics, SPC Edition
January 9, 2008

To borrow a theme from Kevin Meyer, here's some "fun with statistics," or at least this kind of thing is fun to me, being a big proponent of Statistical Process Control (SPC) and the methods taught by Deming and Wheeler (an incredible reference is Wheeler's book Understanding Variation).

I forget the exact context, but a I heard a news story that talked about some data (I forget if it was employment related, housing starts, etc) and the key point made was "it was the 24th consecutive month where the number was lower than the previous year."

OK, kind of convoluted, but the clear implication was that there was an indicator of a downward trend, or that was the conclusion we were supposed to draw.

A different comparison, 24 consecutive declining months, compared month to month instead of to a year prior, certainly would be an indicator of a system that is not in statistical control, as pictured below. The line in the middle is the "mean" and the red lines are calculated "control limits" -- clearly not a process that is "in control." This is a trend we can draw a clear conclusion from. We can probably predict the next month will be lower (in fact, you can draw a control chart with a "slope" that takes into account a consistent increase or decrease).


But, the case mentioned on the radio might very well be a stable system. Again, in this case, I made up data, but trust me, each point 13-24 is lower (even if just slightly) than the data point 12 before.

In this case, we do have a process that is in statistical control, as shown below. It is quite possible that the "24 consecutive months where the number was lower than the previous year" is what you might call "statistical trivia" -- interesting, maybe, but meaningless. What this SPC chart tells us is that the next month is most likely to be in that range from 23 to 78, unless something changes (a "special cause").


There are far too many cases when the media (or business leaders) use (or abuse) year-on-year comparisons. Wheeler makes a compelling case in his book about why trend charts and SPC are much better than comparing two data points. We are less likely to make bad decisions based on trends that aren't there.

Has anyone else out there used SPC for evaluating business metrics? The great thing about using SPC is that you can avoid overreacting to every up and down in your hourly, daily, or weekly numbers. That overreaction creates a lot of "muda" (waste) and really wears everybody down.

Posted by Mark Graban on January 9, 2008 | Comments (1)