Wednesday, October 14, 2009
Six Sigma Metrics...Why?
I have a client who is in a tizzy over whether they are recording short-term or long-term process sigmas. They are not high-speed, high-volume manufacturers, so I told them just use DPMO; don't worry about process sigma. It's not an intuitive metric anyway, nor is it anything like an accurate estimate of what to expect. And once you start putting "long-term" and "short-term" stuff in, you end up with really stupid non-intuitive discussions. I have yet to see an explanation of "short-term sigma using long-term data" that helps anyone understand anything. I'm not quite in the school of the purists who believe that, if you have a process in control, you don't have any undetected shifts. I can show you any number of real data sets to prove that...even using all four of the Western Electric Zone tests. In the interests of trying to maintain some standard metric with the rest of the Six Sigma world, I have had all my clients who were interested in using process sigmas use the standard Motorola tables or the calculator at isixsigma.com. This calculator takes your number of defects and opportunities and looks up a Sigma, applying the 1.5 Sigma shift. The assumptions for the calculator say that it assumes long-term data but provides a short-term sigma. Why should we care, and how does this terminology help anyone or make sense? Long-term data are supposed to be data that come from a process that has run long enough for some shifts to have taken place. If we're going to talk long-term, what we should use for an operational definition is "data that display a significantly different overall standard deviation than that of its local dispersion statistics." What all this is intended to provide is an estimate of what we can expect from a process in the future, given a stable process. Isn't the idea derived from a capability study? Essentially a cpk of 2 (process mean six sigma units from the nearest specification limit) equated to "Six Sigma Quality." If you buy the Motorola 1.5 sigma shift, then you gig yourself a sigma and a half, so it's really 4.5 sigma; instead of 2 ppb defective, you get 3.4 ppm. Now, there are a lot of people who object to predicting parts per million non-conforming from a capability study; I'm not one of those, as long as everyone involved realizes that we can't take any of those predictions too literally or assign too much precision. If I were claiming a process sigma of 6, and a count of defectives in the next million opportunities turned out to be 5 (or even 10 or 15), I wouldn't re-assess my sigma.Another thing I've seen lately is using a transformation to transform perfectly good data to poisson data, then deriving a DPMO and sigma from that. Now there's a great example of what Don Wheeler would call "a victory of computation over common sense." If I have 100 defective parts in a run of 1000, I have 10 percent defective. Assuming that 10% is stable over time, That equates very simply to a DPMO of 100,000. This is also assuming one opportunity per unit. It does make sense to me to do the opposite...if I'm getting more than one defect per unit (so I probably have Poisson data), it makes sense to me to transform the data using e^-dpu; that gives me an approximation to the binomial that lets me estimate DPMO.My biggest question, though, is why--other than to comply with a very short-lived tradition--should we use process sigma at all? It certainly doesn't provide any more information than DPMO, and we always have to translate it into DPMO anyway. If we must continue to process sigma, can we please just can all the short-term/long term stuff and (in a nod to standardization, even if controversial), just assume that shift happens and calculate DPMO from a stable process and use the sigma table? Maybe I'm wrong, and there is some real great reason to continue doing this, but I need an explanation and justification that makes sense. It seems to me that a lot of this is arbitrary and unnecessary.
Wednesday, September 2, 2009
During a recent discussion in the LinkedIn Deming HR group, one of the discussants posted the following link:
http://www.ted.com/talks/view/id/618
I can't recommend it too highly. It's a talk by Daniel Pink, about the science associated with rewards for performance. Anyone who was a follower of Deming, and anyone who has read Alfie Kohn, is already familiar with the concept that reward for performance can be harmful. Daniel's discussion, especially his piece about the candle problem, is an eye-opener. I would like to try that experiment at a conference or with a large class sometime.
My question or concern is the same as Daniel's: why do we continue to do, in business, what the science says is exactly the wrong thing to do? In its most public fashion, we do it on a grand scale with CEO compensation.
Daniel does a good job of pointing out that pay for performance, when it's linked to any job that requires thinking or problem solving, does more harm than good. What his talk doesn't cover, though, is the systems thinking aspect of this topic; the fact that you can't measure the performance of anyone in isolation. It's often the system that creates most of the performance we attribute to individuals.
This is certainly a worry in education these days, with many government officials pushing for performance pay for teachers. Stuck in the old carrot and stick paradigm, with nothing to go on for metrics but aggregate standardized test scores, these schemes will go a long way toward further suboptimizing our education system. Instead, put the money you might use for rewards into building systems like those built by Geoffrey Canada in the Harlem Children's Zone. Canada showed that by taking a systems approach you can improve the performance of even the most underpriveleged student populations, and put those students on an even playing field with the most priveleged.
http://www.ted.com/talks/view/id/618
I can't recommend it too highly. It's a talk by Daniel Pink, about the science associated with rewards for performance. Anyone who was a follower of Deming, and anyone who has read Alfie Kohn, is already familiar with the concept that reward for performance can be harmful. Daniel's discussion, especially his piece about the candle problem, is an eye-opener. I would like to try that experiment at a conference or with a large class sometime.
My question or concern is the same as Daniel's: why do we continue to do, in business, what the science says is exactly the wrong thing to do? In its most public fashion, we do it on a grand scale with CEO compensation.
Daniel does a good job of pointing out that pay for performance, when it's linked to any job that requires thinking or problem solving, does more harm than good. What his talk doesn't cover, though, is the systems thinking aspect of this topic; the fact that you can't measure the performance of anyone in isolation. It's often the system that creates most of the performance we attribute to individuals.
This is certainly a worry in education these days, with many government officials pushing for performance pay for teachers. Stuck in the old carrot and stick paradigm, with nothing to go on for metrics but aggregate standardized test scores, these schemes will go a long way toward further suboptimizing our education system. Instead, put the money you might use for rewards into building systems like those built by Geoffrey Canada in the Harlem Children's Zone. Canada showed that by taking a systems approach you can improve the performance of even the most underpriveleged student populations, and put those students on an even playing field with the most priveleged.
Monday, August 10, 2009
What Happened to the Deming Philosophy?
This is taken from part of a discussion on LinkedIn. Rafael Aguayo, a Consultant in Quality, Management and Strategy and Instructor at Stony Brook University, discussed this history and posed these questions today. See my response below.
Rafael's Post:
Two people in the same situation can have very different experiences. So let's consider some more objective measures of what has occurred. In the late 1980s and early 1990s Quality in the US was largely associated with Deming. Media articles regularly referred to Deming as the preeminent quality expert. Success at Ford, Harley Davidson and many other companies, that had adopted some or much of Deming's ideas, created interest and excitement in quality and other people and movements tried to position themselves as the next big thing. Specifically I would mention Reengineering and Six Sigma. At the time there were many successes accomplished under the banner of TQM and Six Sigma was a minor influence.
My assessment is that Deming-influenced quality represented 50% of the market. Yes, that is subjective, but I think anyone who was active at the time would have said that his influence and recognition was profound. Given the successes,the publicity and seeming strength of the quality movement I would have expected that by 2009 every hospital and US corporation to have been talking red beads and funnels in addition to statistical tools.
Instead, when I joined this group there were 171 members, while the Lean Six Sigma group had about 30,000. That translates to a market share of .57%. Even allowing an order of magnitude error this represents but 5.7% of the market. I have to ask what happened?
The logic of marketing is very different from formal logic. When GM discarded the Oldsmobile brand they expected those buyers to buy other GM cars. Instead they went elsewhere. If a significant amount of success were achieved under the banner of TQM and the brand is then disavowed then those successes are disavowed. At the time I observed the disappearance of some successful and respected consulting firms, such as Joiner Associates. And the interest in Deming shrank precipitously. It is possible that this outcome was inevitable. Once Jack Welch endorsed Six Sigma it may have been inevitable. Also the fiasco of Reengineering that some organizations, including ASQ, implicitly endorsed could not have helped. But whatever the causes the current reality is disappointing. While you say that has not been “your experience” your actions say something very different. By obtaining black belt certification and selling your services as such you implicitly acknowledged that Deming or SoPK is not a viable marketing brand.
It is not just that the market for a more profound understanding of quality has shrunk. Today young people who appreciate the importance of quality and process need not even once see a demonstration of the Red Beads or the Funnel. They can go out and do their best, blissfully unaware that their actions are tampering, on a massive scale. Luckily there are still many people laboring in schools and in firms with a deeper appreciation of the fundamentals.
Maybe I am hallucinating, but the reality of today is so far from what I would have expected that I must ask the question what happened? And what can be done to turn the situation around?
My response:
I'd like to add a comment to at least offer my observations in answer to Rafael's question: "What happened?"
It is, of course, not an easy thing to determine. Part of it was some admitted hubris on the part of those of us who were, or aspired to be, "Deming Disciples." One of the things we admired about Deming was his unyielding and unflinching ability to speak truth to power. He was often seen as curmudgeonly in his approach, but he never let anyone doubt that he didn't suffer fools gladly, and he was unabashed about putting anyone--including CEOs--into that category, if they offered any evidence that they belonged there. He was also very compassionate and thoughtful, and freely offered help and advice to anyone interested in learning. He just didn't have much patience for those who thought they had nothing left to learn. So he was a bitter pill for many CEOs to swallow. They did it, when they thought he could help; and, as Raphael pointed out, for a while Deming was the one person that almost everyone relied on for help.
Once he was gone, and the crisis of the 80's was over, Jack Welch and others were selling Six Sigma--not as a Quality initiative, but as a cost-cutting one--I think many of them jumped at Six Sigma because it seemed simpler, more prescriptive, more programmatic, less lofty and philosophical...maybe instant pudding. They certainly didn't have use for those Deming practitioners who (without Deming's extensive background or credibility) tried to act as Deming had. I have had Quality executives from major corporations tell me that "Deming was just a philosophy," implying that it was pie-in-the-sky, without any practical use for business. It's hard to get these people to listen to you after you explain how ignorant a statement that is...
Another thing that happened is that Six Sigma provides a roadmap that actually does work, when used well. Many companies had a lot of success with their Six Sigma projects. GE had some highly vaunted and publicized success...I will never know how much of it was real, because between making it mandatory and "firing the bottom 10%," who knows which GE numbers can be trusted? In any case, these projects can be very effective, when used as one component of an overall Quality Management System.
I think Rafael's insight about marketing is a good one. Many Deming practitioners were blindsided by Six Sigma, saw its statistical and other flaws, and concluded that it was the enemy, not worthy of consideration. We did get out-marketed, because we had no champion like Welch or Bossidy or Galvin touting huge success stories; most of the stories in Quality Progress and Quality Digest were about Six Sigma. Virtually all the mainstream business literature abandoned Quality; the only mention of it was the occasional Six Sigma story. Then Lean reared its head, and perversely became a competitor to Six Sigma.
When I joined Process Management International, they were working to develop a Deming-based Six Sigma methodology. We had people with a strong Deming foundation who had worked for Motorola and GE, and I think we were successful, with a sound methodology, presented as one set of tools in an overall tranformational approach, that took into consideration all the aspects of SoPK and the 14 points. At least we were able to continue to tell people about Deming, the SoPK and the 14 points, to show the Red Bead and the Funnel. Interestingly, during a conference that included a lot of the Deming and JUSE elite, a consensus position was developed that saw Six Sigma as a [marketing] "vehicle" for quality...a way to explain it and to act as a lever for change, a foot in the door.
Would I have been happier teaching and consuling in "pure" Deming? That's all I wanted to do when I first retired from the Navy. No one was hiring for that, though, because the jobs for consultants who did that were few and far between. In any case, Would I still do it? You bet...I do, as much as I can.
What are your thoughts?
Rafael's Post:
Two people in the same situation can have very different experiences. So let's consider some more objective measures of what has occurred. In the late 1980s and early 1990s Quality in the US was largely associated with Deming. Media articles regularly referred to Deming as the preeminent quality expert. Success at Ford, Harley Davidson and many other companies, that had adopted some or much of Deming's ideas, created interest and excitement in quality and other people and movements tried to position themselves as the next big thing. Specifically I would mention Reengineering and Six Sigma. At the time there were many successes accomplished under the banner of TQM and Six Sigma was a minor influence.
My assessment is that Deming-influenced quality represented 50% of the market. Yes, that is subjective, but I think anyone who was active at the time would have said that his influence and recognition was profound. Given the successes,the publicity and seeming strength of the quality movement I would have expected that by 2009 every hospital and US corporation to have been talking red beads and funnels in addition to statistical tools.
Instead, when I joined this group there were 171 members, while the Lean Six Sigma group had about 30,000. That translates to a market share of .57%. Even allowing an order of magnitude error this represents but 5.7% of the market. I have to ask what happened?
The logic of marketing is very different from formal logic. When GM discarded the Oldsmobile brand they expected those buyers to buy other GM cars. Instead they went elsewhere. If a significant amount of success were achieved under the banner of TQM and the brand is then disavowed then those successes are disavowed. At the time I observed the disappearance of some successful and respected consulting firms, such as Joiner Associates. And the interest in Deming shrank precipitously. It is possible that this outcome was inevitable. Once Jack Welch endorsed Six Sigma it may have been inevitable. Also the fiasco of Reengineering that some organizations, including ASQ, implicitly endorsed could not have helped. But whatever the causes the current reality is disappointing. While you say that has not been “your experience” your actions say something very different. By obtaining black belt certification and selling your services as such you implicitly acknowledged that Deming or SoPK is not a viable marketing brand.
It is not just that the market for a more profound understanding of quality has shrunk. Today young people who appreciate the importance of quality and process need not even once see a demonstration of the Red Beads or the Funnel. They can go out and do their best, blissfully unaware that their actions are tampering, on a massive scale. Luckily there are still many people laboring in schools and in firms with a deeper appreciation of the fundamentals.
Maybe I am hallucinating, but the reality of today is so far from what I would have expected that I must ask the question what happened? And what can be done to turn the situation around?
My response:
I'd like to add a comment to at least offer my observations in answer to Rafael's question: "What happened?"
It is, of course, not an easy thing to determine. Part of it was some admitted hubris on the part of those of us who were, or aspired to be, "Deming Disciples." One of the things we admired about Deming was his unyielding and unflinching ability to speak truth to power. He was often seen as curmudgeonly in his approach, but he never let anyone doubt that he didn't suffer fools gladly, and he was unabashed about putting anyone--including CEOs--into that category, if they offered any evidence that they belonged there. He was also very compassionate and thoughtful, and freely offered help and advice to anyone interested in learning. He just didn't have much patience for those who thought they had nothing left to learn. So he was a bitter pill for many CEOs to swallow. They did it, when they thought he could help; and, as Raphael pointed out, for a while Deming was the one person that almost everyone relied on for help.
Once he was gone, and the crisis of the 80's was over, Jack Welch and others were selling Six Sigma--not as a Quality initiative, but as a cost-cutting one--I think many of them jumped at Six Sigma because it seemed simpler, more prescriptive, more programmatic, less lofty and philosophical...maybe instant pudding. They certainly didn't have use for those Deming practitioners who (without Deming's extensive background or credibility) tried to act as Deming had. I have had Quality executives from major corporations tell me that "Deming was just a philosophy," implying that it was pie-in-the-sky, without any practical use for business. It's hard to get these people to listen to you after you explain how ignorant a statement that is...
Another thing that happened is that Six Sigma provides a roadmap that actually does work, when used well. Many companies had a lot of success with their Six Sigma projects. GE had some highly vaunted and publicized success...I will never know how much of it was real, because between making it mandatory and "firing the bottom 10%," who knows which GE numbers can be trusted? In any case, these projects can be very effective, when used as one component of an overall Quality Management System.
I think Rafael's insight about marketing is a good one. Many Deming practitioners were blindsided by Six Sigma, saw its statistical and other flaws, and concluded that it was the enemy, not worthy of consideration. We did get out-marketed, because we had no champion like Welch or Bossidy or Galvin touting huge success stories; most of the stories in Quality Progress and Quality Digest were about Six Sigma. Virtually all the mainstream business literature abandoned Quality; the only mention of it was the occasional Six Sigma story. Then Lean reared its head, and perversely became a competitor to Six Sigma.
When I joined Process Management International, they were working to develop a Deming-based Six Sigma methodology. We had people with a strong Deming foundation who had worked for Motorola and GE, and I think we were successful, with a sound methodology, presented as one set of tools in an overall tranformational approach, that took into consideration all the aspects of SoPK and the 14 points. At least we were able to continue to tell people about Deming, the SoPK and the 14 points, to show the Red Bead and the Funnel. Interestingly, during a conference that included a lot of the Deming and JUSE elite, a consensus position was developed that saw Six Sigma as a [marketing] "vehicle" for quality...a way to explain it and to act as a lever for change, a foot in the door.
Would I have been happier teaching and consuling in "pure" Deming? That's all I wanted to do when I first retired from the Navy. No one was hiring for that, though, because the jobs for consultants who did that were few and far between. In any case, Would I still do it? You bet...I do, as much as I can.
What are your thoughts?
Thursday, August 6, 2009
Back Again
Sorry I've been out of this loop for so long...my wife is fighting a life-threatening cancer, and dealing with that just sucks every ounce of spare energy and time out of your life.
I'm going to jump back in by recycling my latest answer in the Deming discussion group on LinkedIn. The question was about Six Sigma and Deming. Two of my favorite Deming Disciples, John Dowd and John Constantine, feel that Six Sigma is fundamentally flawed and has very little place in any discussion of Deming. This is what I wrote:
I generally agree with John Dowd and John Constantine on most things, and I agree that Six Sigma-- as taught by many of the companies consulting in it these days--has some serious problems. Some of these approaches are, indeed, incompatible with the Deming philosophy. And some of the consultants using those incompatible approaches are large enough that the argument for "as generally taught" is probably sound. My suggestion, though, is that it doesn't have to be that way. Six Sigma is just a marketing vehicle; there is no standard for it (although ASQ would like to think that their SSBOK is one). As with TQM and every other quality approach there are people who do it well, and people who don't. Unfortunately, those who don't could care less about transformation, because many of them know very little about variation, and next to nothing about systems theory, psychology, or theory of knowledge. My disagreement is in what we do about this. I have been doing what I could, through my consulting practice, any conference appearances or workshops that I am able to do, and any writing that I can get published, to bring Deming principles into Six Sigma, and to use an approach in Six Sigma that is consistent with Deming. Clients who work with me only calculate the "1.5-sigma shift" and the "process sigma" as a curiosity and a metric for communicating with those less enlightened. They see and discuss the Red Bead, the Funnel, systems theory, SoPK, and the 14 points. They learn to see and use Six Sigma projects as one tactic in an overall process management system. I think if more Deming practitioners could find it in their hearts to do something like this, we'd have more success in using Six Sigma as one tactic in our overall aim, and reach more managers and executives (and potential managers and executives) with our message. This is what Lou Schultz and William Scherkenbach taught me many years ago. As to "picking a target and pretending to improve the system by keeping centered on it," I think we need more context. It's true that arbitrary targets are harmful, but this is really the basis for world-class quality; Taguchi defined it in 1960 as "on-target with minimum variation." Getting a process centered on its specified nominal value and constantly reducing variation around it has long been the the goal of anyone trying to understand variation and use that understanding for improvement. I don't think that has changed; used properly, DMAIC projects can help a team achieve the kind of fundamental changes to a system that are needed when the process is stable but off-target or out-of-specification.
I'm going to jump back in by recycling my latest answer in the Deming discussion group on LinkedIn. The question was about Six Sigma and Deming. Two of my favorite Deming Disciples, John Dowd and John Constantine, feel that Six Sigma is fundamentally flawed and has very little place in any discussion of Deming. This is what I wrote:
I generally agree with John Dowd and John Constantine on most things, and I agree that Six Sigma-- as taught by many of the companies consulting in it these days--has some serious problems. Some of these approaches are, indeed, incompatible with the Deming philosophy. And some of the consultants using those incompatible approaches are large enough that the argument for "as generally taught" is probably sound. My suggestion, though, is that it doesn't have to be that way. Six Sigma is just a marketing vehicle; there is no standard for it (although ASQ would like to think that their SSBOK is one). As with TQM and every other quality approach there are people who do it well, and people who don't. Unfortunately, those who don't could care less about transformation, because many of them know very little about variation, and next to nothing about systems theory, psychology, or theory of knowledge. My disagreement is in what we do about this. I have been doing what I could, through my consulting practice, any conference appearances or workshops that I am able to do, and any writing that I can get published, to bring Deming principles into Six Sigma, and to use an approach in Six Sigma that is consistent with Deming. Clients who work with me only calculate the "1.5-sigma shift" and the "process sigma" as a curiosity and a metric for communicating with those less enlightened. They see and discuss the Red Bead, the Funnel, systems theory, SoPK, and the 14 points. They learn to see and use Six Sigma projects as one tactic in an overall process management system. I think if more Deming practitioners could find it in their hearts to do something like this, we'd have more success in using Six Sigma as one tactic in our overall aim, and reach more managers and executives (and potential managers and executives) with our message. This is what Lou Schultz and William Scherkenbach taught me many years ago. As to "picking a target and pretending to improve the system by keeping centered on it," I think we need more context. It's true that arbitrary targets are harmful, but this is really the basis for world-class quality; Taguchi defined it in 1960 as "on-target with minimum variation." Getting a process centered on its specified nominal value and constantly reducing variation around it has long been the the goal of anyone trying to understand variation and use that understanding for improvement. I don't think that has changed; used properly, DMAIC projects can help a team achieve the kind of fundamental changes to a system that are needed when the process is stable but off-target or out-of-specification.
Wednesday, April 1, 2009
If It's Measurable (and Important) Why Aren't You Measuring It?
If you're not measuring something that's important to you, why not?
I am always amazed when I talk to executives or managers--sometimes at a conference, sometimes when they've hired me to help--and find that they are not measuring the things they claim they care about, at least not in any useful or meaningful way. As an example...a couple of years ago I was talking to a manager who wanted a Six Sigma project chartered around reducing scrap for a cut-off process. He said at the outset, "If I could just do something about scrap. It's KILLING me!" I asked, "How much scrap does the process produce?" He said, "Well...last quarter it was about 23%." End of quarter had been almost 2 months prior to this conversation; I said, "OK...what was it yesterday?" "Don't know...the last number I had was from last quarter. I do know it was up from the quarter before..." I got him to agree to attend our next "Statistical Thinking for Leaders" course, and we worked out a plan to start collecting his data differently (and at a useful frequency), so we could learn enough about his scrap to make it worth chartering a project. A colleague of mine, Charles Liedtke, put it this way..."Quarterly numbers? Would you manage your checkbook that way? Using one balance per quarter?" To my scrap manager's credit, at least he was measuring something. I have seen many managers attempt to charter projects without having any data (or any way to get the data). Even given Deming's admonition that the most important numbers are unknown and unknowable, there are measurements that ARE important...if you're not measuring it, and measuring it daily, and tracking it in some useful fashion like a control chart, then why not?
I am always amazed when I talk to executives or managers--sometimes at a conference, sometimes when they've hired me to help--and find that they are not measuring the things they claim they care about, at least not in any useful or meaningful way. As an example...a couple of years ago I was talking to a manager who wanted a Six Sigma project chartered around reducing scrap for a cut-off process. He said at the outset, "If I could just do something about scrap. It's KILLING me!" I asked, "How much scrap does the process produce?" He said, "Well...last quarter it was about 23%." End of quarter had been almost 2 months prior to this conversation; I said, "OK...what was it yesterday?" "Don't know...the last number I had was from last quarter. I do know it was up from the quarter before..." I got him to agree to attend our next "Statistical Thinking for Leaders" course, and we worked out a plan to start collecting his data differently (and at a useful frequency), so we could learn enough about his scrap to make it worth chartering a project. A colleague of mine, Charles Liedtke, put it this way..."Quarterly numbers? Would you manage your checkbook that way? Using one balance per quarter?" To my scrap manager's credit, at least he was measuring something. I have seen many managers attempt to charter projects without having any data (or any way to get the data). Even given Deming's admonition that the most important numbers are unknown and unknowable, there are measurements that ARE important...if you're not measuring it, and measuring it daily, and tracking it in some useful fashion like a control chart, then why not?
Friday, March 27, 2009
More on Performance Evaluation
Note: this was originally a comment in a LinkedIn Discussion. The question was "Two of the problems I currently face: 1. If work standards and numerical goals get eliminated, we have to establish a new performance monitoring system. How do we evaluate performance fairly ? Leadership is hard to measure. 2. How do we design a fair compensation / reward system after that? Any suggestion?"
I think we all wrote reams on this in the DEN several years ago.
The problems are many, but mostly they have to do with an understanding of what it means to measure, and what it means to know. I had a conversation with a young HR person a couple of years ago about the criteria they were using to put people into a training program. I asked whether the peoples' managers actually knew them well enough to be able to predict whether an individual would be successful in the program. She said, "Well, I hope they are using performance evaluations to select them...you know, some objective data instead of just a manager's opinion." I asked her where they got the "objective data" from, and (of course) she told me they came from scores and rankings assigned by the managers. Then I asked, "How is that different from manager opinions?" She looked at me as though I were an alien or an idiot or both, and said, "They assign NUMBERS!"
It's always dangerous to try to make an inherently subjective task objective, but in this case, it's impossible. Anyone with a modicum of understanding of General Systems Theory knows that you can't separate a person acting in a system's performance from the performance of the system. It's one of the primary lessons from the Red Bead. Deming usually pointed out that it's trying to solve an equation with two unknowns.
It also ignores variation theory, although in many cases, the schemes put forward by management attempt to use variation theory as an excuse for some of the bad practices. In any group of people, for any given measure, there will be a distribution of performance. Because of system effects and interactions, the distribution is mostly the result of random variation. Some people will be at the higher end in some years and the lower end in other years, due to random variation.
Some people may end up in the upper tail of the distribution, more than three sigma away from the average. They may even stay there for a few cycles. These are people who are doing better than the system, and should be studied to find out how the system could be improved. They are the ones who should probably be rewarded more. If you asked everyone in the organization "who's our number one person?" all fingers would point at that person.
Some people might end up in the lower end of the distribution...they have managed to underperform the system. If you asked anyone who should go, they would point at that person. Those people are in need of help...maybe a different job in the same organization, maybe some training or a different manager, maybe a job in another organization.
Those people in the tails are relatively easy to identify. They are also rare. Most of the rest of the people are doing their best, and their performance is a result of random variaiton. Rating and ranking them is a step away from reality, and can't be done on any sound rational basis.
If you want to try some strategies that make sense, add "Abolishing Performance Appraisals," by Tom Coens and Mary Jenkins, to your reading list. They provide a very comprehensive treatment of the downsides, but also offer some great suggestions for "what to do instead."
I think we all wrote reams on this in the DEN several years ago.
The problems are many, but mostly they have to do with an understanding of what it means to measure, and what it means to know. I had a conversation with a young HR person a couple of years ago about the criteria they were using to put people into a training program. I asked whether the peoples' managers actually knew them well enough to be able to predict whether an individual would be successful in the program. She said, "Well, I hope they are using performance evaluations to select them...you know, some objective data instead of just a manager's opinion." I asked her where they got the "objective data" from, and (of course) she told me they came from scores and rankings assigned by the managers. Then I asked, "How is that different from manager opinions?" She looked at me as though I were an alien or an idiot or both, and said, "They assign NUMBERS!"
It's always dangerous to try to make an inherently subjective task objective, but in this case, it's impossible. Anyone with a modicum of understanding of General Systems Theory knows that you can't separate a person acting in a system's performance from the performance of the system. It's one of the primary lessons from the Red Bead. Deming usually pointed out that it's trying to solve an equation with two unknowns.
It also ignores variation theory, although in many cases, the schemes put forward by management attempt to use variation theory as an excuse for some of the bad practices. In any group of people, for any given measure, there will be a distribution of performance. Because of system effects and interactions, the distribution is mostly the result of random variation. Some people will be at the higher end in some years and the lower end in other years, due to random variation.
Some people may end up in the upper tail of the distribution, more than three sigma away from the average. They may even stay there for a few cycles. These are people who are doing better than the system, and should be studied to find out how the system could be improved. They are the ones who should probably be rewarded more. If you asked everyone in the organization "who's our number one person?" all fingers would point at that person.
Some people might end up in the lower end of the distribution...they have managed to underperform the system. If you asked anyone who should go, they would point at that person. Those people are in need of help...maybe a different job in the same organization, maybe some training or a different manager, maybe a job in another organization.
Those people in the tails are relatively easy to identify. They are also rare. Most of the rest of the people are doing their best, and their performance is a result of random variaiton. Rating and ranking them is a step away from reality, and can't be done on any sound rational basis.
If you want to try some strategies that make sense, add "Abolishing Performance Appraisals," by Tom Coens and Mary Jenkins, to your reading list. They provide a very comprehensive treatment of the downsides, but also offer some great suggestions for "what to do instead."
Friday, March 20, 2009
"Making Six Sigma Faster"
I recently had a manager tell me that his company had decided that Six Sigma just “takes too long,” and that they were implementing a new mandate to “Make Six Sigma Faster.” I asked him why they thought Six Sigma takes too long. He told me that the average time to complete a project there had been 8-9 months; under the new program, Black Belts were going to be expected to complete projects (at least through the IMPROVE phase) in 90 days or less.
I couldn’t help it…I broke out laughing, and asked him “By what method?”
“Well,” he said, “we’re not sure yet, but we think if they start holding more meetings, and never go into a meeting without having the deliverable already drafted, that will help.”
I had asked Deming’s famous question for a reason. I was very familiar with this particular deployment, and I knew that the reasons their projects always ran long were many, but almost none had to do with the Black Belts or the number of meetings they held.
This organization had decided at the beginning that they just didn’t have time to do a couple of days of Champion training. They had decided instead that they could get along with a 2-hour teleconference and a required reading list. This was a big organization, and they had never bothered to set up any kind of listening posts or other pipeline-feeders, had no project portfolio management, had not coordinated with the PMO, hadn’t trained any middle managers in SPC, Six Sigma familiarization, or anything else. Black Belts were pretty much expected to find their own projects, and in many cases had to hunt down anyone willing to sign on as a Champion (sometimes, they just picked another Black Belt, because “at least I got someone who understands Six Sigma.”) Getting good data was another problem. Usually, there were no data available for even deriving a baseline, much less for stratification or for digging into cause systems to find “x’s.” Just collecting the data for a baseline might involve a couple of months’ worth of work. The organization often relied for stratification on “reason codes,” the use of which were consistently proven unreliable when tested using attribute agreement analysis.
These were the primary factors driving project lead times, but there was no plan to deal with these factors, because it meant getting leadership to change, and no one at my manager friend’s level had the ability to push that noodle uphill. So they were just going to do the usual thing…put it into the expectations for the Black Belts. All you have to do to get the variation narrower is tighten the specs, right?
This was Deming’s point; to paraphrase, “If you could cut the project time by 60 percent this year without a method, then why didn’t you do it last year? Must have been goofing off…”
Listen, executives: This is too important. Six Sigma, implemented properly and led from a systems perspective, is a proven methodology that will make your business better, your customers happier, your revenues higher, and your costs lower. But it’s not something you bolt on, walk away from, and just wait for the cash to roll in. You have to lead it, you have to be engaged, you have to remove obstacles and make Six Sigma a strategic component of a larger Quality Management System. In the end, if it fails, you can’t blame the Black Belts you didn’t support, or the culture you didn’t change, or even the consultants you didn’t listen to. It’s not that it won’t work at your company…but it certainly won’t work if you don’t lead it!
I couldn’t help it…I broke out laughing, and asked him “By what method?”
“Well,” he said, “we’re not sure yet, but we think if they start holding more meetings, and never go into a meeting without having the deliverable already drafted, that will help.”
I had asked Deming’s famous question for a reason. I was very familiar with this particular deployment, and I knew that the reasons their projects always ran long were many, but almost none had to do with the Black Belts or the number of meetings they held.
This organization had decided at the beginning that they just didn’t have time to do a couple of days of Champion training. They had decided instead that they could get along with a 2-hour teleconference and a required reading list. This was a big organization, and they had never bothered to set up any kind of listening posts or other pipeline-feeders, had no project portfolio management, had not coordinated with the PMO, hadn’t trained any middle managers in SPC, Six Sigma familiarization, or anything else. Black Belts were pretty much expected to find their own projects, and in many cases had to hunt down anyone willing to sign on as a Champion (sometimes, they just picked another Black Belt, because “at least I got someone who understands Six Sigma.”) Getting good data was another problem. Usually, there were no data available for even deriving a baseline, much less for stratification or for digging into cause systems to find “x’s.” Just collecting the data for a baseline might involve a couple of months’ worth of work. The organization often relied for stratification on “reason codes,” the use of which were consistently proven unreliable when tested using attribute agreement analysis.
These were the primary factors driving project lead times, but there was no plan to deal with these factors, because it meant getting leadership to change, and no one at my manager friend’s level had the ability to push that noodle uphill. So they were just going to do the usual thing…put it into the expectations for the Black Belts. All you have to do to get the variation narrower is tighten the specs, right?
This was Deming’s point; to paraphrase, “If you could cut the project time by 60 percent this year without a method, then why didn’t you do it last year? Must have been goofing off…”
Listen, executives: This is too important. Six Sigma, implemented properly and led from a systems perspective, is a proven methodology that will make your business better, your customers happier, your revenues higher, and your costs lower. But it’s not something you bolt on, walk away from, and just wait for the cash to roll in. You have to lead it, you have to be engaged, you have to remove obstacles and make Six Sigma a strategic component of a larger Quality Management System. In the end, if it fails, you can’t blame the Black Belts you didn’t support, or the culture you didn’t change, or even the consultants you didn’t listen to. It’s not that it won’t work at your company…but it certainly won’t work if you don’t lead it!
Subscribe to:
Posts (Atom)
.jpg)
