Part 1 (19 May 2013)
Recently it seems I am regularly being contacted by (or informed about) people and
organizations trying to revive CMMI*. The market appears to be shrinking for the
professionals in the process improvement world; at least, the market is not growing
as fast as the number of professionals being certified to teach it or use it for official
appraisals. Add to that, all the "experts" and other consultants who have set up shop
without having official training or recognition and you get into a lose-lose situation.
And so, various groups appear to be talking about making CMMI faster, leaner,
easier, cheaper, more secure. They are offering different approaches and ideas,
some of them very good. My first concern is that the model is easier and cheaper to
implement, it will have to be stripped of a lot of its more advanced requirements,
and the benefits thereof. But the model is not being used as efficiently - or as
frequently as it should.
One of the most difficult choice people need to make regarding a loved one is the
decision that maybe it is time to "pull the plug", to stop trying to keep alive a dying
parent. As a CMMI instructor, appraiser and consultant, it pains to suggest this, but
maybe the time has come to pull the plug on CMMI.
About thirty years ago, the US Department of Defense required that all their software
suppliers should be "Maturity Level 3"; but they have now stopped because they
noticed no significant improvement in the quality or reliability of the products and
services they were purchasing! Companies all over the world have been applying
CMMI and nothing has not noticeably improved!
I believe that it is an excellent product and, through its various transformations from
maturity questionnaire to software-CMM to CMMI v.1.3, the product has generally got
stronger. There are a few things I would have done differently, but overall, the
product got stronger and better.
However, it is not delivering the expected results and, notwithstanding the strong
focus on measurement, there are still no clear statistical data on the benefits.
Yet, it works. I know: I have seen it.
Maybe this is the time to accept that there is a fundamental issue with the model
and an in-depth review of what it is supposed to be is necessary. As I see it, there is a
big weakness in the tool as it tries to be two different things.
If it is to be a measurement tool, used for appraisals and measuring the aptitude of
potential suppliers, then the appraisal methodology needs to be simplified - an ISO
audit typically lasts a couple of days; why can't we do it as efficiently?
On the other hand, if it is to be used as a process or performance improvement tool,
then it should be completed and extended with implementation suggestions.
As long as we don't know what the purpose of the model is, we will be stuck with this
hybrid version, neither here nor there, not this or that, trying to be staged and
Currently, the value of the model for efficiency and continuous quality improvement
is undermined by the vast majority of users who are only interested in getting a maturity level. They are doing the minimum required to get the level. As soon as the
lead appraiser leaves the organization, they stop doing the tasks that were just
recognized. The result is that we see organizations all over the world who have
fooled an appraiser into giving them an undeserved rating - thereby demonstrating
that apparent high-maturity organizations continue to deliver bad quality, late. The
natural conclusion is that the model is a waste of time and money.
At the same time, more consultants and appraisers are continuing to be produced,
with little experience in the real world. The pressures of the market, of the people
trying to game the results, makes it very difficult for these young consultants to have
any sense of solid ethics.
Now, it is time to let go and do a postmortem, identify what went wrong. Then, we
can lay the CMMI to rest and create something which works more efficiently - maybe
(*CMMI is the Capability Maturity Model Integration, licensed by the CMMI Institute. It
is a process model used to measure the maturity of an organization with regard to
their processes and work practices. CMMI is a registered trademark of the Carnegie
Mellon University, based in Pittsburgh, PA)
Part 2 (23 May 2013)
It seems that my previous post called "Reviving CMMI" generated quite a lot of
reactions. Some people seem to understand that I was suggesting to let the old girl
die, some encourage the idea, others were horrified at this attempted matricide. A
few reacted supportively or critically without giving enough information for me to
know what they thought I had said.
So, I decided to clarify my feelings on the subject. I am a process improvement
consultant, I have been living off CMMI for many years and would not recommend
cutting off the hand that feeds me without careful consideration.
CMMI cannot be considered as fit for purpose
This is largely because the owners of the model, the user community and the market
do not agree on its purpose. The CMMI Institute (following on in the footsteps of the
SEI) places a large emphasis on appraisals and maturity levels, publishing numbers
of appraisals, time to reach a level, number of maturity levels per country and per
industry - in fact all the measurements and data produced are directly
measurements of the appraisal results. But, at the same time, we are presenting
CMMI as a tool for process and productivity improvement rather than a certification
Without a clear understanding of the purpose, it is not possible to design something
fit for purpose.
As a Tool for Improvement
As a tool for productivity improvement, the model does not contain enough
information to facilitate a seriously useful and helpful implementation. There are
hidden relationships between process areas and practices which are not easily
identified or understood.Personally, I try to use the model as an efficiency and quality improvement tool; I
need to spend an excessive amount of time clarifying the cause and effect
relationships within the structure of the model. I also need to explain in detail how to
understand the purpose and meaning of things within a business context. The
standard training does not explain the evolution from maturity level 2 to 3 and
beyond. There is a vague statement that it is not recommended to skip levels, but no
clear rationale clarifying what are the risks and consequences. The relationship
between specific and generic practices is not sufficiently clear in the model or the
training. These are vital facts if you want to use the model for your business.
If the model is to be focused on improving quality and productivity, it needs to
include more information on how to apply it successfully.
As an Appraisal Mode
An ISO audit takes a couple of days, a CMMI appraisal can take a couple of weeks.
Why? A number of "certified lead appraisers" do not appear to understand the
purpose of the model. There was a recent case of an organization which was
required, according to their appraiser, to have a separate policy document for each
CMMI process area, clearly stating the name and structure of the PA - this is not the
goal of the model, but people with no experience of the "real" world are being
authorized to appraise successful organizations; they are frequently focused on
respecting the comma of the law without understanding.
The current appraisal method spends a lot of time trying to find evidence of
practices, but could be a lot more focused on the impact and results of successful
implementation of recognized and accepted best practices.
As I stated, I believe it is time to perform an in-depth lessons-learned analysis to find
out what went wrong and how to correct the product, making it into something that
will have the impact which was promised.
This must start with an understanding of the purpose of the beast. If we are talking
about a tool for process improvement, we need an approach to educating of
practitioners and users, which focuses a lot more on the practical side of change
management and improvement. We need more information regarding the
implementation of the practices. Potentially, this may mean that the core model gets
completed with a series of "recipe books" for different industries or contexts.
I would like to see the model completed with clear business related impact and
influence statements, clarifying why things need to be done to save those who are
implementing the letter of the law from their own stupidity.
I would like to see CMMI separated and organized so as to distinguish the
improvement potential from the appraisal requirements. I am not sure if both can
survive with the same name, but trust that the SCAMPI appraisal methodology can
be adapted to other models and standards and be recognized in its own right. The
appraisal methodology needs to focus a lot more on the business and cultural
aspects of the model, stopping lead appraisers seeking to burden businesses with
bureaucracy because a sub-practice says that is the way it should be.CMMI should be perceived as a pragmatic approach to assist organizations increasing
job satisfaction and customer satisfaction. And we should be able to demonstrate
that from the beginning. The appraisal method should focus on measuring the
results, not the practices.
Part 3 (03 June 2013)
Third item on this topic, I know. Some people believe that I spend too much time
complaining without proposing a solution, so here is my proposal: measurement and
analysis should be expected from the start.
Many times, when I have asked for evidence of the implementation of measurement
and analysis, I have been provided with evidence of project monitoring and control.
Measuring that your project tasks are progressing, that you are respecting delays
and budgets is not part of MA, it is part of WMC/PMC. It should not be difficult to
include in either the model or the appraisal methodology a better set of examples of
how MA should be applied, and place this as a requirement in the appraisal process.
Currently, the appraisal method focuses on the practices; during the training, we say
that the goals are what is important and are required, the practices are only
"expected". However, during the appraisal, we focus on measuring the practices
rather than the goals. We assume that if all the practices are in place, then the goal
must be satisfied, if one practice is missing, then the goal is not satisfied. And even if
we did, I cannot help but notice that the goals (and the purpose) statement all focus
on activities, on tasks, on practices, and not on results.
Let's require business measurements and demonstrations of results for each goal.
CMMI is (supposedly) there to help achieve business productivity results and not just
to do a series of tasks to get a certificate we can hang up behind the reception desk.
Why do you do "Requirements Management"? Show me results. Show me that the
number of issues related to unidentified change requests has diminished; show me
the reduction in unexpected requirements appearing during the V&V stage; show me
data that show how the time to find a deviation from customer requirements in your
project plans and other work products has gone down, because you have
implemented a successful "two way traceability". Show me measurements that
demonstrate the impact on the quality of your products and services, on customer
satisfaction. If you cannot show me that, you have not implemented a useful
approach to requirements management. Of course, people will argue that you
cannot measure improvements at maturity level 2, you can only really have useful
measurements at maturity level 4. But that is not true. You do not need control
charts or five years of history to show a trend. Because if you did, no one would
manage to satisfy the expectation for a quality trends in the PPQA process area at
maturity level 2 - or maybe you have just skipped over that passage and focused on
having a static checklist? If you implement a practice, whether from the CMMI or
elsewhere, it is because you expect to see a change one that practice is performed;
if you expect to see a change, there is something which can be measured.This is not a complicated addition to the model, it is only a clarification of the real
purpose behind each one of the goals. A change in the appraisal process of this
magnitude would ensure that people understand that CMMI actually has a benefit,
and it would allow us (finally) to have some decent metrics as to the value of the
I don't know if anyone will read this or pay attention, but I am glad I got it off my
chest. Hopefully, my next post will be about something else. Maybe I will develop a
business based appraisal system of my own, but if I am not supported by the
community at large, no one will use it and I will waste my time: it is so much easier
to do what the model says without thinking.
and you might get a certificate to hang up, a logo to put on your website...