Several professional and consulting organizations publish surveys of Project Management every year. I used to be a big believer in the past but became skeptical in the recent years, as there seems to be weaknesses/bias in the survey design, administration and analysis. I give couple of examples to support my change in belief and suggest the need for organization relevant surveys.
Standish Groups' CHAOS study is famous for painting a bleak picture of software due to high rates reported in its survey findings. In 1994, it was reported that a shocking 16 percent projects were successful, another 53 percent of the projects were challenged, and 31 percent failed outright. While the numbers improved in subsequent years, still the issue highlighted remained the same that software projects are out of control. In 2010, J. Laurenz Eveleens and Chris Verhoef of Vrije Universiteit Amsterdam published "The Rise and Fall of the Chaos Report Figures"(PDF opens in a new window), which highlighted the major flaws in the study and its impact based on an independent database of Projects while following the methodology of Standish Group. The research concluded that the Standish definitions of successful and challenged projects are misleading, one-sided, pervert the estimation practice, and result in meaningless figures.
Recently I have come across PMI 2013 Pulse of the Profession report (PDF opens in a new window) and read with interest the claim that organizations risk, on average, $135 million for every billion dollars spent. Low-performing organizations, however, risk 14 times more money than their high-performing counterparts, Talent management, Standardization of practices and tools and Strategic alignment were identified as key focus areas to become high maturity organisations, which have reported 90% project success. In order to investigate the survey in more detail and I accessed the question wise responses in Pulse Interactive Report (accessible to members).
Based on my preliminary analysis, I found that the definition for success used for the survey is delivering the project's initial scope within the initial time and budget estimates. When I looked at the reasons for failure, I found "Overall Change in organization's priorities" and "Inaccurate requirements gathering" as top ranked. This is not surprising if the triple constraint is the one driving the survey design. In order to apply the findings, the question that needs to be asked is whether your organization is still following the triple constraint. The PMBOK 4th Edition leaves out the definition of the project success. In the fifth edition it is defined in relation to last baselines approved by authorized stakeholders. As the survey participants may not have consistent idea of project success, the responses may not be consistent. The survey findings need to be taken with a pinch of salt, if your project is exploratory and software intensive.
As we enter the last quarter of the year, this is right time to assess the project performance during the preceding twelve months within your organisation with a custom survey designed to identify the strengths and improvement areas of interest to plan for the next year. Even if the number of projects is small, the results will be much more relevant and useful.
What has been your experience with relating to the surveys and also in house surveys? Please share the same.
Standish Groups' CHAOS study is famous for painting a bleak picture of software due to high rates reported in its survey findings. In 1994, it was reported that a shocking 16 percent projects were successful, another 53 percent of the projects were challenged, and 31 percent failed outright. While the numbers improved in subsequent years, still the issue highlighted remained the same that software projects are out of control. In 2010, J. Laurenz Eveleens and Chris Verhoef of Vrije Universiteit Amsterdam published "The Rise and Fall of the Chaos Report Figures"(PDF opens in a new window), which highlighted the major flaws in the study and its impact based on an independent database of Projects while following the methodology of Standish Group. The research concluded that the Standish definitions of successful and challenged projects are misleading, one-sided, pervert the estimation practice, and result in meaningless figures.
Recently I have come across PMI 2013 Pulse of the Profession report (PDF opens in a new window) and read with interest the claim that organizations risk, on average, $135 million for every billion dollars spent. Low-performing organizations, however, risk 14 times more money than their high-performing counterparts, Talent management, Standardization of practices and tools and Strategic alignment were identified as key focus areas to become high maturity organisations, which have reported 90% project success. In order to investigate the survey in more detail and I accessed the question wise responses in Pulse Interactive Report (accessible to members).
Based on my preliminary analysis, I found that the definition for success used for the survey is delivering the project's initial scope within the initial time and budget estimates. When I looked at the reasons for failure, I found "Overall Change in organization's priorities" and "Inaccurate requirements gathering" as top ranked. This is not surprising if the triple constraint is the one driving the survey design. In order to apply the findings, the question that needs to be asked is whether your organization is still following the triple constraint. The PMBOK 4th Edition leaves out the definition of the project success. In the fifth edition it is defined in relation to last baselines approved by authorized stakeholders. As the survey participants may not have consistent idea of project success, the responses may not be consistent. The survey findings need to be taken with a pinch of salt, if your project is exploratory and software intensive.
As we enter the last quarter of the year, this is right time to assess the project performance during the preceding twelve months within your organisation with a custom survey designed to identify the strengths and improvement areas of interest to plan for the next year. Even if the number of projects is small, the results will be much more relevant and useful.
What has been your experience with relating to the surveys and also in house surveys? Please share the same.
No comments:
Post a Comment