Massive budget and schedule overruns continue to plague business and government projects alike. Flyvbjerg and Budzier’s recent research published in the Harvard Business Review shows that among the 1400 projects surveyed, on average they were 27% over budget. Even more alarming, 16% of project had massive overruns, what they call a “black swan” with a cost overrun of 200% on average and late by 70%. These are projects that can cause stunning failures in already troubled companies.
An example discussed – Kmart undertaking an “IT modernization project” costing $1.4 billion in 2000, then realizing that it was so highly customized as to be cost prohibitive to maintain. They then ran a $600 million supply chain software project in 2001 – which they pulled the plug on in 2002 – both contributing to Kmart’s decision to file for bankruptcy.
These reports mirror similar data from the late 90’s when Standish published its original CHAOS report, oft cited by Agilists. Depressingly, the 2009 CHAOS report shows an even more dismal trend of cost/budget overruns.
Why, in the last 15 years, why has the industry not gotten better at delivering large IT projects? Large is a subjective term. Let’s define that in dollars for now, say anything over $10 Million dollars. To answer that, the key may be in the projects studied.
Most of the projects studied by Flyvbjerg and Budzier were government projects (92%). However, they claim the results were mostly the same between them and private company IT projects they looked at. So, it’s probably not only due to the massive incompetence by our fellow technologists in Government that are the problem. From the article, it appeared most of the projects were IT replatforming projects, for example – installing COTS software for existing in house solutions – Oracle, SAP or a CRM system. My theory is that it is these very large re-platforming projects that are frequently the issue. They touch so many systems, so many discrete business units involved across the globe, and the desire to do customization so they become impossible to upgrade.
Now, there may be good reasons to replace a creaky platform, as Jonny Leroy puts it, but the way you do it is just as important as the business justification. One pattern we use at ThoughtWorks is the Strangler Pattern, where you slowly replace the existing application with a new application, to avoid a spectacular project failure.
An example from the article of a successful project by the Emirates bank in 2006, shows a successful way to run a large complex project. This was a project to replace the banking back-end with an upgraded system. The project was threatened to go off the rails when the bank merged with National Bank of Dubai, and they had to work for both banks.
The lessons learned from this successful project were to:
1. Stick to schedule
2. Resist changes to the projects scope
3. Break the project into discrete modules
4. Assemble the right team
5. Prevent turnover among team members
6. Frame the initiative as a business endeavor, not a technical one
7. Focus on a single target – readiness to go live, measuring every activity against it.
Many of these are hallmarks of good Agile projects. Break a large project into smaller pieces, stay focused on your release criteria and make sure that you are solving a business problem, not a technical one. Number two may strike us as an Agile faux pas, not allowing change in scope, but I believe the authors are referring to “Mission Creep” rather than changing featues in a requirements spec. Drifting missions or just unclear goals strikes me as a hallmark of any unsuccessful endeavor.