Gery Derbier forwarded a link on the agile list for “Exploding Software Engineering Myths”, an article by Janie Chang at Microsoft Research. Chang describes the research of the Empirical Software Engineering and Measurement (ESM) research group at Microsoft . The group investigated a number of software engineering myths empirically using the large software engineering teams at Microsoft.
The group used much of the conventional wisdom laid out in The Mythical Man Month as the starting point for their research. I’ve read The Mythical Man-Month and I highly recommend it to anyone that is involved in the business of producing software. So much of what was learned from the OS/360 experience as recounted by Fred Brooks rings true with our own experiences as software craftsmen. Its great to see a group do an empirical study to followup on the wisdom laid out by Fred Brooks.The ideas examined by the ESM group and its lead researcher Nachi Nagappan include: more code coverage leads to more quality, writing test code first leads to more quality, writing assertions leads to more quality, code structure follows organizational structure and geographically dispersed teams have more difficulty creating software. Read the linked article for a summary of the ESM group’s results for each of these ideas. The summary provides links to the details for each of these in the form of research papers published by the ESM group.
I found it quite interesting that test-driven development teams took 15 to 35 percent longer to complete their projects, but had 60 to 90 percent better code quality (as measured in terms of defect density).
I’ll have to read the linked paper for more context, but I wonder if they consider the “time to complete” for non-TDD teams to include the time spent fixing defects after release.
My general feeling is that IDEs and software toolkits are still catching up to TDD and aren’t yet making us as productive as we could be doing TDD. Also, fitting TDD into a legacy codebase definately takes longer than doing “just a quick fix” or “just a quick change” because the original code wasn’t written to be testable.