Surveys on software engineering excellence (SEE) were designed and administrated in 2005, 2006, and 2007 in conjunction with the Japanese Ministry of Economy, Trade and Industry. In the survey, the software engineering capability was measured from the seven viewpoints of deliverables, project management, quality assurance, process improvement, research and development, human development, and customer contacts. In the present paper, we present the results of a cross-section analysis and a panel analysis. Through a cross-section analysis of the SEE data, we found superior deliverables and business performance to be significantly correlated with the effort expended on human resource development, quality assurance, research and development, and process improvement. For the panel analysis, we integrated 233 valid responses for the three years into a new database and identified 151 unique IT firms. Based on the results of the panel analysis, first, most SEE factors for a particular year had significant positive influences on the same factors the following year. Second, there were three paths to improving the level of deliverables, namely, through project management, quality assurance, and research and development, in a particular year. Third, some SEE factors had a significant positive influence on different SEE factors in the following year. Finally, several negative paths were observed, which implies that the effort put into a particular factor did not pay off in the short-term.
Lessons learned from 25 years of process improvement: the rise and fall of the NASA software engineering laboratory