ODC - Orthogonal Defect Classification

1. Introduction

An important question in the area of software engineering is whether there exist measurable cause-effect relationships between the kind of defects and the resultant development effort. If it were true that there existed quantifiable cause-effect relationships, then schemes could be devised to recognize them and take suitable corrective action on the process. It would also form a turning point in trying to make software development into a controllable process much like other manufacturing processes. Although this is a hard question to answer completely, one can formulate sub-problems that could eventually lead to an answer. This paper, investigates one such sub-problem and demonstrates the existence of such relationships. Reliability growth modeling has existed for several years. Growth modeling traditionally deals with prediction after making some fundamental assumptions on the error detection process. The reliability growth is, in a sense, an effect of the defects and the testing process. On the other hand, defect characterization has been used as a means to identify the nature of problems inherent in the process. In another sense, the defects are a cause at least with respect to measurements such as reliability growth. Within this framework a sub-problem that is pertinent to the goals of this paper is to investigate relationships between types of defects and their result on the process measured by, say, reliability growth. Now that we have defined the sub-problem of investigating relationships between defect types and the reliability growth, we can embark on looking at what data is available for such a study. As it happens, there is plenty of data but not quite in the form that would allows us to answer the question. Many labs do not have their defect data elaborately categorized by defect type. The information exists but in the form of plain English text that cannot be easily parsed for semantic content. Thus, the approach of separating the defects into sub-populations by defect type and then investigating their growth characteristics cannot be done. Thus, the analysis gets a little more complicated. Given the lack of defect-type information in the data a reverse approach has to be tried. Potentially, if it were possible to identify sub-populations that had different growth characteristics from the average, the sub-populations could be analyzed for possible differences in their composition of defects. This approach would work, if one could identify such sub-populations. It still does not solve the problem of not having the defects categorized by defect type. However, if the separation could be achieved, then it is possible to sample the defects and categorize them. There have been several studies on the topics of both growth modeling and defect characterization. However, this search for potential cause-effect is not so common. There have been some interesting studies on the relationship between distribution of errors and complexity . is an experimental investigation on the effect of Fortran and Ada on program reliability. is a study of software defects found in the field in operating systems code and their impact on availability. The data used in this paper comes from the development of a fairly large software product. We discuss the details of the data source in Section 2 as also its attributes. We were able to divide the data into sub-populations that demonstrated, in Section 3, how we were able to identify sub-populations that had varying degrees of reliability growth. Section 4 describes how defect type categorization is done and also shows some results that come directly out of the categorization. Section 5 brings together results from the previous two sections, relating defect types (cause) to the differences in the observed growth (effect). Section 6 summarizes the key results from sections 3, 4 and 5.