ODC - Orthogonal Defect Classification

5. Relating Defect types to Symptom Groups

In this section we bring together the results from the earlier two sections. Section 3 discussed partitioning defects into four sub-populations (called symptom-groups) that demonstrated varying degrees of reliability growth. Section 4 developed the defect type distribution. We now proceed to relate the defect types (cause) to the reliability growth (effect) experienced by the symptom groups. We relate defect types to the symptom groups by examining the defect type distribution for each of the four symptom groups. Thus we identify possible differences in the makeup of defect types leading to differences in the corresponding reliability growth curves. Figure 5. contains four bar charts, one for each of the symptom groups, showing the distribution of defect types. Each major defect type is represented by a bar which is sub-divided to show the proportions of its sub-types. Notice that defect type distribution for Symptom Groups 2, 3 and 4 are similar to each other and also the aggregate distribution in Figure 4. That is, these three symptom groups have almost the same relative proportions of the major defect types, i.e., function defects being the largest and documentation being the least. However, there are differences among them when considering the proportions of the individual defect sub-types. Symptom Group 1 is markedly different from the rest both in the major defect types and the sub-types. We now look at these differences in greater detail. Symptom group 1, is a major departure from the other three groups. Firstly, it contains defects from only 3 defect types. (Note, statistically this does not mean that documentation and checking defects are absent in this sub-population. They may exist, but are likely to be very small contributors). Secondly, the fraction of initialization defects is much higher than those in the other three sub-populations or the aggregate. Initialization defects constitute almost 35% whereas it is typically 10-15% in the others. A large fraction of the initialization defects (two thirds) is due to missing initialization. However, missing initialization exists in a nominal (as compared to the aggregate) fraction in Group 2 but does not exist in either Group 3 or Group 4. The other defect types of Group 1, namely function and assignment do not show similar strong differences. Function defects continue to be the largest; slightly higher than others, as expected. The assignment defects are about the same fraction as the aggregate. Recall that Group 1 demonstrated the most inflected growth curve. The inflection quantitatively corresponds to a low r value, and physically to defects that may have a lot of dependence between each other. To visualize dependence, imagine defects on a graph where the edges correspond to execution paths. Two defects along the same path could require the removal of the first to detect the other. Thus, the two defects along the same path could be dependent affecting the shape of the growth curve. However, if they were along disjoint paths they may be independently detected. Dependence can cause an initial delay in the detection process with the rate of detection rapidly increasing once a few defects are detected. Thus, it is conjectured that dependence causes an inflection in the growth curve. A more detailed discussion on the topic is contained in . From the defect type distribution we know that Symptom Group 1 defects have a relatively larger contribution of initialization defects. Given that initialization defects occur upstream in the execution path they are likely to be shared by more than one path, contributing to greater dependence between defects. Therefore, until the initialization defects are fixed others that share the same path are not likely to be detected. Thus indicating that initialization defects could contribute to a very inflected growth curve. However, note that Group 2, which also has a fairly inflected curve, does not have as large a number of initialization defects. This seeming inconsistency is useful to identify the probable cause and clarify the relationship. Recall that although Group 1 has a large fraction of initialization defects, the major difference is due to the missing initialization which accounts for almost two thirds of them. Missing initialization defects also exist in Group 2 but not in Groups 3 or 4. This points to a combination of missing initialization and initialization defects being strongly related to the inflected nature of the growth curve. Analysis of variance can be used to further access the relative contributions of defect types. However, more important is the physical reason behind this plausible relationship and its impact on the overall growth experienced. Primarily, the testing strategy can be modified to weed out missing initialization defects well before further path testing is attempted. Group 2, 3 and 4 do not display a distribution of defects types that immediately draws attention to the type of growth they experience. The only exception is that Group 4 has a much smaller than average size of initialization defects. However, since the Group 4 sub-population is not as well defined by the cluster of r and phi parameters, we are not inclined to analyze it further. It is interesting that we are able to establish one of the cause-effect relationship between a defect type (cause) and the inflection in the reliability growth curve (effect). The fact that there are plausible cause and effect relationships should be welcome. Establishing more such relationships will allow for the software development effort to be understood and controlled better. It will also pave the way for using more advanced techniques in software development, akin to other manufacturing processes. We are pursuing the use of defect type classification in more products to allow us the additional insight that it can provide.