ODC - Orthogonal Defect Classification

Defects in the field

Software released into the field, is usually of a higher quality than when it is in system test or beta test. That reason is one of the objectives of the beta test. Software failure, due to defects, economically impacts the vendor one way or another. The vendor faces the service cost of fixing defects (which can be considerable) and can be adversely impacted by customer dissatisfaction with product quality.

One of the many reasons a customer calls in with a problem is usually due to a software fault. Problem calls originate due to several reasons that may not be due to a programming error. For instance, a customer problem may be due to difficulty in installation, misleading instructions, inadequate documentation etc. In each event, as far as the customer is concerned, it is a problem with the product. A subset of these problems would eventually warrant a change in the code - which in our terms would qualify for a software defect.

For the moment, let us focus on the subset of problems that are associated with defects. A customer call then reflects the failure due to a software fault. There could be several calls from different customers due to the same fault. The problem handling process in IBM allows for a symptomatic search on problems to investigate and identify the rediscovery of already known problems. If a rediscovery occurs and a fix is already available for that problem, it can be shipped to the customer right away. However, if the problem being experienced reflects a new fault hitherto unnoticed, it would require further investigation to provide a resolution.

When a new software fault is suspected through the problem management system, an authorized program analysis report (APAR) is opened to investigate it. The APAR starts the service process off to diagnosis, providing temporary relief when critical, and works toward developing the fix. The APAR thus becomes the vehicle to develop the fix, test the fix and then make the fix available through a world wide distribution network. The APAR thus straddles the service and the developmental process, since it starts off in the field, gets resolved and tested by developers and is integrated into future software releases in that product. The APAR is the king of software defects. It costs a lot of money, and is high up in the priority lists of customers, developers, and service people. No one wants to have APARs, neither the customer nor the developer, and it certainly gets a lot of attention.

At first approximation, an APAR would fit the definition of a software fault. However, on closer examination, it may be more than one fault that is being referred to. While the APAR certainly contains a fix to a fault, in actuality it may have others as they could be targeted to several releases in the field, and might also pre require other APARs. In an over simplified definition, it would pass for a fault. For the purpose of this paper, we can assume APAR and software fault to be synonymous.

It is interesting to note that APARs which are meant to fix problems, might also be the cause of new ones. This occurs when a bad fix is shipped out, i.e. the program temporary fix (PTF) is in Error - causing the infamous PE APAR. This can occur due to several reasons, the most dominant being that testing a fix under the customer environment is very hard. Thus, despite efforts to provide a good fix, the complexity of a customer environment is probably never replicated in the lab - resulting in inadequate test and another fault escaping into the field. PE APARs, though few and far between do exist and get the attention of the customer and the vendor. The figure on the defect lifecycle, thus, shows an origin of errors as fixes, which may sound like a circular statement, but nevertheless is a fact of life.