ODC - Orthogonal Defect Classification

Defects during development

Software defects during the development process do not only occur due to program abnormal terminations or failed tests. In fact, the largest number of defects found during the lifecycle of a software product, often occur without the execution of the code. For example, a code inspection could typically yield 40% to 60% of the overall defects.

A software product usually begins with some set of requirements, either explicitly stated by a customer or identified through a vision of the market. Between the requirements and the availability of the product, there is design, code development and test. The words design, code and test do not automatically imply a pure waterfall process. On the other hand it is hard to imagine a successful product developed without requirements or code written without a design or tests conducted without code. A pure waterfall process tends to have these phases well identified and separated with documents that capture the essence of these phases, providing well defined handoffs between them. In more recent times, a small team with iterative development is commonly talked about, where the reliance of documentation and distinct handoffs are less emphasized. However, having some record of completion, milestones within a process are not only useful for project management but also internal for communication.

No matter what the exact implementation of the software development process, there is a distinct moment when a requirement, design or a piece of code is considered frozen. Its representation may be graphic, English text, video or audio tape. But the fact that there is agreement on an issue, the completion of an artifact in the development process, indicates a clear record of intent. Any further change to this record of intent is a change that is considered a defect. The tracking of a defect depends on how the process allows for a change to be understood, debated and handled. We are not arguing about whether or not there should be a moment that freezes the record of intent, for that would be a subject for a discussion on process existence. Nevertheless, it should suffice to say that if there were no moment when an intent of record is frozen - the process enters the domain of continuous change, which could yield an infinite completion time.

A process usually has several checks interspersed to ensure that the product being developed meets the record of intent. These checks could be reviews, inspections, informal tests, formal functional tests, stress tests, etc. Each of these verification efforts can identify deviations to the intent, which warrant change. Any thing that requires change is a defect. Thus, there are defects that arise from requirements inspection, design inspection (usually, a high level and a low level), a code inspection (of which there might be one or two), a unit test, a functional verification test and a system test. All these verification efforts can yield defects. Depending on what is counted you may find that the number of defects per thousand lines of code vary, from as high as 90 to as low as 10.

Following the final round of systems test within the organization, a product is often released to a subset of the customer base as a beta. The beta tells the customers to set their expectations appropriately while opening up a channel to listen on problems experienced with the product. When the vendor is satisfied about the products stability and performance (or forced to do so for business reasons), the product is announced and made generally available in the market. The product announcement is usually a key milestone in the history of the product, often accompanied by a celebration in the development shop.