Fast Abstracts Archives . .

Software Component Reliability Analysis

Over the years of teaching and applying quantitative methods of analyzing software reliability, I have not been satisfied with having to wait until testing starts before doing any real quantitative analysis of software reliability. Current approaches rely on collecting failure event data during system test and then “fitting” one of a number of reliability growth models to the data . The “fitted” model is used to estimate reliability measures of interest.

To overcome this shortcoming, we have been developing an approach called Software Component Reliability Analysis (SCRA). The approach has been distilled into a six step procedure (see the table below for an outline of what is involved in each step). The procedure starts very early in the software development process, during the architecture or high level design phase. During this phase, the software is divided into modules and the features and functions specified in the requirements are allocated to the modules. The analysis starts by dividing the software into components. The component breakdown follows the software module breakdown but includes additional structure to aid reliability analysis. Descriptive reliability models such as Musa’s Basic Execution Time (BET, see [MUS87]) and Everett’s Extended Execution Time (EET, see [EVE92]) models are used to model the reliability of each component. The parameters of these models can be estimated in terms of the properties of the software and the processing patterns it is subjected to. Thus component reliabilities can be estimated in terms of software and processing properties prior to the start of test. As test cases are developed, they are analyzed in terms of there processing impact on individual components. Also, estimates are developed of the processing impact on components during operational usage. The occurrence frequencies of test cases in operational usage (as specified in the operational profile (see [MUS93]) help in doing this). This information on processing patterns is used in combining component reliabilities to estimate software module and software system reliabilities.

As the reliability analysis is done prior to the start of testing, the output of such analysis can be used in:

• engineering the reliability of software during design
  
• developing a testing strategy to attain optimal reliability growth during test
  
• estimate the amount of testing needed to attain a specified level of reliability.
  

Also, the analysis does not assume that test cases must be executed in an order as specified by the operational profile, thus different orders can be selected and their effect on reliability growth investigated. When testing starts, modeling is used in a different way than current approaches. Rather than using failure data to fit models, the calibrated models from SCRA are used to create control charts. Failures observed during test are tracked on the control charts to determine whether the calibrated SCRA models explain the failures observed during test. When failure events fall outside control limits, this triggers the need to redo reliability analysis, using more information that becomes available during testing.