Fast Abstracts Archives . .

SREPT: An Integrated Approach to Software Reliability Estimation and Prediction

S.S. Gokhale ^1*  S. Ramani ²  K.S. Trivedi ^{2 **}

 ¹Bourns College of Engg.

University of California, Riverside

 

²Dept. of Electrical Engg.

CACC, Duke University

Introduction and Motivation 

With the steadily growing power and reliability of hardware, software has been identified as a major stumbling block in achieving desired levels of system dependability. Various techniques have been proposed in the literature to evaluate the dependability of a software product as exemplified by its reliability, failure rate, number of faults residing in the software, and these include: classification of the software modules into fault-prone and non fault-prone categories and prediction of the number of faults in a software module based on software complexity metrics, reliability prediction based on failure data collected during the testing phase, and reliability prediction taking into account the architecture of the software and the failure behavior of the individual components which constitute the software.  One of the major problems which hinders the application of these techniques to real-life industrial settings is their availability in the form of systematic user-friendly tools. To address this issue, some of the available techniques have been encapsulated into tools. However, these tools are highly specialized in the approaches they implement, and the phase of the life-cycle in which they are applicable. For example, tools such as CASRE, SMERFS, SRE Toolkit, and SoRel [7] use failure data collected during the testing phase to predict various metrics of interest, whereas tools such as EMERALD [5] predict software quality based on software complexity metrics. Also, the recently developed architecture-based techniques which are gaining widespread attention due to the deployment of object-oriented paradigms in designing modern systems, are not available in the form of specialized tools.  In order to build reliable software systems, 
there is an increasing need to assess the quality and reliability of a software product throughout its life-cycle, and none of the existing specialized tools serve this purpose. In this paper we present the high-level architecture of a  Software Reliability Estimation and Prediction Tool (SREPT) which offers a unified framework for reliability estimation and prediction right from the architectural phase of the software to the operational phase. 

Architecture of SREPT

In this section we briefly describe the engines of SREPT in terms of the input data they accept for processing and the output they provide. The interested reader is referred to [3,8] for further study. Figure 1 shows the high level architecture of SREPT. 

In the post-development, pre-testing phase,  SREPT accepts software complexity metrics as input, and produces an estimate of the number of faults in each module using either the fault density approach [6] or regression tree modeling  technique [3]. 

During the testing phase, SREPT offers the user an option of doing analysis based on the failure data collected during testing using the enhanced non homogeneous Poisson process (ENHPP) model to predict the failure intensity, number of faults remaining, coverage and reliability [4]. When using the failure data to obtain reliability predictions, the ENHPP model uses four coverage functions, namely, exponential, Weibull, S-shaped, and log-logistic.  Exponential, Weibull, and S-shaped coverage functions correspond to Goel-Okumoto, Generalized Goel-Okumoto, and S-shaped software reliability growth models respectively, which belong to the class of finite failure non homogeneous Poisson process (NHPP) models [2].  Log-logistic coverage function was proposed to capture the increasing/decreasing nature of the failure occurrence rate per fault, which was observed during the analysis of some data sets, and the former three models were inadequate to capture this behavior. SREPT determines the ``optimal'' among these models subject to goodness-of-fit, bias and bias trend criterions [1].  The ENHPP model can also be driven by test coverage measurements obtained during testing and the estimate of the number of faults based on software complexity metrics.  SREPT thus offers a mechanism to combine software complexity metrics, failure data and coverage based approaches to reliability prediction via the ENHPP model.  Various optimization engines to compute release times of the software subject to various constraints such as maximizing reliability, minimizing cost etc. will also be a part of SREPT. 

Conventional software reliability models assume instantaneous and perfect debugging. SREPT allows the user to analyze the effect of these two factors on the residual number of faults in the software. Various metrics of interest such as the failure intensity and reliability can now be re-computed to reflect the time and resources expended in debugging to obtain realistic estimates. Optimal software release times can also be obtained based on various criteria taking into consideration debugging activities. 

SREPT is planned to accept the architecture of the application modeled either as a discrete time Markov chain (DTMC) or a continuous time Markov chain (CTMC), and the failure behavior of the individual components specified either as a probability of failure (or reliability), a constant failure rate or a time-dependent failure rate.  Analytical modeling techniques rely heavily on the use of SHARPE [9]  for obtaining reliability and performance predictions.   

Implementation of SREPT 

SREPT is being implemented using JAVA, which is portable, robust and architecture neutral. In addition, it satisfies the need for an actual ``programming language'' while providing an extensive library for GUI building.   

References 

[1]  A.A. Abdel-Ghally, P.Y. Chan, and B.Littlewood.  "Evaluation of Competing Software Reliability Predictions,"  IEEE Trans. on Software Engineering, SE-12(9):53 8-546, September 1989. 
 
[2]  W.Farr.  Handbook of Software Reliability Engineering, M. R. Lyu, Editor, chapter Software Reliability Modeling Survey, pages 71--117. McGraw-Hill, New York, NY, 1996. 
 
[3]  S.Gokhale.  "Analysis of Software Reliability and Performance," PhD thesis, Duke University, Durham, NC, June 1998. 

[4]  S.Gokhale, T.Philip, P.N. Marinos, and K.S. Trivedi.  "Unification of Finite Failure Non-Homogeneous Poisson Process Model through Test Coverage,"  In Proc. Intl. Sypmosium on Software Reliability Engineering (ISSRE '96), pages 289-299, White Plains, NY, October 1996. 

[5]  J.P. Hudeophl, S.J. Aud, T.M. Khoshgoftaar, E.B. Allen, and J.Mayrand.  "Emerald: Software Metrics and Models on the Desktop,"  IEEE Software, 13(5):56-61, July 1996. 

[6]  M.Lipow,  "Number of Faults per Line of Code,"  IEEE Trans. on Software Engineering, SE-8(4):437-439, July 1982. 

[7]  M.R. Lyu,  Handbook of Software Reliability Engineering, McGraw-Hill, New York, 1996. 

[8]  S.Ramani, S.Gokhale, and K.S. Trivedi,  "SREPT: Software Reliability Estimation and Prediction Tool,"  In 10th International Conference on Modelling Techniques and Tools for Computer Performance Evaluation - Performance Tools'98, Palma de  Mallorca, Spain, September 1998. 

[9]  R.A. Sahner, K.S. Trivedi, and A.Puliafito,  Performance and Reliability Analysis of Computer Systems: An Example-Based Approach Using the SHARPE Software Package,  Kluwer Academic Publishers, Boston, 1996.

*  This work was done when the author was a graduate student at Duke University. 

2. S. Ramani and K.S. Trivedi, Dept. of Electrical Engg., CACC, Duke University, Durham, NC 27708.  Phone (919) 660 5269. 
E-mail: {sramani,kst}@ee.duke.edu 

**  Supported by a contract from Charles Stark Draper Laboratory and in part by Bellcore as a core project in the Center for Advanced Computing and Communication.