Prediction of Software Reliability bounds in new Operational
Environment
1. Introduction
The reliability of a software product depends on its quality and on the
application environment. The quality is presented by the existence of design
faults, while the application environment defines the activation conditions
of such faults.
The goal of the sketched study is the measured residual failure rate
of a product, demonstrated under a known operational environment (called
hereinafter test profile), to be used for reliability prediction when the
same product is implemented under another operational environment.
A faulty (defect) point of the input space is defined as one that
activates fault(s) when selected.
By definition the input space profile of a software product is given
by the distribution of the probabilities of occurrence of the input points
[Musa93].
Intensity profile is defined as the distribution of occurrence rates
of the input points.
2. Description of the method
The residual faulty points could be located everywhere in the input space. They could be grouped in “faulty regions”, each belonging to a design fault. The smallest faulty region consists of one input point.
The residual failure rate Lr under the old profile is presented as the sum
of the intensities of P faulty regions (1). The failure rate of a
hypothetical region i, which includes ni faulty points with approximately
equal intensities li, is given by (2). Finally, (3) represents the residual
failure rate formed by S faulty regions.
Since no information is available either about the real distribution
of faulty regions over the input space or about their sizes, the worst and
the best case are considered.
The basic assumptions are given bellow:
- Both old and new applications share the same input space range.
- The test is long enough the obtained value of the residual failure rate to be of high confidence,
- The fault activation is due to defect point selection only,
- Old and new intensity profiles are described up to single point intensity.
The worst and the best case conditions in terms of faulty points distribution
over the input space are described by (4) and (5) respectively:
For the sake of brevity we omit the best case related discussion
when possible.
The worst case reliability L*r max
is the maximum sum of S faulty regions' intensities. To find the
solution of (4) with respect to S and ni we can use the constraints
following from the knowledge of the old intensity profile -- (6) and (7).
The variable ni is constrained by the maximum value
Ni of the faulty points that could be grouped around the
point i (with intensity li)
if all the faulty points in the system are concentrated there.
Theorem 1 is proven that gives the solution (8) and (9) of (4) satisfying
(5) and (6) [KD97].
The best case solutions are presented in (10) and (11) and are based
on similar deductions - Theorem 2 [KD97].
To facilitate the solving efforts in the real systems we propose
two ways of reduction of the number of points to be evaluated. The first
one supposes that the whole input space is divided into equally sized
regions. The mean value of the occurrence rate of a point in region
i corresponds to li introduced in (2).
The worst case distribution of the faulty points with respect to
the residual failure rate in the new profile corresponds to that given by
(8) or (9) and the best case – by (10) and (11).
Following the approach of equal partitioning, the worst case requirements
could not be met due to the deviation of the points’ intensities inside
the arbitrary defined regions.
To closer satisfy the worst case conditions we suggest the regions’
determination to be tied to the shape of the old intensity profile. The
partitioning of the input space will then follow the exact maximum number
of faulty points that could be located inside the regions. This way the
input space is partitioned into regions, each region i characterized by
maximum of Ni faulty points. All points satisfying Lr/(Ni+1) < li < Lr/Ni belong to region i.
Once the regions are defined, the same solutions as those described
by (8), (9), (10) and (11) could be found. The better results obtained by
this equipotent partitioning are due to the avoidance of the mean
values of the occurrence rates in the regions.
3. Probabilities of been tested
After a certain testing period Tt the probability
pti(Tt) that a point has been tested differs for
different points. This could be used the maximum number of Ni
faulty points in a region to be corrected to mi such that
mi<Ni.
The mean value for mi out of Ni
not tested faulty points in region i is determined to redefine the
worst case of defect points distribution over the regions. An ordering is
formed (12) where the ratios are numbered in descending order, the maximum
ratio is numbered “0”. To find the worst case distribution (4)
has to be solved with respect to S and ni. A procedure
is given for determination of the distributions in case of limited number
of faults mi in a region. The procedure assigns n0=m0 points
to region “0” and calculates N1corr according to (13). If m1<N1corr
then n1=m1 and
the next step is entered. If m1>N1corr then n1=N1corr
and the procedure stops. In every step the same recalculation (13) is repeated
following the same rules. Theorem 3 shows the correctness of the described
procedure with respect to satisfying worst and best case conditions [KD97].
4. Discussion
The precision of the proposed method depends on the precision of the
residual failure rate determination and the input space profiles' identification.
The method could be applied the designers of a new application to
make a rough analysis whether or not the reuse of a software product meets
the reliability requirements.
Another application of the method could be in test planning. According
to the expression (9) the worst case residual failure rate in the new profile
depends on the maximum value of the ratio
(li*/li).
We can therefore construct a risk profile that will reveal the
most delicate parts of the input space with respect to the worst case distribution.
Definition of the most risky regions could be used for their intensive
tests, thus proving the absence of faulty inputs or removing the existing
ones. It has to be emphasized that the worst case region, in general, does
not coincide with the region of maximum intensity under the new environment.
So, the testing strategy might be adapted to cover the most critical regions
first but not the most intensively used.
Here, at least two cases are possible:
- Proving that the most critical regions are free of faults will shift the worst case conditions to the less critical regions, thus correcting the worst prediction according to the procedure presented in §3. Such type of selective testing could be stopped when the predicted worst case value reaches the required reliability.
- If some faults are removed, a precise analysis of their size could give information for the residual failure rate correction in the old operational profile, followed by a corresponding recalculation of the reliability worst and best cases under the new environment.
5. Conclusions
The proposed method gives an instrument the worst and the best case
failure rates to be predicted when a software product is reused in a new
intensity profile. The method utilizes the operational data collected during
the test in the old intensity profile and the knowledge of the new one.
It could be applied for early reliability estimation and for test planning
when the profile is changed.
References:
[Musa93] John D. Musa, “Operational Profiles in Software Reliability
Engineering”, IEEE Software, March 1993, pp.14-32
[KD97] Krassimir B. Djambazov, “Software Reliability in New
Operational Profiles”, Technical Report, Institute of Computer and
Communication Systems, Sofia, Bulgaria, December 1997, (in Bulgarian)
Contacts:
Prof. Krassimir B. Djambazov
Institute of Computer and Communication Systems, Bulgarian Academy
of Sciences
Acad. G. Bonchev St., bl. 2
1113 Sofia,
Bulgaria,
phone: +359 2 71 90 97
fax: +359 2 72 39 05
e-mail: kbd@iccs.acad.bg