Fast Abstracts Archives . .

Abstract

Literate programming merges the implementation code and its documentation within one document. The programmer should view his implementation as a piece of literature which main goal is to tell the reader, what the author thinks that his implementation does (as opposed to try to tell the processor what the author wants the program to do). Cweb is a tool for literate programming with the C-Language. The documentation language is TEX resp. L^ATEX. But it is not enough to write well documented programs. The programmer must be enabled to test his programs as well.

In this paper, we present a prototype Test-system based on the Cweb system which has been developed as part of a fault tolerant system performing distributed computations on networks of workstations [7,8]. The Test-description, based on a test description language, is integrated into the documentation part of the Cweb program. The here presented system extracts the Test-description from the Cweb program, generates Test-programs, performs the defined tests and provides the programmer with a description of the results and the test coverage.

The main purpose of the system is to provide the programmer with a test system being used, while developing the code. These tests may be reused while testing, enhancing and maintaining so build systems.

1. Cweb and writing tests

In our experience, Cweb [1] is well-suited to implement and document a system with the C Language. It enables the programmer to put a hierarchical structured view on his programs. Each element of this view, so called refinements, may be documented in TEXor L^ATEX After finishing editing the Cweb-file a preprocessor (ctangle) translates the CWeb source into a C-Program which may be fed to a C-Compiler in turn. Another preprocessor (cweave) is able to translate the CWeb-source into a L^ATEXdocument from which e.g. a postscript file may be generated.

But implementing and documenting is not the only thing to be done while building a system. Code changes over the time, bugs are fixed and code is ported to other architectures. Needless to say that some code being originally implemented on one architecture will not work on another one or even worse, may not behave as expected. The same condition holds for bug fixes, which will make the system work on one architecture but may introduce some more errors on another architecture. It is hard work to find out what the error is and why it appears. This field asked for more tools to be integrated into the development environment.

In the first step, we wrote a tool called spectest [9], which is able to generate a test program given the code of a C function and a test description (Figure 1).

  

The test description is written in a test description language, defining the test case by its preconditions, the tests to be performed, stubs to use, shell actions to perform and the expected results (post conditions) to be reached. The elements of the test description language will be explained later. In the next step we integrated [3] this tool together with gct [5], Expect [2], g++ [10] and our development environment [9]. Figure 1 shows the different steps while performing the test. The test-descriptions, being part of the documentation parts of the Cweb program, are extracted and put into a test description file. The spectest tool generates a C-File from which a test-program is generated. The test-program will be instrumented using gct [4] to get the coverage for the test (UNIT-TEST only). The coverage could be used to improve the quality of the tests. While performing the tests, the generated test-program communicates with a test-driver, which schedules additional actions like calling shell scripts. After the tests are performed the result, log and coverage of the test are printed into different files.

The main advantage of our spectest tool in comparison to other test tools in this area, e.g. deja-gnu [6], is the possibility to perform the tests on the basis of a refinement instead of being able only to test a main program. As the smallest refinement is one line of C-Code the smallest unit we are able to test is one line of C-Code.

2. The Test-description language

The elements of the Test-description language are given in figure 2. Multiple tests can be performed with one test description. It is possible to define stubs for functions being called from the tested functions in order to investigate the tested function in isolation (unit test). It is also possible to call the ``real'' function and thereby making an integration test.

  

The Testtyp elements define if we are testing a function or library as Unit- or Integration test. In case of a Unit-test, different stub-functions (Stubs) are distinguished. Within a test-type global variables GLOBALS and special header-files HEADER to use can be defined. Multiple TESTCASEs could be performed. Within a Test-case values may be ASSIGNed to variables and after the test has been EXECUTEd the result will be CHECKed. Within these sections C-statements are used e.g. to assign a value to a variable. Additionally it is possible to trigger shell actions while performing the tests. If after the assignment of the variables on C-level a shell action should be triggered, this could be implemented within the ONASSIGN block. The used language is bourne shell. The IO statements are used to provide the test with data on STDIN and check the expected output on STDOUT and STDERR. The test itself could be documented in the SPEC block using L^ATEX as documentation language.

3. Availability

The here described Test-environment is part of the LIPS Development system of which an Alpha Release for Solaris 2.5.1 exists. See http://www.cdc.informatik.tu-darmstadt.de/LiPS for additional information.

References

 

1 Knuth D.E. and Levy S. The CWEB System of Structured Documentation. Addison Wesley, version 3.0 edition, 1994.

 

2 D. Libes. Exploring Expect: A Tcl-based Toolkit for Automating Interactive Programs. O'Reilly & Associates, Inc., 981 Chestnut Street, Newton, MA 02164, USA, Dec. 1994.

 

3 Lippmann J. Integration einer Testumgebung in LIPS. Diplomarbeit, Universität des Saarlandes, Lehrstuhl Prof. Buchmann, 1997.

 

4 B. Marick. GCT User's Guide. Testing Foundations, 1992.

 

5 B. Marick. The Craft of Software Testing. Prentice Hall, 1995.

 

6 R. Savoye. The DejaGNU Testing Framework. Free Software Foundation, http://www.cygnus.com edition, 1 1996.

 

7 Setz T. Design, Implementation and Performance of a Fault Tolerant Tuple Space Machine. In Proceedings: ICPADS'97: 1997 International Conference on Parallel and Distributed Systems, December 10-13, 1997, Seoul, Korea. IEEE, 12 1997.

 

8 Setz T. and Liefke T. The LIPS Runtime Systems based on Fault-Tolerant Tuple Space Machines. In Proceedings of the Workshop on Runtime Systems for Parallel Programming (RTSPP), 11th International Parallel Processing Symposium (IPPS'97), Geneva, Switzerland, April 1997. Appeared as Technical Report, Vrije Universiteit Amsterdam, Faculteit der Wiskunde en Informatica, No. IR-417, February 1997.

 

9 Setz T., Tews M., and et al. The LIPS Development System, 10 1994. Universität des Saarlandes, Fachbereich Informatik, Lehrstuhl Prof. Buchmann.

 

10 Stallman R. M. Using and Porting gcc. Free Software Foundation, 1994.