CS302: Software Engineering (2021.B.01)

Many CASE tools now support the automatic generation of test data for the specifications in their design products. There are also hundreds of different types of automated testing support tools that are not related to CASE. Some of the functions of these tools include

• static code analyzers 
• dynamic code analyzers 
• assertion generators and processors 
• test data generators 
• test driver 
• output comparators

In Table 17-10, several examples of CASE testing tools are presented. Many other types of testing support tools are available for use outside of a CASE environment. The most common test support tools are summarized below and sample products are listed in Table 17-11.

A code analyzer can range from simple to complex. In general, static code analyzers evaluate the syntax and executability of code without ever executing the code. They cross-reference all references to a line of code. Analyzers can determine code that is never executed, infinite loops, files that are only read once, data type errors, global, common, or parameter errors, and other common problems. Another output of some static analyzers is a cross-reference of all variables and the lines of code on which they are referenced. They are a useful tool, but they cannot determine the worth or reliability of the code which are desired functions. 

A special type of code analyzer audits code for compliance to standards and structured programming (or other) guidelines. Auditors can be customized by each using company to check their conventions for code structure.

A more complex type of code analyzer is a dynamic tool. Dynamic code analyzers run while the program is executing, hence the term dynamic. They can determine one or more of: coverage, tracing, tuning, timing, resource use, symbolic execution, and assertion checking. Coverage analysis of test data determines how much of the program is exercised by the set of test data. Tracing shows the execution path by statement of code. Some tools list values of key variables identified by the programmer. Languages on PCs usually have dynamic tracers as an execute option. Tuning analyzers identify the parts of the program executed most frequently, thus identifying code for tuning should a timing problem occur. Timing analysis reports CPU time used by a module or program. Resource usage software reports physicall/Os, CPU time, number of database transactions, and other hardware and software utilization. Symbolic executors run with symbolic, rather than real data, to identify the logic paths and computations for programmer-specified levels of coverage.

TABLE 17-10 CASE Test Tools

Tool Name	Vendor	Features and Functions
Teamwork	Cadre Technologies, Inc. Providence, RI	Testing Software
Telon and other products	Pansophic Systems, Inc. Lisle,IL	Code Generation, Test Management

An assertion is a statement of fact about the state of some entity. An assertion generator makes facts about the state the data in a program should be in, based on test data supplied by the programmer. If the assertions fail based on program performance, an error is generated. Assertion generators are useful testing tools for artificial intelligence programs and any program language with which a generator can work. Assertion checkers evaluate the truth of programmer-coded assertions within code. For instance, the statement 'Assert make-buy = o. " might be evaluated as true or false.

A test data generator (TDG) is a program that can generate any volume of data records based on programmer specifications. There are four kinds of test data generators: static, pathwise, data specification, and random. A static TDG requires programmer specification for the type, number, and data contents of each field. A simple static TDG, the IEBDG utility from IBM, generates letters or numbers in any number of fields with some specified number of records output. It is useful for generating volumes of test data for timing tests as long as the records contain mostly zeros and ones. Unless the test data generator is easy to use, it quickly becomes more cumbersome than self-made test data.

TABLE 17-11 Other Testing Support Tools

Tool Name	Vendor	Features and Functions
Assist		Coverage analysis, logic flow tracing, tracing, symbolic execution
Attest	University of Massachusetts Amherst, MA	Coverage analysis, test data generation, data flow analysis, automatic path selection, constraint analysis
Automatic Test Data Generator (ATDG)	TRW Systems, Inc. Redondo Beach, CA	Test data generation, path analysis, anomaly detection, variable analysis, constraint evaluation
Autoretest	TRW, Defense Systems Dept. Redondo Beach, CA	Comparator, test driver, test data management, automated comparison of test parameters
C/Spot/Run	Procase Corp. Santa Clara, CA	Syntax analysis, dependency analysis, source code Altering, source code navigation, graphical representation of function calls, error Altering

Tool Name	Vendor	Features and Functions
COBOL Optimizer Instrumentor Cotune	Softool Corp. Goleta, CA	COBOL testing, path flow tracing, tracing, tuning
		Coverage analysis, timing
Datamacs	Management & Computer Services, Inc. Valley Forge, PA	Test file generation, I/O specification analysis, file structure testing
DAVE	Leon Osterwell University of Colorado Boulder, CL	Static analyzer, diagnostics, data flow analysis, interface analysis, cross-reference, standards enforcer, documentation aid
DIFF	Software Consulting Services Allentown, PA	File comparison
FACOM and Fadebug	Fujitsu, Ltd.	Output comparator, anomaly detector
Fortran Optimizer Instrumentor	Softool Corp. Goleta, CA	Coverage analysis Fortran testing, path flow tracing, tracing, tuning
McCabe Tools	M. McCabe & Associates Columbia, MD	Specification analysis, visual path testing generates conditions for untested paths computes metrics
MicroFocus Cobol Workbench	MicroFocus Palo Alto, CA	Source navigation, interactive dynamic debugging, structure analysis, regression testing, tuning
Softool 80	Softool Corp. Goleta, CA	Coverage analysis, tuning, timing, tracing
UX-Metric	Quality Tools for Software Craftsmen Mulino, OR	Static analyzer, syntax checking, path analysis, tuning, volume testing, cyclic tests

Pathwise TDGs use input domain definitions to exercise specific paths in a program. These TDGs read the program code, create a representation of the control flow, select domain data to create representative input for a programmer-specified type of test, and execute the test. The possible programmer choices for test type include all feasible paths, statement coverage, or branch coverage. Since these are white-box techniques, unless a programmer is careful, a test can run for excessively long times.

Test drivers are software that simulate the execution of module tests. The tester writes code in the test driver language to provide for other module stubs, test data input, input/output parameters, files, messages, and global variable areas. The driver uses the test data input to execute the module. The other tester-defined items are used during the test to execute pieces of code without needing physical interfaces to any of the items. The major benefits of test drivers are the ease of developing regression test packages from the individual tests, and the forced standardization of test cases. The main problem with drivers is the need to learn another language to use the driver software.

On-line test drivers are of several types. Batch simulators generate transactions in batch-mode processing to simulate multi-user, on-line processing. Transaction simulators copy a test script as entered in single-user mode for later re-execution with other copied test scripts to simulate multi-user interactions.

Output comparators compare two files and identify differences. This makes checking of databases and large files less time-consuming than it would otherwise be.