CS302: Software Engineering (2021.B.01)

Black-box testing attends to process results as evidenced by data. The test item is treated as a black box whose logic is unknown. The approach is effective for single function modules and for high-level system testing. Three commonly used methods of black box testing are: 

• equivalence partitioning 
• boundary value analysis 
• error guessing

A fourth method that is less common in business, cause-effect graphing, is also used. Each of these methods are described in this section.

Equivalence Partitioning 

The goals for equivalence partitioning are to minimize the number of test cases over other methods and design test cases to be representative of sets of data. For the given level of test, the test item data inputs are divided into equivalent partitions each representing some set of data. Then, test cases are designed using data from each representative, equivalent set. The theory is that by exhaustively testing one item from each set, we can assume that all other equivalent items are also exhaustively tested.

For instance, at the module level, field values identify equivalent sets. If the field domain is a range of values, then one set is allowable values and the other set is disallowed values. The analysis to define equivalent domain sets continues for each data item in the input. 

Equivalence partitioning gains power when used at more abstract levels than fields, however. For instance, interactive programs, for integration tests, can be defined as equivalent sets at the screen, menu selection, or process levels. At the system test level, equivalence can be defined at the transaction, process, or activity level (from Information Engineering).

Test scripts for on-line applications can be black-box equivalence partitioning tools. A test script is an entry-by-entry description of interactive processing. A script identifies what the user enters, what the system displays in response, and what the user response to the system should be. How any of these entries, actions, and displays takes place is not tested.

Boundary Value Analysis 

Boundary value analysis is a stricter form of equivalence partitioning that uses boundary values rather than any value in an equivalent set. A boundary value is at the margin. For example, the domain for a month of the year ranges from one to 12. The boundary values are one and 12 for valid values, and zero and 13 for the invalid values. All four boundary values should be used in test cases. Boundary value analysis is most often used at the module level to define specific data items for testing.

Error Guessing 

Contrary to its name, error guessing is not a random guessing activity. Based on intuition and experience, it is easy for experts to test for many error conditions by guessing which are most likely to occur. For instance, dividing by zero, unless handled properly, causes abnormal ending of modules. If a module contains division, use a test that includes a zero divisor. Since it is based on intuition, error guessing is usually not effective in finding all errors, only the most common ones. If error guessing is used, it should always be used with some other strategy.

Cause-Effect Graphing 

One shortcoming of equivalence and boundary testing is that compound field interactions are not identified. Cause-effect analysis compensates for this shortcoming. A cause-effect graph depicts specific transformations and outputs as effects and identifies the input data causing those effects. The graphical notation identifies iteration, selection, Boolean, and equality conditions (see Figure 17-7). A diagram of the effects works backward to determine and graph all causes. Each circle on the diagram represents a sequence of instructions with no decision or control points. Each line on the diagram represents an equivalent class of data and the condition of its usage. When the graph is done, at least one valid and one invalid value for each equivalent set of data on the graph is translated into test case data. This is considered a black-box approach because it is concerned not with logic, but with testing data value differences and their effect on processing. An example cause-effect graph for Customer Create processing is shown in Figure 17-8.

Cause-effect graphing is a systematic way to create efficient tests. The trade-off is in time to develop the set of graphs for an application versus the time consumed executing large numbers of less efficient, possibly less inclusive test cases. The technique is used more in aerospace than in general business. 

Cause-effect graphs are more readily created from DFDs, PDFDs, and state-transition diagrams than from Booch diagrams even though it is particularly useful for real-time and embedded systems. Both types of systems use state-transition diagrams to show the causes and effects of processing. A cause-effect graph can be superimposed on a state-transition diagram or easily developed from the state-transition diagram. Cause-effect graphing can be used in place of white-box approaches whenever specific logic cannot be realistically tested because of combinatorial effects of multiple logic conditions.

FIGURE 17-7 Cause-Effect Graphical Notation