CS302: Software Engineering (2021.B.01)

In this section the model presented by McCall et al. is used to explain quality factors which are commented upon in the light of developments in the late 1990s. You will see how some of the quality factors are still as fresh and as relevant today as they were in 1977. You will also see how some are redundant and have been integrated into other factors to better reflect modern practice. Under the heading of usability a whole new area of quality is taking shape and some of the topics in that domain are highlighted. Finally, you will see that each quality factor is not an isolated condition to be searched for and evaluated on its own and that in many cases there is an interplay and dependency between factors.

 

2.3.1 Efficiency 

The volume of code or computer resources (eg. time or external storage) needed for a program so that it can fulfil its function. 

McCall et al.

McCall's view of efficiency or performance is concerned with the efficient use of computer code to perform processes and the efficient use of storage resources. There are a number of techniques that can be used to achieve both of these objectives. Typical of these are: 

Programming languages. Selecting the most appropriate programming language for the problem has a major impact on program efficiency. For example, business applications which require substantial volumes of reports might best be programmed in COBOL while programs requiring substantial scientific calculations might be best accommodated by FORTRAN. 

Operating systems. Modern operating systems have the ability to perform multi-tasking thereby improving system performance by facilitating background operations.

Design. Strategies that address cohesion and coupling, normalisation techniques to reduce data redundancy and algorithms that optimise process time should always be employed. 

Access strategies. Algorithms that optimise seek time, rotational delay and data transfer time must be continuously searched out and implemented to improve efficiency. In particular cylinder concepts and hashing algorithms are most important. 

Programming techniques. Typical good programming techniques and practice like:

• Top-down design for complex problems 
• Sequence, selection and iteration constructs 
• Keeping local variables within procedures 
• Good use of parameter passing 
• Meaningful variable and procedure names 
• Proper documentation

The definition used by McCall inadvertently limits resource efficiency to external storage. Developers need to be particularly careful not to ignore, to the detriment of efficiency, modern working storage facilities like CPU RAM, Video RAM or printer RAM and external storage like CD-ROM.

Modern hardware has advanced considerably since 1977 so that efficiency now applies to a much broader set of resources, both internal and external. For example, the processing power of desktop computers plays a major role in software performance. Most readers would be aware of the performance of WINDOWS 3.1 on a '386 processor as compared with its performance on a '586 processor. While software was written for a perceived stable hardware environment in the late 1970s this is not so to-day. In the definition of quality given in section 2.1, a time dimension was included by way of the product's lifetime performance. Specifiers need to be conscious that in the current climate, efficiency is influenced by technological maturity too. 

Communications technologies are also part of modern efficiency. Fibre optic cables obviously support high speed data transfer rates, free of interference. 

Overall it would appear that the currency of the 1977 definition of efficiency has changed substantially with technological advancements. There are many additional considerations involved. 

 

2.3.2 Integrity 

The extent to which illegal access to the programs and data of a product can be controlled. 

McCall et al. 

This definition of integrity confines itself to user confidence that programs and data can not be altered illegally and the access controls that can be put in place to guard against this happening.

Unfortunately programs and data can also be altered innocently by authorised users making mistakes There is nothing illegal about this access and integrity controls must protect against that too. 

So, integrity has to concern itself with controls for preventing inaccurate data entering the system and detecting it if it does. It also has to concern itself with preventing changes to the software which compromise its original purpose.

French (1986) states that the aims of these controls are:

a. to ensure that all data are processed 
b. to preserve the integrity of maintained data 
c. to detect, correct and reprocess all errors 
d. to prevent and detect fraud"

He also suggests that there are five different types of control. Manual checks which are applied to documents before computer processing, data preparation controls, validation checks, batch controls and file controls. Only the last three of these can be built into the software to ensure integrity. 

Validation checks ensure that there are no empty fields where values are essential, datatypes are in keeping with those defined in the code and that data is within a reasonable range. Computer generated batch totals can be compared with manual batch totals to confirm valid data entry. File control totals are included at the end of each file and contain details like number of accounts and net totals.

In a database environment integrity takes on a special significance because of the many users accessing that data. Oxborrow (1986) explains that "the integrity of a database is maintained by means of validation rules which are applied to inputted and updated data, and concurrent access locks and rules which prevent concurrent transactions from interfering with each other". 

The validation rules suggested are the same as those in the traditional file processing environment. Mechanisms are coded into that software to check for data input errors where datatypes are incorrect or a maximum/minimum range is imposed on acceptable values. Reasonableness checks for gross errors are also included. Concurrency access control is necessary to prevent two or more users accessing the same data and processing it in different ways. The lost update and the deadlock problems are typical of what must be protected against. Read-locking and write-locking solutions need to be put in place. Minimum data redundancy in the database environment supports the concept of data integrity. Entity attributes that are stored more than once are likely to become inconsistent thereby compromising the integrity of the data. Database management systems incorporate many tools which support the easy inclusion of integrity features in database systems.

In safety-critical systems integrity implies that a safety specification must be written into the requirements specification. Validation checks at the data input stage are critical. There are many newspaper accounts of hospital cases where patients have received incorrect treatment as a result of invalid input.

 

2.3.3 Reliability 

The extent to which a program can be maintained so that it can fulfil its specific function. 

McCall et al. 

This definition seems to be more appropriate to a different quality factor called maintainability, which will be examined later. Reliability in engineering terms is the ability of a product or component to continue to perform its intended role over a period of time to predefined conditions. And the same applies to the systems environment where reliability is measured in terms of the mean time between failures, the mean time to repair, the mean time to recover, the probability of failure and the general availability of the system.

The mean time between failures - under predefined conditions, the average time between consecutive failures over a given period in the life of a system. 

The mean time to repair - the average time to repair or maintain equipment. 

The mean time to recover - the average time to return a system to operation after a failure. The time involved should include periods taken to reinstate from previous checkpoints. 

The probability of failure - the use of formal methods to predict the likelihood that a system will behave in an expected way under certain circumstances. Probability of failure is most appropriate to safety-critical systems and "continuous running" systems.

Authors don't appear to distinguish between hardware failure and software failure. Both need to be considered together when establishing system reliability. 

Even taking all these criteria into consideration systems reliability philosophy is very different to other branches of engineering and manufacturing. For example, Ghezzi et al. explain that "you do not take delivery of an automobile along with a list of shortcomings". Software on the other hand they point out comes with a disclaimer that the software manufacturer is not responsible for any damages due to software failure. 

Ince (1994) tells us that reliability is normally expected to be totally present in safety critical systems. But can it? Is the software engineering profession sufficiently mature to be able to predict and properly cope with instances like the Three Mile Island accident and the Chernobyl accident? See Bott et al. for an account of these disasters. Sommerville (1992) offers excellent advice on programming (fault avoidance, fault tolerance, exception handling and defensive programming) for reliability where the software is intended for use in high reliability situations.

The availability aspect of reliability is naturally a major concern in on-line and data-base systems. Many of these applications are essential to doing business - factory applications on the McFarlan and McKinney grid - so excessive downtime must be avoided. Furthermore, after a system has been down, users need to be certain that the restored system is accurate with no loss of data. Consequently, reliability has connections and implications for the accuracy and testability quality factors which are examples of the interplay between factors.

 

2.3.4 Usability

The cost/effort to learn and handle a product. 

McCall et al.

Usability is by its very name a quality factor in the HCI domain - usable by humans. To gain a proper understanding of McCall's perspective of usability in 1977, it is appropriate to recall the taxonomy of computers in those days and the purposes for which they were used. They were mainframe and mini computers running major DP applications. User staff were simply required to learn how to operate the system, input data, receive output and generally keep the system running. Software was developed to run on low specification monochrome monitors with simple combinations of green and black text. Software design strategies were generally file based. So it is easy to identify with McCall's definition, as usability was mainly confined to selected users learning how to use it. The era of desktop computing was only beginning. 

Usability in the late 1990s is concerned with a vast number of end-user issues and learning to use systems now embraces so many topics that some authors consider learnability to be a quality factor in its own right. Usability topics include suitability, learnability and adaptability.

Usability is an extremely difficult quality factor to define. The main reason for this is the vast variety of users with their different needs. Some will be novices while others will be experts and a quality product will support them all. 

Curson (1996) advises that "usability will be defined in the forthcoming ISO 9241 standard as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use". This definition suggests to me that the debate will still continue as to what exactly usability is because the definition is far too vague and omits the all important concept of user confidence. But more importantly, a definition that explains itself by re-using the term being defined is of now value at all. In this case usability is defined as the extent to which a product can be used. And what do we mean by used? Isn't that what we're trying to define? Finally the definition appears to ignore the need for a quality product to support all users no matter what their skill level.

In this paper, under the general heading of usability, topics are included that have been internationally discussed and researched. They are presented here under two headings: general ergonomics and software ergonomics. These heading are derived from the requirements of the EU directive on the minimum safety and health requirements for work with display screen equipment. 

General ergonomics is concerned with equipment and the work environment. While not essentially software issues they must be considered since many systems have failed because the factors involved were not properly addressed. Equipment is concerned with the selection of display screens, keyboards, work surfaces and work chairs. The environment like space requirements, lighting and distracting reflections should all be such that the user is as comfortable as possible. Noise, heat radiation and humidity are also considered as part of the desired environment. 

Software ergonomics is concerned with topics like how suitable is the software for the intended operations. How easy is it for users to learn and to master it. It embraces dialogue styles which vary from command line to Graphical User Interfaces. Human factors like perception, memory and human senses are included together with metaphors that aid these factors. Shneiderman's (1987) golden rules for good screen design are essential to software ergonomics. These include consistency of display, feedback and error messages, system performance, ability to reverse the users last action and similar facilities.

The whole subject of usability is very much a leading edge quality factor. Academic and industry ideas, independent standards and legal requirements are all concerned with the issue but unfortunately there seems to be no uniformity of definitions or vocabulary. Perhaps in the future this will be resolved. Sub-dividing it into new and more focused quality factors would seem appropriate. Meanwhile, an IS professional with an awareness of the topics mentioned here will be better equipped to specify, design, commission and evaluate software usability.

 

2.3.5 Accuracy 

The extent to which a program fulfils its specification. 

McCall et al.

Accuracy is a difficult factor to pin down because of the lack of standard terminology. It is easy to use the term interchangeably with other factors like reliability and integrity. Ince (1994) calls the factor correctness. Ghezzi et al. also prefer the term correctness and their definition is "A program is functionally correct if it behaves according to the specifications of the functions it should provide". This is often a "wish-list" quality factor because, for the most part, program specifications are seldom available except for very high budget and safety-critical systems. In these cases, it is economic to employ formal methods to confirm program accuracy. Otherwise, in as much as the various paths through a program can be measured, accuracy is confirmed through testing. Techniques include program inspections, mathematically-based verification, and static program analysis.  Accuracy or correctness can be achieved by using proven designs, algorithms, and reusable code.

 

2.3.6 Maintainability 

The cost of localising and correcting errors. 

McCall et al. 

Maintenance and what it means has changed considerably since 1977. Finding and correcting errors is just one aspect of maintenance. Ghezzi et al. (1991) divide maintenance into three categories: corrective, adaptive and perfective and only corrective is concerned with correcting errors as suggested by McCall.

Corrective maintenance is concerned with removing minor bugs left after development and testing are completed. This process is also involved after other maintenance activities. 

Adaptive maintenance is concerned with changing the software to reflect changes in the user's requirements. For example changes in VAT rates, income tax bands or income tax rates. Or, a user might wish to add more functionality. 

Perfective maintenance seeks to improve the algorithms used in the software to enhance performance. Perfective maintenance is often influenced by technological developments. 

Maintenance starts from the moment a system comes into operation and continues for the remainder of the product's life. For some products this can be twenty years or more. Maintainability is supported by good practices at all phases during the development life cycle. Typical of these practices includes:

A well defined methodology - A methodology like SSADM will guarantee that all aspects of fact finding and system modelling using DFDs and ERDs are completed and documented using standard techniques and standard forms. Some organisation might combine this approach with an end-to-end methodology which ensures that one consistent life-cycle approach applies. 

Good design techniques - Cohesion and coupling techniques ensure the modular construction of systems so that all three aspects of maintenance are easier to achieve. Well structured data using normalisation techniques will also mean a minimum of data updating thus maintaining the integrity of the maintained system. 

Attention to documentation - one of the most important life-cycle activities in the maintenance managers world. McCall's definition above is concerned with cost and a properly documented system will substantially reduce the cost of the corrective, adaptive and perfective functions. Offending code and its module can be easily located. Modules that are affected by change can also be easily identified and corrected. Without documentation the cost begins to rise and the maintainability of the product begins to fall. 

Good programming practice - meaningful names, readable code, sequence, selection and iteration structures only, single entrance and exit points to procedures and similar policies. Maintenance is also well supported by either re-using tried and tested code or by writing new code with reuse in mind.

Maintainability is another of the internal factors not visible or obvious to the client. It is therefore incumbent on the IS professionals involved to ensure that maintainability is properly addressed as part of the requirements specification. 

Finally, for the purpose of updating McCall's definition above we might define maintenance as 

"The non-operational costs associated with a product after a successful user acceptance test".

 

2.3.7 Testability

The cost of program testing for the purpose of safeguarding that the specific requirements are met. 

McCall et al.

Because of its traditional position in development models like the Waterfall or Boehm's spiral, testing is easily identified as a quality factor. The testing process is well matured at this stage. Sommerville (1992) suggests five stages - unit testing, module testing, subsystem testing, system testing and acceptance testing. However, most authors confine it to four stages and the model suggested in ISO 9000-3 names them as - item testing, integration testing, system testing and acceptance testing.

Item testing - standalone components are individually tested to ensure that they function properly. A substantial amount of item or unit testing is completed by programmers as part of their normal role. 

Integration testing - brings together standalone components into modules which are tested to reflect how they link in a new environment. Integration testing is also referred to as module testing. 

System testing - best performed as a full test run of the system that the client is about to receive but done without the client being present. It is the supplier's opportunity to confirm that the requirements specification has been fully achieved. 

Acceptance testing - the client running the new system to ensure that it complies with the original specifications. Acceptance testing is often referred to as Alpha testing.

Testing interacts with all other quality factors. For example, to check accuracy a test plan is needed. To test reliability a test plan is needed. To test efficiency a test plan is needed and so on. So all testing must be performed in accordance with predefined plans, using predefined tests data to achieve predetermined results. Numerous test strategies are used. They include functional or black box testing, structural or white box testing and finally residual defect estimation. These strategies can be employed using difference techniques. The options are top-down testing, bottom-up testing and stress testing.

Top-down testing - testing code that calls other, as yet unwritten, code by writing test stubs (empty routines) which can be called by the code being tested. A typical example would be testing a menu structure. Dummy stubs which represent the branched-to modules are used to test that the menu branching structure is working correctly. Later when the stubs are replaced by working modules, if things don't work properly, then the fault can be isolated to the new module and not the menu system. 

Bottom-up testing - this is the complement of top-down. In this case the modules or units are coded and unit tested, often by different developers at different sites at different times. All are eventually combined and the combining or calling structure is tested for defects.

Stress testing - involves testing systems which are designed to handle a specific load. For example, banking systems might be required to process a specified number of transactions per hour. Stress testing investigates whether the specified number of transactions can be achieved and what happens if they are exceeded. Does the system crash? Is data lost? And similar concerns. 

All of this testing has cost associated with it and testability is concerned with keeping this cost to a minimum. This can be achieved by using automated testing tools, through good cohesion and coupling design strategies and through good programming techniques. McCall et al. (1979) suggest a complexity matrix which involves number and size of modules, size of procedures, depth of nesting, number of errors per unit time and the number of alterations per unit time. 

McCall's definition of testability has stood the test of time and is still very much applicable today.

 

2.3.8 Flexibility 

The cost of product modification. 

McCall et al. 

This is McCall's second element of maintenance and over the years as the maintenance function has taken on new meaning the flexibility quality factor has been fused into maintenance. 

Recent interpretation of flexibility would be more associated with adaptability, ie. being able to change or reconfigure the user interface to suit users' preferences. This is a usability quality issue and is better considered in the usability section.

 

2.3.9 Interface facility 

The cost of connecting two products with one another. 

McCall et al.

Modern authors in keeping with the use of ...ility format have renamed this factor interoperability. This is the development strategy that encourages product development in a manner that it can interact with other products. For example, word processors that can incorporate charts from spreadsheets, or graphics from CAD and graphics packages, or data from databases would be considered to have high interoperability. Another example might be a payroll system - where staff are paid a commission on sales - being able to interface with a sales or order entry system. Interoperability is also the quality feature required to enable software products to interface with each other over a communications network.

Interoperability is very much dependent on an open systems policy being supported by software developers. An example of this in operation can be obtained from the Lotus Corporation (1993) white paper on Communications Architectures. In this document, Lotus advises customers that they, in conjunction with Apple, Borland, IBM, Novel and Work Perfect, are committed to a Vendor Independent Messaging (VIM) policy. They explain that they comply with industry and de facto standards such as x.400, SQL, OLE etc, and that their products like cc:Mail and Lotus Notes have open and published Application Programming Interfaces (APIs). These interfaces enable independent software vendors to develop applications which can interface with Lotus and their partners' products.

 

2.3.10 Re-usability 

The cost of transferring a module or program to another application. 

McCall et al. 

Re-usability addresses the concept of writing code so that it can be used more than once. A typical example is writing procedures or routines to receive variable parameters. The calling code passes the parameters and the called procedure processes them as appropriate. Any result from the process is returned to the calling code. The main advantage of using this approach is that once a procedure has been written and fully tested it can be used with full confidence in its accuracy thereafter. It should never need to be tested again.

Commercial libraries of reusable code are becoming available. The C language comes with a whole series of .lib files for use with the language thus saving developers the cost of writing their own solutions. Visual Basic is another instance of commercially available windows solutions. Object oriented development techniques also support re-usability. 

While re-usability is concerned with the cost of adapting code so that it can be reused in another application there is in the first place a cost of developing reusable code. It takes much longer and therefore costs more to develop reusable code. Quite often budgets and timescales work against the desire to develop reusable code. 

When writing code for reuse programmers should also be conscious of the economic use of code to comply with the efficiency quality factor.

This is another of the internal quality factors where client interests, particularly in relation to copyright and ownership, need to be carefully considered. Does specially written reusable code belong to the supplier or the customer and does code reused from a commercial library attract licence fees? All are items for the project specification and contract documents.

 

2.3.11 Transferability 

The cost of transferring a product from its hardware or operational environment to another. 

McCall et al. 

The modern expression for this quality factor is portability. Portability is the strategy of writing software to run on one operating system or hardware configuration while being conscious of how it might be refined with minimum effort to run on other operating systems and hardware platforms as well. Sommerville (1992) considers portability as a special case of software reuse "where the whole application system is reused by implementing it across a range of different computers and operating systems.

Portability is well supported by recent advances in standards development. Operating systems like UNIX and MS-DOS are well established and are being adhered to by developers. Programming languages also have agreed standards and languages like COBOL, C, FORTRAN and PASCAL are easily ported between systems using compilers that implement the agreed standards. Windows environments like X-Windows and WINDOWS have also imposed de facto standards on developers. Consequently products developed to comply with these standards should be portable to other environments with a minimum of conversion effort.