CS302: Software Engineering

There are seven techniques we use for data gathering during application development. They are interviews, group meetings, observation, temporary job assignment, questionnaires, review of internal and outside documents, and review of software. Each has a use for which it is best served, and each has limitations to the amount and type of information that can be got from the technique. The technique strengths and weaknesses are summarized in Table 4-2, which is referenced throughout this section. 

In general, you always want to validate the information received from any source through triangulation. Triangulation is obtaining the same information from multiple sources. You might ask the same question in several interviews, compare questionnaire responses to each item, or check in-house and external documents for similar information. When a discrepancy is found, you reverify it with the original and triangulated sources as much as possible. If the information is critical to the application being correctly developed, put the definitions, explanations, or other information in writing and have it approved by the users separately from the other documentation. Next, we discuss each data collection technique.

TABLE 4-2 Summary of Data Collection Techniques

TABLE 4-2 Summary of Data Collection Techniques (Continued)

Interviews use two types of questions: open-ended and closed-ended. An open-ended question is one that asks for a multi-sentence response. Open-ended questions are good for eliciting descriptions of current and proposed application functions, and for identifying feelings, opinions, and expectations about a proposed application. They can also be used to obtain any lengthy or explanatory answers. An example of open-ended question openings are: "Can you tell me about ... " or "What do you think about ... " or "Can you describe how you use ... ".

A closed-ended question is one which asks for a yes/no or specific answer. Closed-ended questions are good for eliciting factual information or forcing people to take a position on a sensitive issue. An example of a closed-ended question is: "Do you use the monthly report?" A 'yes' response might be followed by an open-ended question, "Can you explain how?"

The questions can be ordered in such a way that the interview might be structured or unstructured (see Table 4-4). A structured interview is one in which the interviewer has an agenda of items to cover, specific questions to ask, and specific information desired. A mix of open and closed questions is used to elicit details of interest. For instance, the interview might start with "Describe the current rental process." The respondent would describe the process, most often using general terms. The interviewer might then ask specific questions, such as, "What is the daily volume of rentals?" Each structured interview is basically the same because the same questions are asked in the same sequence. Tallying the responses is fairly easy because of the structure.

An unstructured interview is one in which the interview unfolds and is directed by responses of the interviewee. The questions tend to be mostly open-ended. There is no set agenda, so the interviewer, who knows the information desired, uses the responses from the open-ended questions to develop ever more specific questions about the topics. The same questions used above as examples for the structured interview might also be used in an unstructured interview; the difference is that above, they are determined as a 'script' in advance. In an unstructured situation, the questions flow from the conversation.

TABLE 4-4 Comparison of Structured and Unstructured Interviews

Structured interviews are most useful when you know the information desired in advance of the interview (see Table 4-4). Conversely, unstructured interviews are most useful when you cannot anticipate the topics or specific outcome. A typical series of interviews with a user client begins with unstructured interviews to give you an understanding of the problem domain. The interviews get progressively structured and focused as the information you need to complete the analysis also gets more specific.

User interview results should always be communicated back to the interviewee in a short period of time. The interviewee should be given a deadline for their review. If the person and/or information are critical to the application design being correct, you should ask for comments even after the deadline is missed. If the person is not key in the development, the deadline date signifies a period during which you will accept changes, after the date you continue work, assuming the information is correct. 

 It is good practice to develop diagram( s) as part of the interview documentation. At the beginning of the next interview session, you discuss the diagram(s) with the user and give him or her any written notes to verify at a later time. You get immediate feedback on the accuracy of the graphic and your understanding of the application. The benefits of this approach are both technical and psychological. From a technical perspective, you are constantly verifying what you have been told. By the time the analysis is complete, both you and the client have confidence that the depicted application processing is correct and complete. From a psychological perspective, you increase user confidence in your analytical ability by demonstrating your problem understanding. Each time you improve the diagram and deepen the analysis, you also increase user confidence that you will build an application that answers his or her needs.

Interviews are useful for obtaining both qualitative and quantitative information (see Table 4-2). The types of qualitative information are opinions, beliefs, attitudes, policies, and narrative descriptions. The types of quantitative information include frequencies, numbers, and quantities of items to be tracked or used in the application. 

Interviews, and other forms of data collection, can give you misleading, inaccurate, politically motivated, or irrelevant information (see Table 4-2). You need to learn to read the person's body language and behavior to decide on further needs for the same information. Table 4-5 lists respondent behaviors you might see in an interview and the actions you might take in dealing with the behaviors.

For instance, if you suspect the interviewee of lying or 'selectively remembering' information, try to cross-check the answers with other, more reliable sources. If the interview information is found to be false, ask the interviewee to please explain the differences between his or her answers and the other information. The session does not need to be a confrontation, rather, it is a simple request for explanation. Be careful not to accuse or condemn, simply try to get the correct information.

Persistence and triangulation are key to getting complete, accurate information. You are not required to become 'friends' with the application users, but interviews are smoother, yield more information for the time spent, and usually have less' game-playing' if you are 'friendly' than if you are viewed as distant, overly-objective, or uninterested.

Source: Sue Conger, https://resources.saylor.org/CS/CS302/OER/The_New_Software_Engineering.pdf
This work is licensed under a Creative Commons Attribution 3.0 License.