Chapter 2. The analysis and evaluation process
Techniques for acquiring needed information
Your analysis will rely on many kinds of information that can be gathered in a variety of ways. While most of the tools and techniques we identify later are associated with a particular phase of the analytical process, data gathering techniques can be used whenever they fit the situation. You can use survey research methods such as self-administered or telephone questionnaires to capture data about some characteristic, attitude, or opinions of users and stakeholders during the initial problem definition stage, during evaluation, or at any point in between. For example, interviews can be used to assess stakeholders at the beginning of the process, to gather information about best and current practices in the middle, and to evaluate a prototype near the end. Similarly, simulations and process mapping can be used to understand current processes, and to design or evaluate new ones. All of the following data-gathering techniques will be described more fully in the tools installment.
-
Library and document research - The purpose of these systematic searches of print and electronic sources is to identify, review, analyze, and evaluate the recorded experience and assessments of others who have dealt with a similar information problem, or who have used a technology similar to the one you are considering.
-
Internet research - Web searching is a specialized kind of literature review that involves using Internet search engines to locate pertinent information about a given topic. By entering various key words and phrases, you can search the Internet for Web sites, publications, listservs, and other resources about your topic. An important point to remember about the Web: contrary to much popular opinion, the Web doesn't provide you access to every worthwhile thing ever written - it only directs you to those things people have made available.
-
Self-administered questionnaires - When you need information from a large number of people or organizations, self-administered questionnaires are the right method. They allow you to collect structured data fairly quickly from many people in different locations. They usually employ short, simple questions and follow a careful sampling plan. (Link to a more detailed description of surveys in Part Two.)
-
Interviews - When you want to gather detailed information about people's impressions, experiences, ideas, and attitudes, interviews are often the best method. Face-to-face or telephone interviews yield rich information, and are good way to understand complex topics.
-
Experiments - Use an experiment to learn what influences the way some process or activity actually works. The data is typically a result of direct observation of behavior in a controlled situation. The natural setting involves a combination of many interacting influences that make it very difficult to sort out the independent effects of one factor or another. By contrast, an experiment is designed to control some factors so the impacts of specific ones can be assessed. Experimenting with new process models or user interfaces, for example, might provide a team with insights into their proposed changes.
-
Simulations - System simulations provide a structured approach to visualizing and understanding how complex social and managerial systems give rise to problem behaviors, as well as what types of solutions might be applied.
-
Process analysis - Business process maps (sometimes referred to as process models) are a graphical representation of the steps followed to achieve specific purpose, like hiring a new staff member or approving a permit application. Process models help you understand and communicate about a business process. They can be used to improve a process by evaluating, reorganizing, changing, or eliminating steps. They can also be used to track the flow of resources, such as information, through a process.
Some of these data gathering techniques, like Web searching and basic interviewing, are easily learned. Others, like experimental design or simulation, require considerable expertise. For these, you may want to consult with or hire an expert.
