In this chapter, we present an analytical process that can be used in any project that applies IT to a service delivery or administrative goal. The process begins by defining a problem or purpose within its environmental context. It then goes on to identify and test possible solutions. The third phase focuses on the evaluation of alternatives and the selection of a preferred approach. For simplicity, we present these phases as sequential steps, but in action they are often iterative. What you learn in one phase may prompt you to return to an earlier one to refine your thinking before moving forward again.

Although this process of feedback and learning takes extra time, it is critical to comprehensive understanding, and it allows you to build a business case that reflects all the essential risks and options. The final steps described in chapters three and four are the preparation and presentation of that business case. Each phase builds on the perspective gained from earlier ones. The result is a multi-faceted analysis of a proposed project that has a high likelihood of accurately predicting real costs and outcomes. Figure 1 illustrates the entire process.

First things first: Choose a "good" problem

The research, testing, and evaluation that go into this analysis are labor intensive processes. They can be applied to any problem, but not every problem deserves this level of attention. This is a process that is worthwhile for complex, mission-critical, information-intensive problems. We call these "good" problems because their solution has high positive impact. For these problems, you need to pay close attention to the political, economic, legal, and organizational environments as well as the technologies involved. Not every IT decision has these characteristics or warrants the investment of resources that this process entails. Here are some administrative problems and service delivery questions that have successfully used this process:

• How effective are current service packages in helping homeless families achieve independence?
• How can the complete history of relationships between a state agency and local governments be captured for ongoing use in current and future technical assistance?
• How can information about the health and well being of children be made readily available to professionals and individuals concerned with the design of children's service programs?

The first question represents an information sharing and program evaluation problem. The homeless services system involves not only state agencies, but local governments, and scores of nonprofit shelter and service providers, all with different kinds of information maintained in different systems and formats. These organizations also respond to different funding streams and to a variety of legal requirements.

The second problem involves the central office of a major administrative agency and its regional offices distributed around the state. Each regional office is responsible for monitoring and advising local governments in its geographic area on their financial affairs. Each region collects information and does things in its own way. There is no statewide repository of information that can provide context, history, best practices, or overall performance information about this important aspect of municipal government.

The third example does not focus on operational concerns, but on the availability and usability of statistical information about children. In this case, the information comes from 13 state agencies, is compiled once a year into a printed book by a central coordinating agency, and is used by hundreds of municipalities, nonprofit service agencies, and research organizations. The data is collected according to different time periods, using different definitions, and covering different geographic distributions.

Despite their differences, these problems have some similarities. First, they are mission critical to the agencies that sponsor them. The homeless services project is deeply embedded in the core services and values of all involved organizations. The financial health of municipalities is one of a handful of overriding mission goals for the administrative agency. And a visible public focus on the needs of children is the entire reason for the existence of the coordinating agency.

Second, these problems are information-intensive situations. The solution to each problem depends, in large part, on the quality, timeliness, and accessibility of information. In most cases, some information systems are already in place and need to be taken into account in any new approach.

Third, they all exist in an environment of high complexity. Both the homelessness and children's data projects involve many independent organizations, each with its own practices, values, and rules. Similarly, the municipal affairs project covers the whole state and must deal with local diversity ranging from huge sophisticated cities to tiny rural towns and villages. All three projects must deal with public opinion, public budgeting and legislative cycles, and legal requirements. Civil service rules circumscribe staffing assignments and compensation. Organizational rules, traditions, and structures set boundaries. Many different business processes already in place will need to be understood, and may need to be changed.

In projects with these characteristics, opportunities abound for wrong assumptions, premature decisions, and dangerous oversimplification. This kind of complexity seems overwhelming, and a common reaction is to try to cut through to the part of the project that is more concrete and manageable - the technology. This is almost always a mistake. Projects like these demand a careful analysis that works through and manages the complexity at every level, from the larger environment, to the organizational considerations, to the work processes and data needs, to the technology choices.

For projects like those described above, the analytical process is not a solitary process. These projects involve many people in different organizations or organizational units. One person may lead an effort, or might collect, organize, and present information, but groups of people will inevitably become involved and group processes will be needed.

Government information technology projects can involve dozens of people. Individuals with vastly different work styles, backgrounds, and talents are often brought together, asked to form a cohesive group, and charged with solving a problem. Often people from different organizations need to work together to plan and implement a project. But their individual differences, and group dynamics, can make it difficult for the group to reach its goals. Consensus-building tools and skilled group facilitation can be very helpful in guiding a group through the steps necessary to make effective decisions.

Consensus-finding and -building tools are often needed to help a group resolve different views and conflicting objectives or interests. Groups also frequently need to be introduced to models for collaboration, especially if they've never worked together before. Effective teamwork may also involve difficult trade-offs and other choices, so some decision-making tools and techniques can be useful. Group processes take skill and time to work effectively, but they result in well-documented and well-understood decisions that can then guide the work group to a successful outcome. Tools and techniques for all of these topics are described in Part Two.

Your analysis will rely on many kinds of information that can be gathered in a variety of ways. While most of the tools and techniques we identify later are associated with a particular phase of the analytical process, data gathering techniques can be used whenever they fit the situation. You can use survey research methods such as self-administered or telephone questionnaires to capture data about some characteristic, attitude, or opinions of users and stakeholders during the initial problem definition stage, during evaluation, or at any point in between. For example, interviews can be used to assess stakeholders at the beginning of the process, to gather information about best and current practices in the middle, and to evaluate a prototype near the end. Similarly, simulations and process mapping can be used to understand current processes, and to design or evaluate new ones. All of the following data-gathering techniques will be described more fully in the tools installment.

• Library and document research - The purpose of these systematic searches of print and electronic sources is to identify, review, analyze, and evaluate the recorded experience and assessments of others who have dealt with a similar information problem, or who have used a technology similar to the one you are considering.
• Internet research - Web searching is a specialized kind of literature review that involves using Internet search engines to locate pertinent information about a given topic. By entering various key words and phrases, you can search the Internet for Web sites, publications, listservs, and other resources about your topic. An important point to remember about the Web: contrary to much popular opinion, the Web doesn't provide you access to every worthwhile thing ever written - it only directs you to those things people have made available.
• Self-administered questionnaires - When you need information from a large number of people or organizations, self-administered questionnaires are the right method. They allow you to collect structured data fairly quickly from many people in different locations. They usually employ short, simple questions and follow a careful sampling plan. (Link to a more detailed description of surveys in Part Two.)
• Interviews - When you want to gather detailed information about people's impressions, experiences, ideas, and attitudes, interviews are often the best method. Face-to-face or telephone interviews yield rich information, and are good way to understand complex topics.
• Experiments - Use an experiment to learn what influences the way some process or activity actually works. The data is typically a result of direct observation of behavior in a controlled situation. The natural setting involves a combination of many interacting influences that make it very difficult to sort out the independent effects of one factor or another. By contrast, an experiment is designed to control some factors so the impacts of specific ones can be assessed. Experimenting with new process models or user interfaces, for example, might provide a team with insights into their proposed changes.
• Simulations - System simulations provide a structured approach to visualizing and understanding how complex social and managerial systems give rise to problem behaviors, as well as what types of solutions might be applied.
• Process analysis - Business process maps (sometimes referred to as process models) are a graphical representation of the steps followed to achieve specific purpose, like hiring a new staff member or approving a permit application. Process models help you understand and communicate about a business process. They can be used to improve a process by evaluating, reorganizing, changing, or eliminating steps. They can also be used to track the flow of resources, such as information, through a process.

Some of these data gathering techniques, like Web searching and basic interviewing, are easily learned. Others, like experimental design or simulation, require considerable expertise. For these, you may want to consult with or hire an expert.

Having identified a significant need or problem, the first phase of the analysis is to understand it as fully as possible in the context in which it occurs. Three kinds of work help you reach this depth of understanding: specifying objectives, identifying and assessing the influence of all stakeholders, and then analyzing the need or problem in detail. A more detailed description of the tools that can help gain a greater understanding of the problem and its context can be found in Part Two.

The first kind of work leads you and the others involved to a clear, unambiguous, shared understanding of the business or program objective you want to achieve. This sounds simple, but in practice is often very difficult. Many projects go wrong at this very first step because those responsible assume everyone sees the situation and its resolution in the same way. This is almost never the case. Even simple programs or processes can be approached from different points of view. One participant may see a service that could be more accessible to its customers, another can look at the same service and want to reduce the cost and effort to deliver it, and still another may take an evaluator's perspective and ask what value it delivers to society.

Specify objectives

Several tools can be used to work through this essential first step of setting objectives. One simple tool is called a hopes and fears exercise in which the members of a work group individually state what they hope the project will accomplish - and what they are afraid might happen instead or as a consequence. These individual statements can be grouped into themes that often reveal multiple, competing, and sometimes conflicting objectives. Once specified in this public way, the group can work toward the specification of a shared statement of objectives. The public statement of fears works in a similar way. It represents early indications of problems that are likely to be encountered along the way.

Other tools such as visioning, service objective and strategic framework are different approaches to the same goal - a clearly specified, unambiguous, agreed-upon objective.

Identify and assess stakeholders

The next step in this phase is careful identification of all stakeholders and the ways they can influence or be affected by the project. This can be done with a stakeholder analysis exercise. Many projects limit stakeholder considerations to those who are directly involved in the development of a system. Generally, this is not enough. Those who are indirectly affected count, too. Often, this kind of analysis does a good job of identifying positive effects such as who saves time or money, but they often ignore the negative effects such as who gets lower priority or picks up the cost of making things cheaper for someone else. Stakeholder influences are also critical. Some stakeholders are essential because of their legitimacy or power regardless of how active they are in a project.

A positioning chart is a good first stakeholder analysis tool. The chart places stakeholders on a two-dimensional grid that shows both the degree to which they support or oppose the project and their importance to its success. Those stakeholders who are important to success, whether they are supporters or opponents, deserve careful scrutiny and a clear-cut plan of action. A partisan analysis can be used to uncover potential conflicts and competing interests among stakeholders.

Analyze the problem and the process

The last work in this phase is where you begin to tackle the problem itself. Often the problem or objective is embedded in at least one work process. Process analysis is therefore a good way to delve into the details, understand the bottlenecks, see the handoffs, and identify where information is added or recorded. A process map that shows this in sufficient detail is then ready for preliminary process improvement ideas where the work team identifies ways to streamline or get more value from it.

Other problems are not process oriented so they demand different tools. Self-assessment tools can help gather information about the current situation. Models of the problems can help to make your situation more explicit. Sometimes you need customer satisfaction information and may want to conduct interviews or a survey. Statistical data about some aspect of performance or cost may be needed. The important point is to analyze the problem fully, using the tools at your disposal. You will often be surprised by unexpected findings and new ways to understand cause and effect.

Phase 1 arms you with a very detailed understanding of your objective and the context in which you must try to achieve it. In Phase 2, the work turns to a search for reasonable solutions that you can investigate and test. This phase makes substantial use of the experiences of others who have tackled similar goals. It also leads you to develop alternative solutions and offers some ways to try them out in low-cost, low-risk ways. You are not building a system yet-but you are building a great deal of knowledge about the system that you will eventually construct. A more detailed description of the tools that can help identify and test possible solutions are available in Part Two.

No matter how unique your problem may seem, some other organization has probably faced it. That organization may be another agency in your own state; it could be a government somewhere else in the US or the world; it could be a commercial or nonprofit organization. It might be much bigger or much smaller than your organization, or engaged in a very different kind of work. Regardless of their settings, you can learn from them. The first part of the work in Phase 2 is the search for relevant practices, tools, and techniques whose use in other places can teach you something that you don't have to learn on your own.

Find relevant practices, tools, and technques

You can learn from the experiences of others by using a variety of research tools. Best and current practice research is a good general-purpose tool that any professional can use effectively. You can conduct this kind of research in the library, on the Internet, at professional meetings, in face-to-face interviews, and on the phone. The key is to specify, as clearly and narrowly as you can, the questions you want to answer, and then to evaluate critically what you read and hear.

One benefit of working in the public sector is the general willingness of public managers to share their good and bad experiences with one another. You can often learn more by picking up the phone and calling a contact person than by just reading a story in a newsletter or Web site. Probe for the complete story of how a project unfolded or a technique was used. Ask about critical success factors and ask also what should have been done differently.

Know your environment

An environment scan can help determine what is happening in the overall environment that may have an impact on the project. The results of this kind of review may lead you to reconsider some aspects of your own service or business objective. You may want to return to and refine your focus before moving forward to other parts of the analysis. You might also discover other stakeholders need to be considered or that some part of your problem analysis needs more attention. Take advantage of these early opportunities to improve your fundamental analysis - they are more valuable and less costly now than they will be later in the project.

Benchmarking and building technology awareness through trade shows, demos, and vendor presentations are other good ways to collect current information that you then evaluate for comparability, completeness, and relevance to your own situation. These investigations may help you understand the features and applications of technologies you are considering for your project. They may also help you understand the broader infrastructure needs you will have to take into account.

Identify modest, moderate, and elaborate solutions

Phase 2 is completed by specifying several alternative approaches or solutions that seem to fit well with the context, problem, and stakeholder analyses from Phase 1 as well as the information you gathered and evaluated from other places at the start of Phase 2. At this point, your work group may have several ideas to pursue. You should specify them all in similar ways so that you can compare them. Models of solutions can be built to can help minimize risks and get all the potential development costs on paper.

A description of features and functionality at modest, moderate, and elaborate levels of investment is one way to do this. First, list the key elements of a solution. These will vary according to your objective, but some common elements are the means of customer access, response time, degree of customization, level of security, extent of manual data handling, and degree of integration with other processes or systems. The next step involves describing how a minimally useful solution would address each of these elements. This is the "modest" solution or the one that accomplishes the least worth doing. Then specify how a "moderate" solution would address each element. This one offers greater functionality, more convenience, or other improvements over the modest level. Third, list how an "elaborate" solution would address each element. This is the most advanced solution that an organization might attempt. Finally, for all three alternative solutions, state the benefits and who reaps them. Benefits may be quantifiable as dollar or time savings. We think of these as the "cheaper" or "faster" benefits. Another category of benefits might be categorized as "better." These benefits come from qualitative changes such as improvements in service quality or availability.

Prototype when possible

Testing your alternatives may start by gathering and comparing performance data from existing projects that are operating elsewhere. If all the alternatives are operationally feasible, then testing the concept with appropriate stakeholders may be sufficient. If the proposals involve some real unknowns, then try to create test conditions that are as close to real life as possible, such as building one or more prototype systems tested under field conditions in a controlled experiment. Also remember that the more costly the solution (in dollars, effort, and change to the status quo) the more reason you have to conduct a life-like field test.

One surprising result of these tests may be the extent to which non-technology solutions fill your needs. Often significant improvements in business processes or information flow go a long way toward meeting your objectives, even without the application of new technologies. In one agency we worked with, $3 million had been set aside for an imaging system to improve customer service transactions. After successfully building and testing a prototype that reflected completely revamped business processes, the agency decided not to spend the money on new technology. Why? The prototype effort showed that most of the benefits could be obtained by process improvements alone.

The final analysis of alternatives is mainly an opportunity to integrate and compare information. Your work group has looked at the problem and potential solutions from many points of view using several tools. Now is the time to step back and ask what the entire body of data has to say. What are the main findings and conclusions that emerge? This final analysis helps you see what can and cannot be done to achieve your goals. Often, at this point you have several reasonable approaches to consider. This is the stage where you compare and contrast them along important dimensions such as risk, cost, and acceptability to key stakeholders. A more detailed description of the tools that can help you evaluate alternatives are available in Part Two.

Usually one or two of the possible solutions seem better to you than the others. The analysis you conduct at this stage needs to use the full range of data to test these preferences and justify why a particular course of action is preferable.

The most important limitation of any analysis is the quality and completeness of the data on which it is based. Weak data can't be improved by powerful analytical tools or fancy presentations. Be sure to pay attention to assumptions, estimates, and just plain guesses, and be honest with yourself and others in revealing what part they play in your analysis. Remember, too, that everyone has biases. Try to identify yours and counter balance them with solid analysis and reasonable alternative interpretations of the data.

Compare risks

In this phase you will make an explicit analysis of the risks associated with various tested alternatives. Notice that you have been identifying and dealing with risks from the outset. Opposing stakeholders represent risks, new technologies bring risk, and so on. Risk analysis can be very formal and quantitative or more qualitative. Scenario building is a technique in which you describe a hypothetical situation in a way that helps you predict the consequences of decisions and actions. It helps you see what could go wrong in different situations. You could also model these situations on paper or in simulations to understand how different actions or decisions might lead to unexpected results. This kind of analysis improves the confidence level of decision makers. These realistic projections of risk will help them understand the possible consequences of different choices.

Compare costs and expected performance improvements

Costs, of course, are critical to the final analysis of alternatives. You begin to identify and specify them much earlier in the process, just as you did with risks. This is the time to examine them in more detail to be certain that all cost factors are accounted for and that future costs are considered along with the costs of design and implementation. In projects that involve more than one organization or level of government, be sure to assess the costs to all players. Avoid the common mistake of costing out only the direct investments that your own organization must make.

Cost estimates can be obtained from historical data such as budgets or spending records, feasibility studies, or from outside consultants or agencies that have attempted similar projects. A cost-benefit analysis can be a simple comparison of costs and projected savings, or it can be a more detailed financial model. You can also conduct a robust cost-benefit or cost-performance analysis by considering all the players and effects of the system over time. Whatever you choose, the results have to be a convincing aspect of the business case you will develop.

Make and explain final choices

At this point you may have a clear best choice and need go no further to consider other options. If you still have at least two feasible alternatives, though, some selection tools will be useful. These are designed to compare alternative courses of action in a structured and explicit way.

The multi-attribute utility model or MAU model is a versatile tool for comparing alternatives. Its academic-sounding name makes it seem forbidding, but if you have been involved in any major procurements, you have probably already used it. The MAU model rests on the specification of weighted evaluation criteria. These might be such items as total cost, convenience to users, time to completion, reliance on contractors, or a host of others. Each criterion is weighted relative to the others, often by distributing 100 points among them. Then each alternative solution is scored on all criteria. Simple math gives each alternative a total score. Sometimes the scores demonstrate a clear best choice, sometimes they narrow the field, and other decision- making processes are needed. Occasionally the numerical scores do not "fit" the intuitive assessments of the evaluation group or decision maker. Often this means that an important criterion is not being made explicit and should be added to the model.

SWOT (for "strengths, weakness, opportunities, and threats") analysis presents another way to compare alternatives. This and other prioritizing and choice-making tools to consider will be described in the tools installment.

If you have done a thorough job of working through the three phases of analysis and evaluation, you will now be in a good position to make "smart" choices. You will have a clear understanding of the project objective(s) and will know what your stakeholders think about it. Relevant business processes will have been identified and improved. You will have information about alternative ways of approaching your goal, including the experiences of others who have done similar work. These alternatives will have been tested in some way with stakeholders and users, and the costs and risks of each one will have been identified. If more than one approach makes sense, you've conducted a comparison process that reveals their relative strengths and weaknesses. Now you are ready to make your recommendation for action in the form of a business case.

Get ready to prepare a business case

Gather together the findings from each stage of your analysis and review them all. Even if you were involved in all parts of the project, you may find that new information and insights emerge only when you step back and look at all the evidence together. Look for patterns, reinforcing information, conflicts, and gaps. Try to state the main findings in a few sentences or bulleted key phrases to create a framework for the final analysis and recommendations.

Get all the people on your team together to discuss the results and to decide how best to present your key findings and recommendations. Your business case will need to answer several questions in a direct and convincing way in order to garner support from decision makers. Try to state the answers to the following questions in a few powerful sentences and then organize your detailed data to back them up.

• What are the characteristics and dimensions of the problem to be solved or initiative to be undertaken? (Review your service objectives.)
• What are the expected benefits of solving this problem or taking this initiative? (Review the strategic framework and features and functionality analysis.)
• What alternatives are available and how do they compare in terms of cost, feasibility, effect on stakeholders, and other key criteria that matter to decision makers? (Draw from cost-performance and stakeholder analyses.)
• What course of action do you recommend and why? (Use the results of tests and evaluations.)
• What are the one-time and ongoing costs of implementing your proposed solution? (These are part of the cost analysis.)
• What are the drawbacks in your recommendation? (Refer to the risk analysis.)
• What are the next steps?

Chapter 3 presents the essential elements of a business case for IT investment along with recommendations for presenting the case to various audiences of decision makers and stakeholders.

Many situations present a need for thorough analysis, but neither the time nor the staff resources to conduct one. Fortunately, the process presented here is not an all-or-nothing proposition. We urge you to do as much analysis as your resources allow because each inquiry adds valuable information to your ultimate decision. But in those all too common situations where time pressures mean only a few items can be considered, what should you do? Even if you have only a few weeks, you can gather crucial information that will help your project succeed. To do this effectively, you need a team of people who represent different perspectives and skills. We strongly recommend these activities:

• Start with a hopes and fears exercise
  Gather a representative group of stakeholders together for a morning or afternoon. This activity identifies and makes explicit the variety of initial expectations that people bring to a new project. Where hopes differ, the group can work toward consensus on either a common goal or an agreed upon priority for multiple goals. Stated fears give you strong signals about the risks you face and an early start on the work it will take to address them.
  
  
• For each goal, create a structured service objective
  For each goal, reach agreement about what would be done for whom with what effect. We often use this fill-in-the-blank sentence: "Our service (or business) objective is to do something specific for a specific customer so that the customer will receive the following benefits.
  
  This exercise forces your planning group to be specific about three things: the practical nature of each goal, the intended customer or beneficiary, and how that customer will be affected. It begins to give you a sense of how straightforward or complex your project will be.
  
  If you have one objective and one customer, your job will be easier than if you have several of either.
  
  
• Conduct some form of analysis that actively involves system users
  There are many ways to do this. The crucial element is to engage the users directly. This might be done in focus groups, site visits, workshops, or process mapping sessions. Engaging users directly is the best way to test and refine your objectives and to understand the existing reality that you intend to change with your project. These activities can build trust with users, who are key stakeholders, and can prevent you from making big mistakes based on untested assumptions about work practices, skills, and organizational culture.
  
  
• Conduct targeted best and current practice research
  List the top four or five questions you have about your project and find out who has experience that can help you answer them. You can use the Web or publications to locate likely sources, but don't rely on these exclusively. The best information will come from actually talking with these people. Most government managers are very willing to share their knowledge and lessons learned. Take full advantage of this opportunity to learn from the experience of others.
  
  
• Specify modest, moderate, and elaborate alternatives and estimate their costs and benefits
  Using the information compiled from the previous activities, specify the characteristics of modest, moderate, and elaborate versions of your project. Then describe the cheaper, faster, and better benefits of each one. Finally estimate the cost of each one for design and implementation and for ongoing operations.
  
  This abbreviated analysis is far from perfect, but it will give you a great deal of specific information to support your recommendations.