Chapter 2. The analysis and evaluation process
Phase 2. Identify and test solutions
Phase 1 arms you with a very detailed understanding of your objective and the context in which you must try to achieve it. In Phase 2, the work turns to a search for reasonable solutions that you can investigate and test. This phase makes substantial use of the experiences of others who have tackled similar goals. It also leads you to develop alternative solutions and offers some ways to try them out in low-cost, low-risk ways. You are not building a system yet-but you are building a great deal of knowledge about the system that you will eventually construct. A more detailed description of the tools that can help identify and test possible solutions are available in Part Two
No matter how unique your problem may seem, some other organization has probably faced it. That organization may be another agency in your own state; it could be a government somewhere else in the US or the world; it could be a commercial or nonprofit organization. It might be much bigger or much smaller than your organization, or engaged in a very different kind of work. Regardless of their settings, you can learn from them. The first part of the work in Phase 2 is the search for relevant practices, tools, and techniques whose use in other places can teach you something that you don't have to learn on your own.
Find relevant practices, tools, and technques
You can learn from the experiences of others by using a variety of research tools. Best and current practice research
is a good general-purpose tool that any professional can use effectively. You can conduct this kind of research in the library, on the Internet, at professional meetings, in face-to-face interviews, and on the phone. The key is to specify, as clearly and narrowly as you can, the questions you want to answer, and then to evaluate critically what you read and hear.
One benefit of working in the public sector is the general willingness of public managers to share their good and bad experiences with one another. You can often learn more by picking up the phone and calling a contact person than by just reading a story in a newsletter or Web site. Probe for the complete story of how a project unfolded or a technique was used. Ask about critical success factors and ask also what should have been done differently.
Know your environment
An environment scan
can help determine what is happening in the overall environment that may have an impact on the project. The results of this kind of review may lead you to reconsider some aspects of your own service or business objective. You may want to return to and refine your focus before moving forward to other parts of the analysis. You might also discover other stakeholders need to be considered or that some part of your problem analysis needs more attention. Take advantage of these early opportunities to improve your fundamental analysis - they are more valuable and less costly now than they will be later in the project.
and building technology awareness
through trade shows, demos, and vendor presentations are other good ways to collect current information that you then evaluate for comparability, completeness, and relevance to your own situation. These investigations may help you understand the features and applications of technologies you are considering for your project. They may also help you understand the broader infrastructure needs you will have to take into account.
Identify modest, moderate, and elaborate solutions
Phase 2 is completed by specifying several alternative approaches or solutions that seem to fit well with the context, problem, and stakeholder analyses from Phase 1 as well as the information you gathered and evaluated from other places at the start of Phase 2. At this point, your work group may have several ideas to pursue. You should specify them all in similar ways so that you can compare them. Models of solutions
can be built to can help minimize risks and get all the potential development costs on paper.
A description of features and functionality at modest, moderate, and elaborate levels of investment
is one way to do this. First, list the key elements of a solution. These will vary according to your objective, but some common elements are the means of customer access, response time, degree of customization, level of security, extent of manual data handling, and degree of integration with other processes or systems. The next step involves describing how a minimally useful solution would address each of these elements. This is the "modest" solution or the one that accomplishes the least worth doing. Then specify how a "moderate" solution would address each element. This one offers greater functionality, more convenience, or other improvements over the modest level. Third, list how an "elaborate" solution would address each element. This is the most advanced solution that an organization might attempt. Finally, for all three alternative solutions, state the benefits and who reaps them. Benefits may be quantifiable as dollar or time savings. We think of these as the "cheaper" or "faster" benefits. Another category of benefits might be categorized as "better." These benefits come from qualitative changes such as improvements in service quality or availability.
Prototype when possible
Testing your alternatives may start by gathering and comparing performance data from existing projects that are operating elsewhere. If all the alternatives are operationally feasible, then testing the concept with appropriate stakeholders may be sufficient. If the proposals involve some real unknowns, then try to create test conditions that are as close to real life as possible, such as building one or more prototype systems
tested under field conditions in a controlled experiment. Also remember that the more costly the solution (in dollars, effort, and change to the status quo) the more reason you have to conduct a life-like field test.
One surprising result of these tests may be the extent to which non-technology solutions fill your needs. Often significant improvements in business processes or information flow go a long way toward meeting your objectives, even without the application of new technologies. In one agency we worked with, $3 million had been set aside for an imaging system to improve customer service transactions. After successfully building and testing a prototype that reflected completely revamped business processes, the agency decided not to spend the money on new technology. Why? The prototype effort showed that most of the benefits could be obtained by process improvements alone.