These tools can be used individually or in combination to assist an individual or organization during the analysis that precedes the development of a business case for an IT investment.
Visioning is a tool you use to establish an image of what you want your organization or project to look like in the future. The time frame associated with the vision depends on the needs of the group and may range from months to years.
A way to stretch your thinking. Creating a vision is a way to stretch your organization and establish a vision of a "preferred state." Growth in terms of size or scope of operations may form part of a vision, but doesn’t always constitute a vision. The circumstances facing your group need to inform the vision. Being realistic is important, as is remembering the concept of stretch. Ultimately, the vision should guide or frame the work that all participants will need to do in order to accomplish the desired outcomes.
Various methods. The task of visioning can be completed in several ways. Regardless of which method you use, your main focus is to develop ideas. You must get everyone to share their ideas, reach a common understanding, build consensus, and craft a meaningful vision statement.
Developing shared goals. Vision statements are often very good at "getting everyone on the same page." The process of constructing a vision statement involves discussions and interactions that will help the group reach consensus on ultimate goals.
Reflecting interests, needs, skills. Remember that vision statements should reflect your interests and be attuned to their specific needs and capabilities. Otherwise, the likelihood of accomplishing the vision will be greatly reduced.
Team building exercise. In short, a well-crafted vision statement that has buy-in from everyone involved is often a crucial first step in beginning any group project.
We've done this before. Almost everyone has been through a process like this at one time or another. Some of the most prolific buzzwords around involve the words "vision," "mission," "empower," and "group consensus." Depending on previous experiences, the level of cynicism may be very high when an exercise like this begins and may remain high even when a vision statement is developed.
Address skeptics. Perhaps the best advice is to directly address participants' cynicism. Let them know that they are in the room to make things different. Participants have to find a way to cooperate and take responsibility for the outcomes of their efforts.
Predict the future. The final pitfall associated with the visioning process is that people are often poor prognosticators. Time and experience may require you to revisit and modify the vision. The real key here is to see the vision as a dynamic statement and not simply a motto meant for hanging on the wall. Consequently, visioning should be used periodically as a project unfolds.
- Use a round robin facilitation format to elicit people's thoughts about the characteristics they want to see embodied in the project.
- Display all of the responses from step 1 (adhere them to a wall or project them to a screen).
- Clarify what is being expressed in each characteristics, but avoid debate at this time.
- Establish one or more small groups to take the characteristics and develop a vision statement(s) that reflect the key ideas.
- Encourage the full group to discuss all of the statements and begin developing a common vision upon which the group can agree -- this is when debate begins.
- Repeat steps 4 and 5 until you produce a vision statement that satisfies your needs.
French, W. L., and C. Bell, Jr. (1999, 6th ed.) Organization Development: Behavioral Science Interventions for Organization Improvement, New Jersey: Prentice Hall.
Kakabadse, A., F. Nortier, and N. Abramovici (1998). Success in Sight: Visioning. Boston: International Thomson Business Press.
Quinn, R. E., S. Faerman, M. Thompson, and M. McGrath (2003, 3rd ed.) Becoming a Master Manager: A Competency Framework, New York: John Wiley & Sons, Inc.
Hopes and Fears exercises are techniques that help members of working groups share their perspectives on the task at hand and build a common understanding of goals and potential problems. This technique is similar to a number of other group and team building facilitation exercises. It is a way to help answer the question, "What's on the minds of those who have to accomplish this thing?"
Ways to share hopes and fears for project outcomes. Hopes and Fears exercises are used to make explicit the various hopes and fears associated with the project. The discussion generated from this activity can include ideas about how to ensure that the most important hopes are realized, as well as how to prevent the most important or costly fears from being realized.
Building common understanding of goals, barriers, and enablers. These exercises are most useful in building a work group's shared perspective of its expectations and potential problems. This is particularly important near the beginning of a project. It is at these early stages that the members are likely to have the greatest differences of opinion about what they are supposed to do.
Sharing perspectives for effective work. Coming to a more thoroughly shared perspective is necessary for effective group work and communication among its members. It is also useful to identify where hopes held by some members may be unattainable or even inappropriate to your overall goal.
Preventing sources of frustration. If unrealistic or inappropriate hopes are identified early in the process, they are less likely to become sources of frustration and resentment that can interfere with your group’s effectiveness. It is also reassuring to some members to learn that others in the group share their respective fears.
Simple and effective icebreakers. Hopes and Fears exercises are simple, unthreatening activities that are useful as icebreakers for new groups. They allow the members to learn about each other and begin group interaction smoothly. The process of eliciting individual members' hopes and fears, and giving them credence also emphasizes the value of each person’s contributions and can promote more enthusiastic participation in subsequent activities.
Drawing on intuitions and knowledge of the environment. These exercises bring out the background, experiences, and personalities of participants in ways not enabled through formal modeling techniques.
Reluctance to reveal feelings. Since these exercises are commonly used for new groups, members may be hesitant to reveal their hopes and fears to an unfamiliar group of people.
Skillful facilitation necessary ingredient. A good facilitator is required to get the hopes and fears process moving. Even with skillful facilitation, it is likely that some members will withhold information for strategic purposes or simply from embarrassment. The full range of hopes and fears may be unavailable for discussion.
Smaller groups most effective. Because the process depends on active and relatively free-flowing discussion, it's inappropriate for very large groups.
- A facilitator or team leader first asks each member of the group to articulate his or her hopes for the outcomes of the project.
- The hopes are then organized into related clusters on a wall or other display, and each item is discussed to be sure that it is well understood by the members.
- The same process is then used to elicit and discuss the fears. Each member is asked to articulate what undesirable outcomes they fear might occur, followed by clustering and discussion.
- The facilitator may use rating or prioritizing techniques to help the group identify the relative importance of the different various clusters of hopes and fears.
A service objective is a structured way to express the goals of your project. The process of creating a commonly understood and agreed upon service objective often reveals differences in thinking, different assumptions, and conflicting perspectives that all must be discussed and resolved before the service objective statement is complete.
Clear statement of project beneficiaries and goals. Once the group has developed a vision or mission statement, the service objective sets a course for them to reach specific goals. Developing a service objective is both an individual and a group activity.
Reaching agreement on intent. This exercise provides a framework for discussing the similarities or differences in team members' intentions or expectations. You can use it to determine if you are expressing the same intent in different ways or if members of the group have different objectives in mind for the project.
Developing common language. Once a common intent is agreed to, expressing that intent in a common language increases the likelihood that group members will talk about the project in the same terms to people who are outside of the group, such as agency colleagues and decision-makers.
Creating a foundation for future action. The service objective provides a powerful reminder about key goals that each member can work toward when completing his or her specific tasks.
Agreeing too soon. Often people hear what they expect to hear when talking about broad project objectives. Failure to explore the underlying intent of each participant can lead to a service objective that is not really a shared understanding, but just a placeholder for each person's individual ideas. Beware of too-quick agreement and encourage debate and discussion before adopting a service objective.
How to create a service objective using a "Madlib" exercise
A Madlib is a fill-in-the-blanks sentence that we often use to generate a service objective statement. The sentence we use is this:
Our service objective is to provide (who) with (what) that allows them to (action) so that (outcomes).
This format forces the group to specify who the project is designed to help, how it will help them do something specific, and how that specific action will result in something of value. When all the blanks are filled in, a service objective emerges. It usually takes many iterations before a group is satisfied that they understand and agree on the objective. The following steps will take you through the process:
- Each individual should write down what they believe the objective of the project is using the Madlib format.
- Each individual shares their Madlib with the group by reading it and posting it on the wall.
- After all of the individual objectives are shared, similarities and differences (and their underlying logic and assumptions) are discussed.
- Sometimes additional information will be needed to explore and evaluate different versions, so often multiple meetings are needed to allow for this.
- Finally, the group develops a single agreed upon service objectives or a set of related service objectives for the project.
A strategic framework is a structured way to understand a project proposal by helping you clearly define each key service objective and its customers. The framework then helps you identify the resources, partners, and innovations that might contribute to success. To be most effective, the strategic framework should work with one project-specific objective at a time. Strategic frameworks can be devised by one person and then presented to and reviewed by others, or they can be created through a facilitated group decision conference.
An analysis of the internal and external factors. The framework leads to an initial identification of potential resources, including partners, and to a closer look at potential uses for information technology and other innovations. To be most effective the framework should work with one service objective at a time.
The strategic framework helps project teams identify four factors (customers, resources, innovations, and partners) that will influence the definition, development, and operationalization of their project goals.
- Customers are people or organizations who make use of the service you intend to provide
- Partners are willing participants in a joint enterprise who invest staff time, equipment, money, or credibility in the creation and operation of the service. Partners share costs, risks, and benefits and engage in active, trustful working relationships with one another.
- Resources are something of value that is necessary to the success of the service. When using the strategic framework, it is usually useful to specify what resource(s) are associated with an organization, rather than just an organization's name.
- Innovations are products and services that could be used to design, develop, or deliver a new service or to offer an existing service in a new way.
Taking a high-level view. The framework lays out the full array of internal and environmental factors that can support a particular service objective by:
- Identifying existing and potential partners to help achieve those objectives
- Identifying information and other resources that will be needed
- Identifying innovative products and services that might be relevant
- Getting more specific about the customers of the service
Recognizing multiple roles. In using the framework, you are likely to find that specific people or organizations appear in more than one place. For example, if your service objective is "to create a repository of geographic data for use by state and local governments," a particular state agency could supply data to the clearinghouse (i.e., be a resource) and use the clearinghouse to find data provided by others (i.e., be a customer). The same agency might play a major role in the design, operation, or financing of the clearinghouse (i.e., be a partner). As a result, you will eventually need to discuss issues related to developing and managing relationships with an organization that may play a combination of roles. Often these different roles will be played by different people or units within the same organzation.
Refining objectives in light of what customers need and what the environment has to offer. Another strength of the strategic framework is its capacity to reveal different "points of view." or POVs. You can experiment with this idea by completing separate frameworks from the POV of two different customers. (In the clearinghouse example above, these might be state agencies and local governments.) Compare the results. You are likely to find that the resources, partners, and innovations that would make one customer happy, are not entirely the same as the ones that would satisfy the other. Understanding this disparity will help you sharpen your service objective and will become an important point of departure when you move on to define the scope of your entire project. You may need to narrow your scope in a way that makes it customer-specific--or broaden it to include features needed to satisfy additional kinds of customers.
Focuses on "enablers," but ignores barriers. You are more likely to identify barriers through modelling, prototyping, and best practice reviews.
Lacks the detail needed to craft a project plan or design a system. Most importantly, this tool does not deal directly with the availability or cost of identified innovations, resources, or partners. It focuses your attention on what is possible rather than what is practical.

- State your service objective as clearly as possible in the center box. If you have more than one objective, create more than one framework.
- Then fill in the factors that are important in achieving that objective.
- Who are or will be the customers of the service? Are they external, internal, or both?
- What information and other resources (human, material, financial, political) will you need?
- What innovative service approaches, technologies, or other products might be useful?
- Who might be your partners in this endeavor?
- Look at the results and ask yourself the following questions:
- Are we trying to serve more than one kind of customer? If so, which is most important?
- Who needs to be on the project design and development team?
- Do we have or can we get the required resources?
- Is there a good match between our customers' capabilities and the technologies we propose to use?
- How will we engage in partnerships?
- Have we pushed ourselves to think broadly about each factor, or are we staying with what we already know best?
- Does this picture make sense?
- Based on your answers, refine your approach and decide when and how to proceed with your project.
Andersen, D., S. Belardo, and S. Dawes (1994) "Strategic Information Management: Conceptual Frameworks for the Public Sector." Public Productivity and Management Review, 17 (Summer) 4, 335-353.
Center for Technology in Government (1996), Developing & Delivering Government Services on the World Wide Web: Recommended Practices for New York State. Center for Technology in Government, University at Albany, SUNY. See Chapter 4, Case 3, Example 3A for a completed strategic framework and Exercise 1, Using a Strategic Framework to define a problem.
www.ctg.albany.edu/publications/guides/developing_on_the_web/ developing_on_the_web.pdf
Stakeholders are individual and groups who are affected by or have influence over your initiative. Every project needs a careful assessment of stakeholders in order to understand who cares about it, how they can affect it, and how they will be impacted by it.
Structured examination of who cares. Stakeholder analyses are structured examinations of the relationships between a proposed project and key players in the environment. These analyses are ways of answering the question, "Who cares about this project and why?" Anyone who has an interest (a "stake") is considered a stakeholder. Stakeholders can be examined in terms of their roles, degree of support for the initiative, influence over decisions or resources, or the ways in which the project will affect them in both positive and negative ways. A variety of tools can be used for this analysis -- two (positioning charts and partisan analysis) are described in the sections below.
Refining project scope. The results of a stakeholder analysis may expand or reduce the scope of a project design and strategy. Often planners have a good understanding of internal stakeholders, but less appreciation for the needs and influence of external ones. Stakeholder analysis give them information to take all important stakeholders into account.
Refining service objective and strategic framework. A stakeholder analysis tests the usability of the service objective and strategic framework that has been set by the group. Because a service objective focuses on a single user or beneficiary, planners often find a stakeholder analysis reveals additional users (with different needs) and therefore requires additional service objectives for a complete understanding of the project.
Examining impacts in order to design a better plan. There are many stakeholders in the environment of a government program, and most information systems have multiple features or products that will affect stakeholders in different ways. Some will see increased access to services, or better quality service. Others may experience higher costs or more competition for scarce resources. This analysis can help you anticipate these effects before a full-blown project gets underway.
Expanding understanding of the environment. Most organizations are better at understanding internal dynamics than external ones. The stakeholder analysis forces you to look outside your organizational boundaries to estimate the impacts and outcomes of a new initiative.
Predicting potential results. The stakeholder analysis requires you to be specific about how various elements of a proposal will affect stakeholder groups. It helps you move from very general descriptions to more specific and measurable ones.
Identifying high-priority features and stakeholders. Once you understand the different ways the proposal will affect different stakeholders, you should be able to see which areas need priority attention. You should also be able to identify measures of how your initiative will impact different stakeholders and estimate the magnitude of those effects.
Assumptions required. The analysis requires assumptions about causal relationships and processes. Since you have imperfect data, make educated guesses about causes and influences. Keep testing these assumptions as your project proceeds.
Mitchell, R., B. Agle, and D. Wood (1997) "Toward a theory of stakeholder identification and salience. Defining the principle of who and what really counts," Academy of Management Review, vol. 22, pp. 853-866.
Tennert, J. and A. Schroeder (1999) "Stakeholder analysis," presented at 60th Annual Meeting of the American Society for Public Administration, Orlando, FL
The Management Sciences for Health and UNICEF have a guide on how to conduct a basic stakeholder analysis. http://erc.msh.org/quality/ittools/itstkan.cfm . [Retrieved on May 27, 2003.]
The UK Department for International Development has an extensive guide to doing a Stakeholder Analysis at http://www.dfid.gov.uk/foi/tools/chapter_02.htm . [Retrieved on May 27, 2003.]
Positioning charts show the relationships among people, groups, or other elements of a problem in terms of their positions. A positioning chart is a good first stakeholder analysis tool. The chart usually shows two factors important to the problem, with the people or alternatives arranged in the chart according to where they fit on the two dimensions.
As shown below, placing stakeholders on a positioning chart helps identify what different approaches or strategies will be most effective for the different positions. For the sample chart, different strategies can be chosen for dealing with different stakeholders according to whether they support or oppose the proposal and by their importance to its success. This kind of analysis can show that resources could be wasted on trying to generate greater support from those with low ability to help, or failing to recognize antagonistic stakeholders who could damage prospects for success.

Exercises to classify stakeholders along key dimensions. Positioning charts can be useful in mapping where particular stakeholders fit into the project. Knowing who has the inclination to champion the project and who has power and/or inclination to bring it to a halt can be helpful as you develop a project management plan.
Understanding potential influences. This type of chart allows you to better understand how various stakeholders can influence your initiative and gives you a start on developing strategies that take those influences into account.
Communication. Representing this kind of analysis in a positioning chart is not only a good planning exercise, it is also an effective device for communicating the results to others. Equally important, because it tells you important things about various stakeholders, it helps you devise communication strategies for working with each one.
Somewhat arbitrary process. Placing stakeholders on the chart is often an inexact, even arbitrary process that relies on experience and judgment. Wrong assumptions and lack of confirming information can lead to substantial errors in positioning, which results in flawed conclusions.
Oversimplify relationships. A chart may also oversimplify relationships in a complex setting, especially when more than two dimensions are involved or the relationships are not stable over time.
Bryson, John M. (1995). Strategic Planning for Public and Nonprofit Organizations. San Francisco, CA: Jossey-Bass.
Partisan analysis recognizes that competing interests and conflicts are natural and unavoidable in any significant government action. Any new project requires careful attention to the partisan or political nature of the process.
An inexact science. Partisan analysis use a number of different approaches and ways of thinking about interactions. Like many of the tools presented here, a partisan analysis is more a craft than an exact science. However, some basic questions can guide the analysis.
Way to determine wants and needs of participants. Partisan analysis includes finding out what participants want in general, or what they stand to gain or lose.
Method for understanding a wide range of issues. Partisan compromises often involve negotiation over a wide range of issues that may be unrelated to the immediate concern. In legislatures, this is referred to as logrolling. It is also important to understand both individual and organizational interests and desires. Those who speak for a group or organization do not necessarily share all the group's desires and objectives.
Framework for identifying key relationships. Projects typically involve parties with existing relationships and histories. It is important to know who are friends and enemies, where natural alliances and rivalries exist or may form, and what kinds of coalitions are possible or desirable. Consider where trust has developed or been betrayed and where old friendships or wounds will shape current perspectives and actions. These issues are often critical to forming the coalitions necessary to move forward.
An activity to find out who has power. A partisan analysis considers what power resources the parties bring to the table. These include: official status or authority; ability to punish or reward other participants; special expertise, status, skills, or reputation; and access to information. It's useful to know participants' preferences for different kinds of power and how they have acted in the past.
Way to analyze rules of the game. Effective strategies for managing projects depend on knowing what kinds of actions are acceptable and what tactics are the most successful in your organizational and political culture. These include preferred styles of negotiation or influencing others, limits or penalties for actions, and understanding the importance of signals and symbols of play.
Wild cards. Uncertainty plays a part in any environment. One major element of uncertainty is whether any outside actor or force will affect your plans. Partisan analysis often involves scanning the environment for possible external factors that may become involved. This scanning can also include analysis of the risks and probabilities of these kinds of events and the potential range of impacts.
Planning. Use partisan analysis to plan how to present your ideas (and your business case) to participants and outside audiences, and what to emphasize as your main selling points. You can also use it to decide the timing and format of presentations, what groups to make them to, and when.
Collaborating. This approach is an effective planning strategy for forming collaborations and work groups.
Strategizing. Use partisan analysis to develop a strategy for making potentially controversial decisions and for mobilizing support among participants and stakeholders.
Quality, amount of available information. The value of your partisan analysis depends in large part on the quality and amount of information available about the people and groups involved in your project. In most environments, people tend not to announce their true objectives and strategies. In fact, there can be substantial incentives to mask or deliberately misrepresent their true goals and interests. Judgments based on inferences about other people's goals and interests should be evaluated and tested against actions and other evidence.
Lack of definitive answers. Assessing the goals and interests of others involves a lot of uncertainty. There may be discord among groups about their goals and interests. It is often difficult to evaluate the accuracy and stability of statements and actions expressed by all groups involved in the project.
No history. Historical information may be an effective basis for judgment. However, iIn new initiatives, histories may be absent. Information about past actions and events may be unavailable, unreliable, inconsistent, or badly distorted by selective memory or interpretation.
The "process" of studying the way work is done is called business process analysis or process modeling. The study of business processes, or work flow, is a very important part of gaining an understanding of how organizations do their work.
One of the best ways to document and analyze a business process is by talking to the people who actually carry out the work. Through facilitated discussions, a work process can be documented using graphical representations or models to formulate a collaborative view. These graphical representations (or models) of business processes allow organiza- tions to learn more about the specific steps within a process. This baseline knowledge allows organizations to understand and potentially measure existing processes and to consider alternative ways of improving or changing them.
Diagram of business activities A business process model is a diagram (often called a model) that visually depicts the sequence of activities (or steps) involved in a particular work process. The goal is to document every step of the process, including decision points, action steps, information flow, documents produced or changed, and products developed or services delivered. It is important to include the appropriate people in the process in order to document their perspective.
Documentation of every step in a process. People often assume that everyone in a work group understands the business processes of that group in the same way. Often, activities such as process modeling reveal these assumptions and show that they are incorrect.
Activities represented by symbols. Process steps are mapped out using a series of standard symbols that represent various activities. For example, a circle represents information or action that starts or ends a process. Boxes indicate an activity of some kind. Diamonds show decision points or questions. And arrows are used to show flow within a process. Using these symbols, business processes can be mapped out a variety of different ways. The most straightforward approach is through the use of white boards, flip charts, or paper. Computer-based modeling packages are available for modeling more complex and sophisticated processes. The modeling tool used and the symbols selected are of secondary importance to capturing information from those involved in the process and using that information to create a model for group review.
Identifying and analyzing problems. A model helps organizations recognize the underlying causes of a problem, bottleneck, or breakdown in the process. The step-by-step nature of the model allows analysts to look at the individual elements and figure out how to change them to solve problems. The model becomes the focal point of discussion, allowing those involved to walk through the steps necessary to capture the institutional memory or knowledge of the work process. Convening the people connected with a process and documenting it through the creation of a process diagram or model results in a broader and more consistent understanding of the process. Assumptions are made explicit, expert knowledge is documented and different perspectives are presented and reconciled. These models can then help you predict where problems may occur and provide you with the capability to add or clarify activities that eliminate or avoid problems.
Investigating improvements. Once you have mapped out the steps, you can analyze a process to determine areas that can be improved. A model will show unexpected complexity and unnecessary steps. Identifying these areas can lead to improved workflow and performance.
Developing "What if" analyses. Business process models help you compare the perceived process to the actual and the ideal. You can then engage in "what if" analysis that may lead to changes and improvements.
Standardizing to help make the implicit explicit. This type of analysis allows you to standardize activities and decision points to improve workflow and decision making. It also ties implicit assumptions about how the system works to explicit performance measures and structured analysis.
Training and communication tool. Business process modeling is a great training tool to help people understand a complex process. It helps people who are new to the process or the organization better understand the flow of work. It also helps managers communicate effectively and efficiently with each and with those who are associated with the process.
Different levels of complexity. Developing a business process model involves varying levels of complexity. You may only want to map out the key components of a process for historical purposes, or you may need to fully describe every minute detail of the process to support the design of an automated workflow. The completeness and complexity of your model will vary depending on the type of analysis you want to conduct. The answer to "how much is enough" is not always evident at the onset of the analysis -- it may only become clear as the modeling begins to unfold.
Potential for bias. Typically, the people who operate or manage a process participate in the development of diagrams or models that analyze the process. This personal investment in the work can lead to bias in reporting and analyzing the workflow. It is important to guard against this in the construction of any model.
Expensive and time consuming. Mapping out all the steps in a business process is tougher than it may seem. It takes a lot of time and effort to accurately describe all steps required and to come to consensus about that model within a group. You must involve enough people who know about different aspects of a process to get a full accounting of it and you must have enough of their time to allow them to review the models and improve on them.
Fischer, L., ed. (2003). Workflow Handbook 2003. Lighthouse Point, FL: Future Strategies.
Hammer, M. and J. Champy (1993). Reengineering the Corporation: A Manifesto for Business Revolution. (First ed.). New York: HarperCollins Publishers.
Modell, M. E. (1996). A Professional's Guide to Systems Analysis. 2nd Edition. New York: McGraw Hill.
Brint.com, a business technology portal, contains a section of resources about business process reengineering and innovation at http://www.brint.com/BPR.htm. [Retrieved June 2, 2003]
The Business Process Reengineering Online Learning Center, hosted by ProSci, contains helpful tool kits, articles, tutorials, and other resources at http://www.prosci.com. [Retrieved June 2, 2003]
Ericson, J. "Starting Over With BPM." Portals Magazine. May 29, 2003. http://www.portalsmag.com/articles/default.asp?ArticleID=4695&TopicID=7 [Retrieved June 4, 2003]
Wreden, N. "Model Business Processes". Information Week
Online. September 28, 1998. http://www.informationweek.com/702/02iuprc.htm [Retrieved June 2, 2003]
Self-assessment tools include a wide range of methods to gather information about a current situation or performance. They are designed to answer the question, "How are we doing?" These tools range from something as simple as a newspaper survey asking readers to rate their knowledge of the Internet to the kind of year-long institutional self-assessment procedures used in hospital accreditation. The essential element in all self-assessment tools is that only the group being assessed answers the questions.
Self-administered questionnaires. The simplest form of self-assessment is a self-administered questionnaire or test. In order for the assessment to be effective, the questions must be sound and the participants must understand what is being asked and provide valid responses.
Informal evaluations. For informal self-assessments, neither the questions nor the resulting interpretation scales have any particular scientific basis. They are at best rough guides.
Formal evaluations. More comprehensive self-assessments and accreditation procedures usually involve the participation of those being assessed in setting and reviewing goals. In the typical institutional assessment, the evaluation criteria are a mix of external standards developed by the accrediting body combined with the institution's own goals and criteria.
Evaluating performance. If well designed, a self-assessment can be a highly efficient kind of performance or status evaluation. Once developed, the tools may be used repeatedly without the need for outside intervention.
Building consensus and morale. The deliberative processes in setting goals and reviewing performance can have positive effects on the organization by building consensus, enhancing morale, and increasing understanding of operations.
Screening for problems. Self-assessment instruments can also be screening devices to identify possible problems or areas for further attention. Self-administered surveys are often used in organizational development work to identify these areas and issues.
Planning. Self-assessment can also be used as a planning tool. It can assess past and current progress to inform strategic project management planning.
Hard to validate. Self-assessment tools are difficult to validate. The fact that they produce a measurement or evaluation result does not mean that it is accurate or meaningful. Interpretation must be done with careful attention to the validity of the tools and how they were used.
Distorted results. While all data collection results can be distorted in a number of ways, the nature of self-assessments may be more susceptible to this phenomenon. Those being assessed may deliberately provide false or misleading information to promote their own interests; provide inaccurate data due to their own biases, faulty memories, or flawed perceptions; or be unduly influenced by others in the process, either deliberately or inadvertently.
Bias of the tools. The tools themselves could be badly designed or insufficiently tested, or there may be accidental but serious flaws in the information produced. The assessment tools could be deliberately designed to favor a particular point of view or desired outcome.
Resource intensive. The kinds of information called for by assessment tools can be quite extensive and complex to assemble. While you may avoid the costs of external consultants and analysts, the kinds of internal deliberations involved in institutional self-assessments require considerable staff time.
Strategic Planning Process Self-Assessment Questionnaire for Federal/Government Agencies. www.strategicfutures.com/articles/stratpln/gov/quesgov.htm. Retrieved May 27, 2003.
Building Smart Communities (1997). San Diego, CA: International Center for Communications, San Diego State University. www.smartcommunities.org/index2.html. Retrieved May 27, 2003.
After an IT system is designed, built, and installed, you may discover that it solves only a part of the initial problem or that the problem was poorly understood to begin with. This happens when participants have different mental images, or models of what the problems are and how they should be remedied. A formal model of the problem makes these perspectives more explicit. A model represents a small-scale simplification of the problem being addressed. It can take the form of a system diagram, a process flow chart, a spreadsheet, a set of equations, or even a computer model.
Collective visions of the problem. The formal model is a public visualization of a problem. It allows people to see how their own assumptions of the problem fit into a larger view that is shared by all.
Reproductions of the problem. A good model can represent the essence of the problem. This helps ensure that the problem is understood well enough to begin investing resources in framing, testing, and evaluating solutions.
More than one thing. A good model is a collection of perspectives that work together to create a precise description of a problem to be solved. Although there are many different types of formal models, most of them share a number of common features including:
- "Stories" of how it works and what's wrong with it. These accounts typically reflect many different points of view. They can be gathered either through a group process or individual interviews.
- Common visuals of the system that needs fixing. These visuals usually take the form of business process maps (see next tool) or structured system flow models.
- Numbers and other measures of key variables. Quantitative models need to be explicit about which measurements are key indicators of the problem.
- Analyses of sources of the problem. The logical reasons for a problem are often revealed by drawing a diagram of a process or system. The elements of the process can be extracted from verbal accounts of the system, and lead to measures of system performance that reproduces the problem.
- Lists of proposed solutions. Ironically, many managers define problems in terms of their solution (e.g., the problem is that we need more people in the field). By collecting lists of what managers think the solutions ought to be, you can more clearly define the problems that are implicit in those solutions.
Making the implicit explicit. Modeling makes individual implicit assumptions and mental pictures explicit. It also imposes a process on a team's thinking about a problem that helps prevent premature conclusions.
Creating common definition of the problem and possible solutions. The process of modeling serves to focus discussion on the root causes of observed problems in order to develop common definitions and potential solutions.
Identifying underlying forces. Models can help analysts and managers come to grips with the causes of problems.
Measuring. Modeling pushes the team to identify which key performance variables really count.
Developing "what if" analyses. Models help participants see how problems would get better or worse under different sets of possible circumstances.
Communicating with external audiences and decision makers. Models help project team members communicate their reasoning to the external audiences that need to be involved in solving the problem or who need to make investment or other decisions.
Expensive and time consuming. Some approaches to modeling a problem (or its solution) may require specialized knowledge and techniques that are hard to find or expensive to apply.
Level of complexity. Sometimes the models themselves can get so complicated that they can't easily be understood. Overly complex models don't help illuminate the core of a problem. But models that are too simple fail to capture enough complexity of a problem to provide new understanding.
Bias. Sometimes the modeling approach that a team chooses has a subtle and biasing effect on how they will look at the problem. For example, spreadsheet models emphasize the financial aspects of a problem, whereas process maps tend to look at workflow.
Validity. Models can be wrong. When this happens, you have a whole group of people aligned around a view of the problem that won't yield solutions. Fortunately, this probably occurs less often with models than without them.
How to model a problem: one approach.
Because there are so many different approaches to modeling a problem, no single way of getting started always works. However, a number of common sense steps can get you far enough down the path to decide whether you can finish a problem-centered model yourself, or need to call in some expert help.
- Using a facilitated group process, gather each participant's concerns on paper (or on flip charts or white boards).
- As a group, create a system flow diagram, business process map, or some other explicit picture that captures these concerns using common vocabulary. This creates a common view of the problem.
- Once the common view (often a diagram) of the problem has been constructed, use it to elicit discussion about what is and is not important in the system. These discussions can help a group simplify the model or identify the most important features.
- If you and your team have arrived at a coherent and complete view of the problem you need to solve, and if the model you have developed seems adequate, then proceed toward exploring alternative solutions.
- If you are developing a quantitative model, now is the time to get some numbers to help tie the emerging model back to the real system. (Getting numbers can be as easy as using group process and expert judgment to calibrate key variables in the system or as complex as using extensive surveys, interviews, experiments, and data analysis exercises to measure critical aspects of system performance.)
- Once you have a common view of the problem tied to some preliminary numbers, you can begin to test the model by changing some of the numbers or revising part of the process to try to understand what those changes would do to the overall system.
- If your preliminary analyses are turning up questions that lack answers, if members of the team are arguing about the details of the model, or if a clearly defined problem is not emerging, then you may need to enlist the help of someone with more experience in modeling problems. You may have hit upon one of the many complex issues in the public sector that requires detailed analysis at the early problem-finding stage.
Gaylord, R.J. and L. D'Andria (1998), Simulating Society: A Mathematica Toolkit for Modeling Socioeconomic Behavior. Springer: New York.
Stokey, E. and R. Zeckhauser (1978) A Primer for Policy Analysis. New York: W. W. Norton.
© 2003 Center for Technology in Government
