Risk analysis covers a range of techniques and analysis tools used to assess the likelihood of failure or undesirable outcomes from decisions or policies. As one researcher put it, risk assessment "is the application of…knowledge of past mistakes in an attempt to prevent new mistakes in a new situation" (Wilson and Crouch, 1987). The methods rely primarily on mathematical modeling, statistics, uncertainty, and decision analysis.

A way to identify threats that can derail success. As applied to business case development and decision making in IT projects, the most important elements of risk analysis are identifying the threats to success and assessing the probabilities and potential costs of the threats materializing.

A method for learning from past mistakes. A variety of modeling, statistical, and analysis tools can be used to examine past projects, determine where mistakes were made, and devise methods to avoid repeating them.

Identifying threats, possibility of damage. Careful risk analysis is needed to provide two kinds of information. One is a clear and detailed identification of threats or possible mistakes that can damage an initiative. The other is an estimate of the likelihood of each kind of damage actually occurring.

Outlining potential process problems. A number of important risks are associated with innovations in business processes. These include internal resistance to change or even subversion of objectives by unhappy participants. The costs and complexities of needed changes may be underestimated, leading to insufficient resource commitment. An inadequate or inaccurate model of the business process may be used, or inaccurate data about that process may lead to mistakes. Differences in the cultures of the organizations involved may produce conflicts that undermine success. Competition or lack of trust can inhibit communication and collaboration. And it may be impossible to generate the support from top leadership to sustain large-scale changes.

Identifying potential political opposition. Political opposition can lead to problems and barriers. Risk analysis should involve the positioning analysis described above, with special attention to estimating the strength of likely opposition from influential players. Risks can include failure to manage expectations about success or immediate results, as well as missing the possible influences of other large initiatives on the political agendas of supporters and champions.

Defining IT risks. A number of risks are associated with the use of information technology, including rapid obsolescence and emergence of alternative technologies after investments have been made. Avoid the tendency to over-promise the benefits of technology or underestimate the effort of implementation -- both lead to disillusionment and loss of support.

Describing environmental and organizational risks. Planning and risk analysis should take into account the kinds of policy shifts, as well as the sources of support and opposition to such policy changes, that constitute the greatest threat to your initiative. Demands and costs of human resources can also shift, due to labor market forces, and put a project in jeopardy. Careful environmental scanning can help mitigate or anticipate these possible threats.

Technical problems. The technical problems of statistical risk analysis can be substantial, since they depend on models of threats and probabilities. For complex projects, such models may be unavailable or even impossible to construct. In addition, statistical risk analysis often depends on historical information that may be unavailable for new projects, technologies, or collaborations. This problem may be mitigated in some circumstances by tools, such as system dynamics models or other simulations that allow for exploration of various scenarios or alternatives.

Long-term perspectives, short-term adaptability. This basic dilemma in mitigating and managing risk is especially acute in technology projects. IT plans and system designs based on current knowledge and technologies are unavoidably at risk. Systems built with smaller components or modules can provide for more flexible response to rapid changes, but their success depends in large part on accurate anticipation of technology trends, which is demanding and error-prone at best.

Chapman, C. and S. Ward (1996). Project Risk Management: Processes, Techniques and Insights. Chichester: John Wiley & Sons.

Bedford, T. and R. Cooke (2001) Probabilistic Risk Analysis: Foundations and Methods. Cambridge: Cabridge University Press.

Kammen, D. and D. Hassenzahl (1999) Should We Risk It? Exploring Environmental, Health, and Technological Problem Solving. Princeton, N.J.: Princeton University Press.

Kemshall, H. and J. Pritchard (1996) Good Practice in Risk Assessment and Risk Management. Bristol, PA: Jessica Kingsley.

Kemshall, H. and J. Pritchard (1997) Good Practice in Risk Assessment and Risk Management 2: Protection, Rights and Responsibility. Bristol, PA: Jessica Kingsley.

Stern, P., H. Fineberg, and the National Research Council (1999) Understanding Risk: Informing Decisions in a Democratic Society. Washington, D.C.: National Academy Press.

Wilson R., and E.A. Crouch (1987) “Risk Assessment and Comparisons: An Introduction.” Science, 236-267.

Scenario building and forecasting tools help design the future of a project. They are a way of answering the question, "Where should we be going and how will we get there?" These strategic planning tools, as distinct from the others, are usually about the big decisions organizations face about their future. They are strategic in that they involve decisions and actions with major consequences that extend over long time periods, and attend to the short and long term environmental factors that may affect events.

Scenario building. This is a process of designing a hypothetical situation in a way that helps you predict the consequences of decisions and actions. For example, Massachusetts has proposed legislation to require all state agencies to consult a database of outstanding arrest warrants when a citizen is seeking a service or benefit. Officials could examine the possible consequences of such a new policy by creating a scenario. This scenario would assume reasonable values for the number of times the policy would generate arrests of various types, and compute the increased demand on jails or law enforcement officers.

Forecasting. This tool is also used to predict future events, but it uses calculations based on historical data. Forecasting typically uses data that have been collected on some events over time and uses them to project trends into the future. Populations, crime statistics, and budgets often have ample historical data for forecasting. The mathematical models used in forecasting may take into account the forces that influence trends to adjust the predictions.

Simplifying reality for testing. These kinds of models provide a simplified version of reality against which to test ideas and explore consequences. They are most useful in the kind of complex situations characteristic of justice systems and their information flows.

Exploring possible actions. A model can be a very powerful tool to explore possible courses of action or decisions. Consequences can be explored in hypothetical rather than real situations, so the costs of errors or bad decisions are limited.

Developing a common understanding. The development of models also provides a way of creating a shared understanding of complex systems among those that work in them. This shared understanding can be of great value as an aid in collaboration.

Require advanced technical skills. The kinds of models described here require relatively high levels of technical skill for their construction and interpretation. If these skills are not available in your organization, it will require the intervention of external experts, usually at considerable cost.

Quality depends on data. In addition, the quality of the analysis resulting from the model is no better than the model itself and the data on which it is based. Careful testing and validation are necessary to avoid conclusions or actions based on a flawed model.

Presentation, communication. Models of this sort often pose problems of presentation and communication as well. They frequently involve complex mathematical operations or graphic images that are hard to understand and explain to non-technical audiences. A well-designed interpretation and presentation must accompany the modeling work for non- technical audiences and policy makers.

Ward, J., P. Griffiths, and P.Whitmore (1990). Strategic Planning for Information Systems. New York: John Wiley & Sons.

A way to assess initial and future total costs. Working out the costs of a project requires careful attention to what cost information is relevant, what's available, and how it can be interpreted and used. Although it can seem like a straightforward task, a comprehensive cost analysis can be quite complex and demanding. A cost is something of value that is given up or exchanged for a particular reason. It might be as obvious as the financial outlay for some new equipment or as subtle as the extra time it takes a supervisor to explain new procedures to a staff member. And costs continue beyond the initial start up phase of a new system or program and must be estimated for recurring annual expenses.

Detailed breakdown of costs. Cost estimation helps specify all the different categories of cost associated with both the start up and the ongoing operation of a system or service. The list of categories can be quite long if the project is large and complex. The cost estimate should include the costs of developing and maintaining the system, preparing the agency computing infrastructure to support it, and training staff and other end users to use it. It should also include the cost of all of the staff time involved in the planning, decision-making, and training for the project. Both one-time costs and ongoing costs should be included.

Determination of the costs of alternative approaches. The goal of cost estimation is to approximate the costs of alternative plans as thoroughly and explicitly as possible so they can be compared.

Avoiding underestimation. People often underestimate the costs of IT initiatives. Many times there are so many different kinds of expenses that some are overlooked. Frequently, the true cost of human resources is not made explicit, especially if most or all of the people to be involved are already on the payroll. The ongoing annual costs of hardware and software maintenance, upgrades, and training are often missed. By listing all the categories of cost that can be reasonably attributed the project, you are in a better position to later judge which alternatives give the best pay back.

Project evaluation. After a project is operational, initial cost categories and estimates can be used to evaluate how well the project performed with respect to its budget.

Failure to consider indirect and opportunity costs. Although these kinds of costs are often more difficult to identify and many not lend themselves to quantification, they should at least be identified and described so they can be factored into later choices about the cost-benefit of various alternatives. Direct costs of a new project are usually the easiest to identify and analyze, since they typically are the financial costs that are part of ordinary budget making and planning.

Indirect costs are usually based on estimates or pro-rating of shared resources, such as portion of infrastructure maintenance and depreciation or overall administration expense. These costs are usually more difficult to identify and analyze, since the estimates they require are often based on uncertain assumptions and limited knowledge of actual impact.

Opportunity costs are the losses or costs to the organization that result from implementing the new system rather than the alternative uses of those resources. These costs are real and can be important, but are very difficult to measure and document. Participants in the development and implementation of a new system are often very sensitive to opportunity costs, since these affect their day-to-day work. But these costs are not part of any formal accounting system and so may be ignored by planners and budget makers, often to the detriment of implementation. At the very least, you should attempt to identify the possible opportunity costs involved in your project and discuss ways to ameliorate negative impacts.

Difficulty estimating future costs. The cost of out years cannot be estimated with the precision or confidence that pertains to the first year. Some informed guessing is inevitable, especially in an environment of significant technological change.

Sample Cost Estimation Worksheet
	MODEST		MODERATE		ELABORATE
	First Year		First Year		First Year
		Annual		Annual		Annual
Project leadership
Project management
Organizational readiness training
Equipment for users
Program development and maintenance
Hardware acquisition
Hardware maintenance
. . . . .
. . . . .
. . . . .
HUMAN RESOURCE SUBTOTAL
INFRASTRUCTURE AND OTHER SUBTOTAL
TOTAL

Pardo, T. A., S. S. Dawes and A. M. Cresswell (2000) . Opening Gateways: A Practical Guide for Designing Electronic Records Access Programs. Center for Technology in Government. See the "Cost Estimation Tool" http://www.ctg.albany.edu/publications/guides/gateways?chapter=6

Boehm, B. W. "A Spiral Model of Software Development and Enhancement." Computer, (May 1988), 61-72.

Cost-performance and cost-benefit analyses estimations are used to determine the level of investment that is appropriate for your project. They are ways of answering the questions, "Is this worth doing?" and "How will we know whether it was worth it when we're done?" These tools are methods for assessing the value of a project by comparing its costs to measures of its performance, or more generally to the value of benefits it produces. The analysis requires accurate cost data, as well as measures of performance in appropriate units and overall benefits. Cost-benefit analysis tends to emphasize quantitative evaluation, while the broader concept of cost-performance analysis more readily accounts for qualitative results.

Cost and performance data can be obtained from operational records, direct observation, surveys, or group meetings at which those who perform the operations report and discuss costs and performance measures. Both one-time costs and ongoing costs should be included. If you have used the tools above to generate modest, moderate and elaborate alternatives and to estimate costs, then you are ready to make comparisons on a cost-performance or cost-benefit basis.

A way to understand the value proposition. Many books and articles have been written about cost-benefit analysis and it can be extremely intensive and elaborate in implementation. At its heart, however, this kind of analysis helps decision makers understand the value that can be obtained (in terms of goals achieved) for the investment to be made.

A way to compare competing alternatives. Cost-benefit or cost-performance analyses that compare alternative approaches on the same criteria help decision makers choose the best alternative given desired goals and resources available.

Making informed choices. When decision makers look at a business case, information from these kinds of analyses are important in choosing whether and how to go ahead with a project. They lay out the value proposition for various alternatives by showing how they can achieve program goals and what it will cost to reach them. Decision makers can see the strengths and weaknesses of different approaches and make the trade-offs that lead to a final decision about whether and how to proceed.

Identifying new alternatives. By explicitly comparing the costs and benefits of the various approaches against a given set of criteria, you may be able to spot additional alternatives that combine different elements of the original set. For example, you may see a way to take advantage of elements of one approach early on and then move to a second approach later.

Evaluating a project. The results of your cost-benefit or cost-performance analyses form an important basis for evaluation. After a project is implemented, these cost and performance measures can be used to evaluate whether the initiative actually achieved its goals within its expected budget.

Complex environment. A comprehensive analysis of your project's impact may be difficult to prepare because of the complex environment in which public sector programs reside, and the many factors that may affect the intended outcomes of the project. It is often difficult or impossible to attribute outcomes and impacts directly to one specific initiative to the exclusion of all other efforts that are seeking similar goals.

Can over-emphasize quantitative data. Classic cost-benefit analysis seeks to place a quantitative value on every aspect of cost and benefit. For example, if a project saves lives, classic cost benefit would put a dollar figure on every life saved. In the public sector, many efforts are aimed at qualitative goals that are difficult or impossible to quantify. These goals may improve convenience for citizens, improve quality of life, or act in concert with other programs so the direct effects are impossible to discern. Nevertheless, these benefits need to be identified, described and taken into account.

Bloniarz, P., and K. Larsen (2000). "A Cost and Performance Model for Supporting Web Service Investments." Communications of the Association for Computing Machinery, 43 (2) 109-116.

Boardman, A. E., D. H. Greenberg, A. Vining, and D. L. Weimer, eds. (2000) Cost Benefit Analysis: Concepts and Practice. 2nd Ed. Upper Saddle River, NJ: Prentice Hall.

Boehm, B. W. (1988) "A Spiral Model of Software Development and Enhancement." Computer, (May), 61-72

Center for Technology in Government. (2003) Combining cost and performance assessments for decision support. Available at http://www.ctg.albany.edu/publications/guides/costperfmodel?chapter=8&PrintVersion=2 [Retrieved August 10, 2003].

Center for Technology in Government. "Return on Investment in IT: A Guide for Government Decision Makers." www.ctg.albany.edu/publications [Retrieved August 10, 2003].

Levin, H., and P. McEwan, eds. (2000) Cost Effectiveness Analysis: Methods and Applications. Thousand Oaks, CA: Sage Publications.

Nas, T. F. (1996) Cost Benefit Analysis: Theory and Application. Thousand Oaks, CA: Sage Publications.

Multi-attribute utility (MAU) models are mathematical tools for evaluating and comparing alternatives to assist in choosing among them. They are designed to answer the question, "Given the factors we care about, what's the best choice?" MAU models are based on the assumption that the desirability of a particular alternative depends on how well its attributes measure up against key evaluation factors. For example, if you are shopping for a new car, you will prefer one over another based on how well each one scores on the factors you think are important, such as price, reliability, safety ratings, fuel economy, and style. These models can be applied in all kinds of decision situations and are often used in the technical and programmatic parts of procurement evaluations.

Methods to evaluate alternatives. MAU models offer a structured way to weight, evaluate, and compare possible alternatives. They offer a quantifiable method for choosing among options.

Ways to conduct sensitivity analysis. A MAU model can also be used to explore the consequences of changing the attributes, their weights, or the scores they receive. Since the model usually is embodied in a simple spreadsheet, it is possible to make any number of changes and review the results. For example, if it appears that some attribute is too important in determining the results, the weights can be adjusted to produce different overall scores and to see if those differences really matter to the final decision.

Clarifying the nature and importance of evaluation criteria. One of the most useful benefits of using a MAU model is that it makes clear to all involved the basis on which the alternatives are being evaluated. This is particularly important in group decision-making situations in which many different points of view and decision alternatives have to be reviewed and taken into account.

Managing complex comparisons. Some choices need to reflect evaluation of many criteria. A MAU model helps manage that complexity by converting the evaluation to a numerical score while still presenting the logic behind the score.

Requires group consensus. MAU models are typically used in a group situation. To be effective, the group must be able to come to consensus on the attributes in the model and on the weights to be used to indicate their relative importance. It may be very difficult and time consuming, or even impossible to achieve consensus on very controversial decisions.

Conflicts often arise. The level of detail and specification necessary in the discussion of attributes, their overall importance, and the extent to which each alternative meets them can result in considerable conflict and contention, rather than the move toward consensus. However, conflict often indicates that some important criterion has not be made explicit and this possibility should be explored through candid group discussion.

Edwards, W. (1982). Multiattribute Evaluation. Beverly Hills, CA: Sage Publications.

Prioritizing methods establish the relative value of choices or alternatives. They answer the question, "How do these items rank in importance?" You can prioritize answers to this question in a rank-ordered list of the choices to show what should be done first, what requires the greatest attention, and what needs the most resources.

Objective-based priorities. Actions or choices can be prioritized in terms of how they affect the achievement of an objective or fit into a structured process. These can be called objective priorities.

Ways to identify the critical path. Program Evaluation and Review Technique (PERT) is an example of an objective priority-setting process. A PERT analysis shows which activities in a structured process are part of the "critical path." This is the sequence of events that determines the overall pace of your project. Activities on the critical path usually receive priority attention because delays will affect overall progress.

Triage activities. Triage is another objective-based priority-setting process in which choices are made according to whether they will affect the overall achievement of objectives. In medical triage, for example, cases that are not in immediate need of attention receive lower priority, as do cases where the likely success of action is small. Those that combine urgency with potential for success get top priority.

Criteria priorities. Priorities can also be based on a set of criteria. Cost-benefit or cost-performance analyses are examples of this sort of priority setting. Whatever choices yield the greatest value on the criterion measure get highest priority. Results of a MAU model (below) would also be a form of criteria-based priority setting, but one that may be linked to a group decision process.

Group voting techniques. Priorities chosen through voting are based on the subjective preferences of the voters. These include one-person one-vote methods, where vote totals can determine priorities. Multi-voting methods are also used in which each voter gets some fixed number of votes to distribute among the choices.

Investment decisions. These methods attempt to identify those factors that will have the greatest influence on progress or success and therefore point to the people, organizations, tools, and activities that should be highest on the list for investment of resources.

Group decision-making. A single person can use these methods, but often they are used in a group setting. In these cases, prioritizing methods are used in situations where a variety of perspectives or preferences must be taken into account. In these cases, the prioritizing is an open process which reveals differences that can then be explored in an effort to achieve consensus.

Tough choices among desirable things. Priorities often involve choices among competing "goods." The process of identifying and setting priorities will almost certainly involve conflict and controversy. Some planning and preparation are necessary to keep the work on track.

More tough choices. Setting priorities does not end the tough decision process. Even though you know which choices are most important, you still have to figure out exactly how to allocate resources and work assignments.

Existing preferences, policies. Priorities set by objectives or voting methods may become irrelevant if they fail to align with an organization's preferences and policies. Therefore, the effort invested in priority setting activities may not always determine outcomes. Active consultation with top executives or policy leaders should be a part of the policy deliberations to avoid conflicts and wasted efforts.

Modell, M. (1996) A Professional's Guide to Systems Analysis. 2nd Edition. New York: McGraw Hill.

http://www.projectreview.net/prio.asp gives a quick guide on how to do a group prioritization matrix to narrow a team's focus. [Retrieved July 7, 2003]

For a more detailed explanation of the Program Evaluation and Review Technique (PERT), Critical Path Method (CPM) and Gantt charts, see http://studentweb.tulane.edu/~mtruill/dev-pert.html [Retrieved July 7, 2003]

SWOT analysis is a simple framework to help answer the question, "What are the prospects for success?" The approach recognizes that any project should be examined for both positive and negative influences from internal and external perspectives. A SWOT framework prompts you to look in detail at both sides of the coin. That is, the strengths and weaknesses of your project are only meaningful in terms of the opportunities and threats in its environment.

A way to identify strengths and weaknesses. To obtain this knowledge about yourself (strengths and weaknesses) and others (opportunities and threats) requires identifying the SWOT elements and analyzing them in depth. This is typically done in interactive groups where people can discuss, assess, and elaborate on what is identified in each category.

A method for maximizing the positive and minimizing the negative. The analysis and deliberation are designed to identify ways to take advantage of strengths and exploit opportunities, as well as minimize the impacts of weaknesses and protect against threats.

Testing feasibility. SWOT analysis is best suited to a stage in business case development when the nature of the objective is reasonably well known. It is a useful way to test the feasibility of project objectives and methods.

Determining how to move forward. This type of analysis helps you start to identify what is needed to move your project to the next phase armed with a good understanding of both internal capabilities and environmental factors and their interaction.

Expressing different viewpoints. The interactive process can provide people with an opportunity to express their views about the project and discuss their implications. Advocates of usually emphasize strengths and opportunities. Opponents tend to emphasize weaknesses and threats. Neither view alone creates the balanced or comprehensive analysis needed to make the right choices. The SWOT framework provides legitimate exposure for both perspectives and an opportunity to reconcile opposing points of view.

A basis for further planning. The results of a detailed SWOT analysis also provide valuable material for continued planning and support-generating activities. The strengths can be presented and emphasized to potential supporters. Discussion of weaknesses and threats provides useful information for strengthening the project or business plan where possible. Conversely, problems and weaknesses that cannot be eliminated become better understood. As a result mitigation plans and contingency planning can take place.

Information quantity and quality. The key to effective SWOT analysis is the sufficiency and quality of available information. Participants' understanding of your project, its resources, and weaknesses must be deep and detailed. Similarly, analysis of the environment in terms of opportunities and threats must be based on extensive experience, thorough scanning, and collection of data from a wide variety of sources.

The future cannot be predicted precisely. Complete information about the environment is never available and projections about future events and trends are always subject to error. So the SWOT analysis must include consideration of the reliability of the information used and the conclusions reached. Considerable technical resources may also be needed in some circumstances to provide forecasts and projections for assessing the opportunities and threats in the environment.

Assumes shared goals. The process of SWOT analysis is based on the assumption that the participants all share the same goals. This, of course, is not always true. Because the process depends information provided by participants, as well as their collaboration, the analysis may be vulnerable to disruptive or subversive behavior.

Collett, Stacy (1999) "SWOT Analysis". Computer World 33 (29) p. 58.

Stewart, R., S. Mohamed, and R. Daet (2002) "Strategic implementation of IT/IS projects in construction: a case study." Automation in Construction 11, 681-94.

Management Sciences for Health and UNICEF. "SWOT Analysis". http://erc.msh.org/quality/ittools/itswot.cfm [Retrieved July 7, 2003].

About.com's site addressing how to perform a SWOT Analysis. http://businessmajors.about.com/library/weekly/aa123002a.htm [Retrieved July 7, 2003]