Choosing and using the various methods of ROI analysis requires both good knowledge and good judgment: knowledge about the methods and judgment about how best to apply them to a particular IT project and its setting. This section presents an introduction to the methods. It is intended to impart enough knowledge so that the reader can exercise effective judgment about whether to use a particular method, how and to what extent it may be applied, and where to go for more complete information and resources.

We begin with a discussion of the issues associated with project time frames and scale, then present various approaches to process analysis and measurement. We conclude with a brief overview of risk analysis.

The selection of method will depend to some degree on the time frame and the scale of the project and the ROI analysis. Time frame refers to both the time perspective for the analysis and the overall time period of the project to be included in the analysis framework.

Anticipating the future. ROI analysis is commonly used prospectively. The results of the analysis are intended to inform a decision about a future IT investment. For a prospective analysis, estimates of anticipated cost and performance are based on assumptions about the future that may involve considerable uncertainty. The results of the analysis will depend, to a large degree, on how accurate those assumptions turn out to be.

Learning from the past. A retrospective ROI analysis will show actual performance data about the IT project’s costs and returns. The longer timeline and complexity of larger projects can lead to substantially different results. The analyst must take these possibilities into account when evaluating the results of pilot projects. These concerns become part of the risk analysis discussed in a later section.

Importance of scale. Even when there are actual data from such a pilot project, the problem of the accuracy of assumptions remains. There is no guarantee that costs or returns from a small pilot project will accurately predict what happens in a larger effort. The scale of the larger effort can itself lead to different results.

In the case study described in Appendix A (page 28), the analysis combines the two perspectives. The developers created a pilot project, which was a small part of a larger project they planned to undertake. They then used the data gathered from a retrospective analysis of the pilot project to estimate the costs and returns that would result from the full development.

IT projects can have large and wide-ranging impacts on business processes. These impacts can occur directly in the business processes involved in the project and in other related processes. Methods for the analysis of business processes are too large a subject to cover in detail in this guide, so a brief introduction to these techniques is included in the section on methods below. The reasons for including attention to these methods in ROI work is related to the overall strategic objective of the project and how it fits into the rest of the organization.

The business process perspective concen- trates on where an IT project fits within the larger picture of the organization’s mission or core purpose. For example, a state department of transportation typically has responsibility for ensuring the safety of the transportation infrastructure. This would usually include responsibility for inspecting bridges on state highway systems and contracting for repairs where needed. The main elements of that business process includes many activities, from decision making about inspection policies and schedules through contracting for repair and construction work. The activities could include scheduling and performing inspections, reporting and analyzing results, then moving into the RFI, RFP development, bidding, contracting, and project supervision. Any change in one part of the IT system in an interconnected business process like this one, would likely have important impacts on other business processes. An IT project that places electronic sensors on bridges that send stress data to the central office will affect more than the inspection process. It will have ripple effects all the way from the employment and training of inspectors to the costs of new construction and the public’s perception of bridge safety. These latter concerns are all potential elements in an ROI analysis.

The interconnectedness of IT projects and the overall business process is nicely illustrated in one recent experience with a Web project in New York City. The City government added a section on its Web site reporting the results of its Health Department inspections of restaurants. The IT staff did not anticipate the popularity and high demand for such information. Nor did they understand the effects that demand would have on other linked elements of the business process, which includes both producing and disseminating inspection results. The Web server was initially overwhelmed by hits on the inspection information soon after it was posted; word of mouth communication rapidly spread the news. That problem could be easily fixed with additional server capacity. However, the high-level of public demand for this information and its significant impacts on restaurateurs put new pressures on the Health Department. They were pressed to increase the timelines of inspections and to use more plain language to ensure better understanding of the results. These were substantial cost factors directly related to the Web site project. A narrow focus on the IT component of the project led the planners to miss the effects on linked elements of the business process. The results were both unanticipated costs resulting from down-stream impacts on the business process and unanticipated benefits due to increased service levels to the citizens.

The linkages in this example extend from the specific details of an IT project—new data on a Web site—all the way to the relationship of the organization to its political environment. These linkages and their importance are obvious in retrospect, as are many implications for both costs and returns. The goal of good business process and ROI analyses is to provide the same insights and information for planning and decision making in advance. How that can be done is outlined in the description of methods below.

Business process models make the implicit assumptions and mental models of individual managers and stakeholders more explicit and open for discussion. They also:

• inhibit premature jumping to a solution because of the way they structure thinking about a problem;
• create an externalized definition of the problem that can serve as a focus of discussion and can help to align thinking about what are root causes of observed problems;
• force managers and analysts to come to grips with the precise logic or causal forces that are causing a problem;
• require attention to decide which key variables to measure;
• allow analysts and managers to communicate their reasoning effectively and efficiently to external audiences;
• can provide a simulation of how the full system will operate within a full context of organizational and human factors;
• push managers to see the implications of a limited prototype when it is expanded to full scale operations, requiring careful attention to technical, organizational, and policy issues; and
• can include financial elements that allow explicit exploration of costs and benefits of proposed solutions.

Sensitivity analyses of models provide answers to "what if" questions about various types of system functionalities and possible organizational and human effects. This helps planners anticipate issues and problems before they are encountered in a real world system implementation

IT investments of any sort must be viewed as part of the business processes in their organizations. Both the costs and returns of the investment are tied to how the new technology fits with and how it alters business activities. The notion of what constitutes IT must be expanded to include not only the chips, wires, and software, but also the activities and interactions that generate the costs and value that results from using the IT (i.e., the enterprise architecture). Failing to map and understand the enterprise invites a badly flawed understanding of how the IT investment will work, and can be a short cut to failure. For example, the U.S. Department of Education recently invested millions of dollars in a Web site for college students to apply for financial aid. The site generated very little use and was closed down. It failed because it ignored the important . role of college financial aid officers in the overall process of students seeking and receiving aid. The designers did not understand or ignored the overall business process of financial aid administration.

Business process analysis tools and methods range from simple flow and GANTT charts to sophisticated formal and mathematical modeling. Simple flow charts are sufficient to identify the basic elements of most business processes and some of the more obvious dependencies. How far business process modeling needs to go beyond simple diagrams depends on the questions to be answered and the resources available for the work.

The basic components of a business process are activities, flows, controls, and dependencies. Analyzing the business process therefore means identifying the various kinds of activities, what kinds of flows they generate and support, and how the activities and flows are controlled and linked to produce some particular collection of goods or services. Flows can consist of information, persons, resources, control inputs, and work products moving from one activity to another. The analysis consists of gathering information about the activities and flows in as much detail as necessary. Depending on the kinds of modeling to be used, this information can include descriptions of activities and flows in strictly qualitative terms or detailed measurements of resources used, flow metrics, and outcome measures.

Approaches to process modeling can be grouped roughly into three levels or types: descriptive, analytical, and dynamic. Descriptive models are the most fundamental. They describe the elements of a business process and the relationships among them. A flow chart that describes how travel expense reimbursements are processed, for example, might show how the request originates, how it moves from one work process to another, what actions or decisions take place at each point, and the elapsed time for each step. Descriptive models may involve some measurements, such as personnel time, expenditures, other resources used, and outputs. The flow chart in Figure 2 below illustrates the actual billing information flows for a nonprofit provider of services to mentally retarded and developmentally disabled clients. It shows how information about services delivered is converted into bills submitted to the various government sources of funding for these client services.⁷ This flow chart was produced to help understand how a change in the reporting and processing technology for one of the funding agency’s would affect operations among the providers who deal with multiple agencies. The IT project was planned by one state agency, but the planners recognized that the business process involved a range of other agencies and local practices. Therefore their analysis of the costs and potential benefits of the project had to include attention to the overall business process, not just what happened within one state agency.

Figure 2. Example of Process Flow Chart

The level of detail used for such a model is a matter of judgment, in this case dictating a model at an intermediate level of detail. For some purposes, a more highly aggregated model may be sufficient, such as the following.

It might be necessary to examine a much finer level of detail, such as how administrators review service records before approving billing in Figure 2 on page 17. The level of detail depends on the questions that guide the overall effort. A descriptive model influences the choice of what is to be measured and whether more formal modeling will be needed.

Analytical or formal models go beyond description to represent the performance of a business process or project in some quantitative or mathematical form. This kind of model has many advantages for analysts and decision makers, along with some substantial problems and limitations. There are at least four kinds of advantages.

• Formal models require making the implicit explicit, and to do so in precise and systematic ways.
• Many formal modeling methods allow exploring "what if?" questions and work with alternative scenarios by simulating the results of using the model of the new system. The planner can thus test ideas and alternatives without risk to real systems, operations, or data.
• The analysis yielded by the modeling can reveal insights into the behavior of persons or organizations that would otherwise be difficult or impossible to detect.
• The modeling process, if done in a collaborative way, produces a shared understanding of the work among managers and stakeholders that can lead to improved decisions about the project and its ultimate implementation.

Because of these advantages, the use of these techniques for process improve- ment and project planning has grown substantially and spawned an industry producing software applications, consulting, and training in one or more of these methods (see resources in Appendix D).

These methods are not without problems or limitations, the most important of which is that they raise the costs of analysis in several ways. These models typically require more detailed and extensive information, particularly performance and outcome measures, than simpler descriptive methods. In addition, the construction and use of these modeling methods requires considerable time and expertise. Organizations that do not have staff prepared to do the modeling work themselves would have to invest either in extensive training or employing consultants. It is important to review carefully whether the complexity of the project justifies the investment in formal modeling before taking that course of action. The review of alternative methods below is intended to help in that regard.

Agent-based modeling. The basic idea of agent-based modeling is that complex behaviors of collectives (social groups, members of an organization, population) can be modeled by simple computational rules. The rules treat the members of the collective as autonomous agents that follow simple rules of behavior, rules that can be modeled on a computer. What happens to the collective is the result of the individual agents interacting with each other according to the rules. Changes in the rules change the overall results, sometimes in predictable and sometimes unpredictable ways.

One early example of agent-based modeling in the social arena was a model of residential segregation developed in the late 1960’s.⁸ The model starts with a hypothetical city with a mixed race population randomly distributed over the residential area. Each household either stays or moves according to two simple rules: a preference for living near persons of the same race, and deciding to move if the population in surrounding areas has too low a proportion of such households. There is no rule involving dislike for households of different race or desire to avoid them. Nonetheless, the action of these rules over time produces a city of racially segregated neighborhoods.

In agent-based modeling, the rules are the model or theory. Thus they make explicit the assumptions and motives of the agents and what governs their actions. The value of the model depends on the degree to which the rules produce results that match the actions and outcomes for the actual collective of interest. If rules are developed that reproduce the results of interest, they can provide a powerful tool for understanding how those results occurred and for testing ideas about how to change or improve them. This kind of modeling may be particularly appropriate for information technologies that will involve the activities of large numbers of persons engaged in the same type of activity, such as large numbers of citizens seeking the same kinds of information on a Web site. A valid model of how people will use a system can be a valuable tool for exploring what costs and benefits to expect.

Agent-based modeling is not for the faint of heart, however. It is mathematical, often requiring advanced knowledge of algorithms and computing to implement. There are special modeling languages for developing the computer programs to pursue agent-based work, but they require experienced users or a considerable investment in training. If the rules are sufficiently complex or abstract they may be difficult to communicate to participants in the planning or to key stakeholders.

Operations research and statistical modeling. If the factors that influence a business process can be identified and measured, modeling that process by operations research and statistical methods may be feasible. These modeling approaches use mathematical equations to represent the business process. This requires measurements of the business process itself and of the factors that are thought to influence how the process works. Then the modeling equations are chosen to fit the way the significant elements of the business process are conceptualized.

• If the process is seen as a series of activities that consume resources and occur in particular interdependent sequences, project description tools such as PERT charts or critical path analysis may be used.
• If the sequence of events in the business process is uncertain, or if multiple paths or outcomes are possible, probabilistic methods such as Markov chain models may effectively represent the process.
• If the sequence or path of influence is not known, but some measure of their general effects is available, linear modeling and regression methods may be useful to represent the overall process or predict the results of changes.
• If particular combinations of influences are thought to work together to affect the flow and/or performance of the business process, then scenario analysis may be useful.

These modeling methods are similar to agent-based methods in that they require measurements. They differ in that they are based less on specific theories or assumptions about the activities that make up the business process. Rather, they use simplifying assumptions about the underlying cause and effect relationships that drive the process. That makes the models easier to use in some respects. Little original theory building is needed. In addition, the methods are supported by commonly available applications, and training in many of the particular methods is typically part of management education programs. Choosing among these alternatives is not a straightforward task and a reasonably high level of expertise is needed. The same applies to interpreting the results and understanding the limiting assumptions that apply to each.

System dynamics modeling. Many business processes involve feedback, that is, chains of effects in which what happens at one point in the process has an impact elsewhere that in turn influences (increases or decreases) the original effect. For example, a new database application may produce a new kind of report that users find particularly valuable. Seeing one possible new outcome prompts the users to request additional kinds of reports that produce more valuable results, and more new requests, and so forth. Any kind of process that can be represented as stocks or supplies that flow from one place to another with either reinforcing or negative feedback can be represented using the methods of system dynamics modeling. These models can provide both a concep- tual and mathematical representation of the processes. The conceptual or graphical representation is usually referred to as a causal loop model. These models can be built by analysts working with the staff involved in the process, making use of staff knowledge and insights. The image created is a potentially powerful tool to evoke knowledge about the process and to move the participants to a shared understanding. An example of such a diagram is shown in Figure 3 (page 21). This particular model represents a working theory about how work progresses. Tasks can exist in one of the four following states.

• Work to do
• Undiscovered rework
• Known rework
• Work really done

At the start of work, all tasks are in the stock of work to do. As participants perform work, correctly done tasks move to the stock of work really done, with a probability of 1- error rate (a fraction). The error rate is based on the idea that it is impossible to perform all tasks correctly on the first try. Tasks performed incorrectly require rework, so as work proceeds some propor- tion of the tasks enter the accumulation undiscovered rework, according to the probability of an error occurring, error rate. Similarly, problematic tasks can be redone correctly (moving from the accumulation of known rework to work really done) or incorrectly (returning to undiscovered rework), based on the error fraction. The concreteness of instructions and transformability of the tasks themselves are shown as affecting the ability to do work right in the first place or to correct errors. With appropriate assumptions and values for the variables and parameters in such a model, a mathematical representation can be built that reproduces the behavior of the process itself. Analysts, who should have appropriate expertise in system dynamics modeling, can change assumptions or values and explore the consequences, yielding new insights into the possible costs and performance of a project.

Figure 3. Causal Loop Diagram of Project Management Processes

Unified Modeling Language (UML) and Use Case Modeling. UML is a programming and modeling language that provides a system for representing and documenting object-oriented software development and the business process in which it resides.⁹ It is particularly tailored to analyze the application development process and for use by development teams in that environment. The overall conceptual scheme for use of the UML includes what is referred to as Use Case Modeling as a component. A Use Case is a guide for software design based on a behavioral case study of how an application is used in an actual business process. This differs from traditional approaches that specify software requirements in terms of technical needs and goals. For IT projects that are primarily application development, UML can be a very useful and powerful tool for modeling the project itself for management purposes, as well as for its linkages to the business process. However, the UML is highly technical and requires considerable study and programming experience to use effectively.

Investment in the use of UML may be justified, especially for large application development projects that involve multiple teams. As the UML documentation states:

"In the face of increasingly complex systems, visualization and modeling become essential. The UML is a well-defined and widely accepted response to that need. It is the visual modeling language of choice for building object–oriented and component-based systems.¹⁰"

Workflow modeling. The workflow modeling approach is similar to UML but involves a different focus and approach. Instead of using a programming language to model work processes, workflow methods use generic models of work processes and the relationships among them. The components of these generic models represent business activities, resources, dependencies, and controls. The design of the models is aimed particularly at enhancing coordination by identifying dependencies, failure modes, and problems of handling exceptions. For complex business processes, a flow chart will not reveal the consequences of errors at critical points, unanticipated resource shortages, or missing factors during some point in the process. Workflow models allow these situations to be explored and provide analysis of where processes are particularly vulnerable to failure or waste.

Workflow modeling technologies and commercial products may be useful to analyze an existing business process or re-engineer one. But they are designed primarily to develop automated work processes. These depend on computer- based controls, typically where the flow of information or materials involves networked links among the activities. However, the basic concepts of dependencies, exception handling, and error modes may be applied regardless of the level of automation in a workflow. So the conceptual tools of workflow analysis may contribute to a process model concerned with costs and returns to a new system.

What is measurable within a government accounting frame is set by the generally accepted and/or legally mandated accounting standards and practices that apply to the particular government organization. Definitions and structures for financial data are standardized and designed to serve specific government purposes. The Office of the New York State Comptroller describes the state framework in these terms:

"The purpose of classifying accounts is to provide a standard format for recording and reporting financial transactions which allows comparisons to be made with others municipalities or other financial periods."

"The classification system serves as a basis for budgeting, accounting, and reporting as well as for administrative control purposes, accountability to the Office of the State Comptroller and the general public, cost accounting, and the compilation of financial statistical data on the state level.¹¹"

This framework organizes the accounting data about costs/expenditures and revenues according to general organizational programs, functions or divisions, and by objects of expenditure (e.g., salaries, equipment, etc.). This is sometimes called a function-object accounting or budgeting system. A typical function-object accounting system for a state agency has a highly detailed and structured way of defining what kind of financial information is to be recorded and how it is to be organized.

The strengths of a government accounting framework for analysis of costs and benefits are also its weaknesses, namely standardization and rigid structures and rules. Standards and rules make such a frame-work useful for control of finances, generating standardized reports, and overall fiscal accountability. And if the existing categories of accounts happen to provide the particular cost and return data needed for ROI analysis, the frame-work would be readily available and useful. However this is seldom the case, particularly for IT investments. The full value of an IT investment is typically a mixture of many types of expenditures and other costs, the accurate accounting of which is not possible in the traditional accounting system. These systems will usually record the costs of equipment and personnel attached directly and full-time to IT activities. But they seldom record the changes in personnel costs for those who use the technology or the investments in training and learning to operate new systems. Maintenance and supplies may appear in a range of separate accounts, and many managerial and analytical costs are very difficult to estimate or prorate accurately.

In its most basic economic sense, a cost is whatever you have to give up to get something else. And a return is anything good that you get as a result, whether that good is measured in financial terms or otherwise. This is, of course, a much broader view of costs and returns than is found in most accounting systems. For example, an accounting system would not likely include as part of the cost of a new computer system the amount of time devoted to the purchase decision. Nor would the accounting system record the benefits of increased staff morale from a more functional or reliable system.

A full consideration of costs requires attention both to opportunities and to indirect costs. An opportunity costframework takes into account what alternative actions or returns would be missed or foregone as a result of a particular investment decision. The path of any IT project necessarily excludes other paths not taken. If opportunities along those foregone paths can be identified and valued, they can become part of the cost calculation. If I choose to implement a new procurement processing system, for example, the staff time spent training for implementing the new system is time unavailable for training in some other area of work skills. If a value can be assigned, this lost opportunity may be considered part of the overall cost calculation. Indirect costs (sometime referred to as imputed costs) are those that are not incurred or measured directly, but are calculated or estimated from other measures. For example, the cost of using existing network cabling for a project could be calculated from the amount of new traffic generated or from some other prorating factor. Similarly, apartment renters do not pay real property taxes directly, but the renter’s portion of that tax can be calculated as an indirect component of rent.

These costs are often "off the books" in the sense that typical public accounting and financial management systems do not provide this kind of cost analysis. There are no standard accounting practices to guide the measurement of these costs. An agency may have an existing standard indirect cost factor that is used in budgeting for Federal grants or contracts (e.g., some fixed percentage of personnel expenditures). But these standard factors are based on averages over many projects and are not particularly useful measures for any one particular project. Opportunity costs are even more problematic and are often not included, leading to possibly significant underestimation of overall costs. Underestimating true costs can make an investment’s returns look more attractive than they may really be.

Assessing effectiveness requires identifying the outputs of the project and its implementation in business processes or program terms. That means identifying meaningful units of output that can be related to the cost estimates. For example, the effectiveness of a new system to process business permit applications over the Web could be measured in terms of increased numbers of permits processed during a given time. Lower costs per processed permit would be a useful cost/effectiveness indicator. Producing such a cost/ effectiveness measure requires detailed data about permit processing and a way of assigning costs on a per permit basis. Without some enhancement, most government accounting systems do not provide the basic data necessary for such a calculation. It may be necessary to do detailed data collection on a sample of permit transactions, for example, to establish a baseline unit cost figure. A business process model could provide the necessary data if so designed. Even if such a unit cost measure is available, it may have problems, such as the distorted assumption that all permits are equally important or costly.

Efficiency is a way of describing the effectiveness of a project or system in relationship to costs (or other inputs). Efficiency cannot be separated from effectiveness, since using resources and failing to achieve a desired outcome can be little more than waste; you don’t save money by building half a bridge. Efficiency is usually expressed in terms of optimizing the value of a return for a given cost or input, or alternatively minimizing the cost for a given value of result. It is possible, of course, to improve efficiency without necessarily achieving an optimum. As long as it is possible to compare cost/effectiveness or return ratios for alternative systems or methods it is possible to make judgments about efficiency. To demonstrate an optimum result or projection, it is necessary to have a simulation or optimization calculation to provide the data. Some operations research and simulations, such as linear program- ming or workflow simulations, can generate this type of calculation if the necessary models and data are available. However most IT projects do not have the data or analytical resources to include these methods in an ROI analysis.

The identification of variables that make up a social cost or benefit calculation along with their definitions is broader than both the economic and accounting frames. They are based either on the specific program results desired by an agency or on general social benefits and improved quality of life. These impact measures can come from several sources.

Operational data. Some useful measures of social or broader economic outcomes may be available in the data collected during ordinary program operations. Providers of services for homeless persons, for example, routinely collect data about programs and activities of clients that could be indicators of impact on their quality of life or progress toward independence. Similarly, law enforcement agencies routinely collect crime statistics that can be useful indicators of neighborhood climate or quality of life.

Social and economic statistics.Government agencies collect statistics on social and economic indicators that are relevant to the overall social and economic status of their jurisdiction. These range from the enormous resources of the US Census Bureau to state and regional planning agencies, and include market research statistics available from private sources.

Special studies and evaluations. For large, high cost projects separate data collection activities, such as surveys or field research, may be used to collect and analyze data about social and economic impacts of a project. These efforts may be expensive and time-consuming, but may also be the only way to obtain data about particular outcomes. The case example in Appendix C on Social Return on Investment provides some examples of how this may be done.

Time has important effects on the measurements and calculations. Both the costs and returns of an IT investment extend over time. Moreover, the costs may be incurred long before the returns are realized. So when estimating the money value of costs and returns, it may be necessary to take into account the effect of the time perspective on the value of money. That is, a dollar in hand today is worth more than the promise of a dollar in hand a year from now. How much less that future dollar is worth, known as the discount, depends on the current time value of money, which is usually called the interest or discount rate. Thus if the discount rate is 5 percent per year, then the promise of a dollar a year from now is worth only 95 cents today. The farther out in the future, or the larger the discount rate, the less the present value of the promise of that future dollar. Calculations based on the discounted value of future returns are useful to estimate what size of return is needed to justify the costs of an investment. The typical methods used in this kind of financial analysis are called Net Present Value and Internal Rate of Return calculations. Both methods use an initial cost and some assumed figures for a future stream of returns, along with a reasonable assumption for the interest (or discount) rate. The Net Present Value (NPV) calculation estimates the value today of returns coming in the future, less the cost of the investment. If the NPV result is less than the cost, then the investment does not appear to yield a net gain. The Internal Rate of Return (IRR) calculation yields the same sort of result by a different path. Using the same basic information as the NPV calculation the IRR calculation yields the rate of return for the investment assuming a break even, or NPV of zero. If the IRR is less than the current interest rate, this indicates that putting the investment in an interest bearing bank account would yield a bigger financial payoff than investing in the project. An example of NPV and IRR calculations are shown in Table 3 (page 26). This example is based on a hypothetical investment analyzed over a six year period. It shows that when the time value of future returns is taken into account, the total returns must exceed the total cost by a substantial margin to generate a positive net present value or internal rate of return.¹²

Table 3. Net Present Value Calculation ¹³

Year	Costs	Returns
1	$ 100,000	$ 15,000
2	$ 5,000	$ 20,000
3	$ 4,000	$ 25,000
4	$ 4,000	$ 25,000
5	$ 4,000	$ 30,000
6	$ 3,000	$ 35,000
Total	$ 120,000	$ 150,000
Discount Rate		3.0%
Net Present Value		$ 13,297
Internal Rate of Return		6.0%

Notice that any calculation of this sort requires specifying a time-frame within which the value of the project or investment will be assessed. This time frame is sometimes referred to as the life cycle of the project. Any IT investment is assumed to have a limited useful life, beyond which the returns are not considered, or when the expected returns from some new technology will lead to replacement of the previous investment. For IT investments, the rapid change in technology makes the choice of a reasonable life cycle or planning framework a difficult but necessary part of the analysis.

Risk analysis consists of assessing the importance of threats and how to mitigate or eliminate them. We usually evaluate a threat in terms of how likely it is to materialize and how much damage or cost would result if it did materialize. We know, for example, that a large asteroid could hit the Earth and destroy a continent—an enormous threat in terms of consequences. But astronomers tell us that the likelihood of that happening is very, very remote—therefore, a small threat in terms of probability. Because of the low probability, the threat is seen as small enough that few of us (aside from some science fiction writers and astronomers) take it into account in day-to-day life. The same logic applies to risks in IT investment. Consequently risk analysis can be thought of as having four basic steps:

• identifying the source of threats,
• assessing the extent of potential damage or cost to the project,
• assessing the likelihood of the threat materializing, and
• devising ways to reduce or eliminate the threat (i.e., mitigate the risk).

Threats can be reduced by taking steps to lessen the damage or cost that would occur if the threat materializes and also by reducing the probability that the threatening event or action will occur. For example, a project could call for using the most reliable platform for a critical database application, reducing the probability that the system will experience a failure. The project could also implement a backup system capable of taking over processing if the primary system does fail, reducing the potential damage of a failure.

The overall subject of risk analysis is too large to treat in detail here. However, most of the risk assessment issues described above involve problems of thinking beyond the boundaries of the project, measuring factors, or determining probabilities. This should not discourage risk analysis.

The experience of those involved in IT projects can be a rich source of intelligence and experiential data on which to base reasonable estimates of risk potential and problem sources. The literature on IT investment is another rich source of analyses of successes and failures that provide additional insights into risks and mitigation strategies. Simply recognizing where uncertainty and potential damage lie is half the battle. Careful risk analysis, based on the best available data and estimates, will surely assist in ROI analysis and improve planning, even if the amount or quality of data is less than ideal.