Once the decision has been made to conduct an ROI analysis, what should it look like?

The choice of how to conduct the analysis should be based on four critical principles pertaining to:

• the strategic objective(s) of the ROI analysis,
• the place (and importance) of the IT investment in the overall enterprise architecture,
• the type of analysis that should be conducted (i.e., what data and methods of analysis are best suited to those objectives), and
• how the ROI analysis fits in the overall decision context for IT investments.

Understanding the strategic objectives of an ROI analysis will determine how the analysis is ultimately done and used. A handful of questions like these can help managers decide what the objectives of the analysis should be.

• Is the proposed project critical to the business objectives of your agencies or government?
• What are the risk factors associated with the investment?
• Who will be impacted—positively or negatively—by the proposed project?
• Is an ROI necessary for approval and support of the proposed project?
• Is the proposed project worth the investment of an ROI analysis? And if so, how detailed should it be?

Answering these questions will help identify the resources needed to conduct an ROI analysis, which in some cases can itself be a substantial investment. Extending its level of detail beyond what is needed for effective decision making is a waste of resources. Focusing on strategic objectives keeps attention on the full range of benefits to be expected from the investment, and how to measure them.

Any IT investment project is embedded in an organization’s technology infrastructure (enterprise architecture), relevant business processes, organizational environment, and external relationships.

• Technology infrastructure. There are direct costs associated with the technology and services in which you invest, and there will also be costs in terms of its impact on other technology systems already in place. The benefits range from more efficient automation and workflow to improved collection, storage, and access to information.

• Business processes. An ROI analysis must not only account for the improvements to relevant business processes, but also for the costs associated with training staff involved in using the proposed technology system.

• Organizational environment. Other costs and returns will be linked to the organization, for example through altered resource flows, performance changes, changes in work flows and internal relationships.

• External relationships. Linkages with the external environment may be significant as well. Resources may be committed from this environment to support the project and additional costs may be imposed on external persons or organizations by changes in the way services are delivered or other business is conducted.

Choosing and using the various methods of ROI analysis requires sound knowledge and judgment: knowledge about the methods and judgment about how best to apply them. The methods chosen should fit the particular questions asked of an ROI analysis. Different questions require different measurement approaches to fit them. In general, there are four types of questions that prompt or drive an ROI analysis: financial, effectiveness, efficiency, and impact.

An ROI analysis that answers these questions is based on expected savings and revenue increases compared to the dollar cost of all expenditures on the new system. The measures are set by generally accepted or legally mandated accounting standards and practices that apply to the particular government organization. The costs and savings or revenue might be projected over a multi-year time span to show a payback period or to estimate the present value of future returns.

The ROI analysis that will answer these types of questions considers how much the investment contributes to achieving program goals and producing the desired results. It considers direct, indirect, and opportunity costs. The indirect costs include such things as training and administration over time. An opportunity cost could be the loss of return or revenue you would have received had you chosen a different alternative. The measurement of returns will be expanded beyond cost savings to include levels of performance relative to program or project goals.

The ROI that tackles this question requires information about the greatest possible value relative to its costs. Efficiency cannot be separated from effectiveness. It is usually expressed in terms of optimizing the value of a return for a given cost or input. Establishing that some particular result is the best of all possible results requires either examining many alternatives or simulating performance in some way that provides a valid picture of what is possible. This can be done for some (but not all) kinds of systems with sufficient resources and data. However to do so can substantially increase the cost and complexity of the analysis.

The analysis that answers impact questions will be concerned with the larger social and economic benefits and costs of a project. To define and measure variables that represent social costs or benefits requires more than the typical economic or accounting frameworks. These measures are based on either the specific program results desired by an agency or on general social benefits and improved quality of life. Though not impossible, the breadth and complexity of this kind of ROI analysis is rarely found in IT investment planning.