Today more than ever, government decision makers must make the most of scarce resources and at the same time respond to ever-increasing demands for improved performance and new technology. These competing demands generate close scrutiny of proposals for new information technology (IT) investments. What’s more, high profile IT system failures have raised concerns about why these investments so often fail to live up to expectations. As a result, many IT investment planning processes now require some analysis of the costs and returns expected from that proposed investment. Unfortunately, public sector managers often lack models that can guide them through such analyses. This Guide is offered to help fill that gap.

The Guide provides that help by presenting a practical approach to understanding what ROI analysis can and cannot do. A meaningful return on investment (ROI) analysis in information technology is a little like saying you want to live a healthier lifestyle. Like lifestyle changes, ROI analysis is not just a single component. Instead, it is a collection of methods, skills, tools, activities, and ideas. They can be combined and used in many different ways to assess the relative value of an investment over time. Applying this collection in a particular situation requires making many choices among the ideas and methods available and conducting an analysis appropriate to the decision at hand.

Different choices will produce different results. Therefore the Guide presents a framework of the key questions that should lead to an appropriate ROI analysis. It then presents a review of the methods and resources needed along with examples of different approaches in detailed case studies.

Once the decision has been made to conduct an ROI analysis, what should it look like?

The choice of how to conduct the analysis should be based on four critical principles pertaining to:

• the strategic objective(s) of the ROI analysis,
• the place (and importance) of the IT investment in the overall enterprise architecture,
• the type of analysis that should be conducted (i.e., what data and methods of analysis are best suited to those objectives), and
• how the ROI analysis fits in the overall decision context for IT investments.

Understanding the strategic objectives of an ROI analysis will determine how the analysis is ultimately done and used. A handful of questions like these can help managers decide what the objectives of the analysis should be.

• Is the proposed project critical to the business objectives of your agencies or government?
• What are the risk factors associated with the investment?
• Who will be impacted—positively or negatively—by the proposed project?
• Is an ROI necessary for approval and support of the proposed project?
• Is the proposed project worth the investment of an ROI analysis? And if so, how detailed should it be?

Answering these questions will help identify the resources needed to conduct an ROI analysis, which in some cases can itself be a substantial investment. Extending its level of detail beyond what is needed for effective decision making is a waste of resources. Focusing on strategic objectives keeps attention on the full range of benefits to be expected from the investment, and how to measure them.

Any IT investment project is embedded in an organization’s technology infrastructure (enterprise architecture), relevant business processes, organizational environment, and external relationships.

• Technology infrastructure. There are direct costs associated with the technology and services in which you invest, and there will also be costs in terms of its impact on other technology systems already in place. The benefits range from more efficient automation and workflow to improved collection, storage, and access to information.

• Business processes. An ROI analysis must not only account for the improvements to relevant business processes, but also for the costs associated with training staff involved in using the proposed technology system.

• Organizational environment. Other costs and returns will be linked to the organization, for example through altered resource flows, performance changes, changes in work flows and internal relationships.

• External relationships. Linkages with the external environment may be significant as well. Resources may be committed from this environment to support the project and additional costs may be imposed on external persons or organizations by changes in the way services are delivered or other business is conducted.

Choosing and using the various methods of ROI analysis requires sound knowledge and judgment: knowledge about the methods and judgment about how best to apply them. The methods chosen should fit the particular questions asked of an ROI analysis. Different questions require different measurement approaches to fit them. In general, there are four types of questions that prompt or drive an ROI analysis: financial, effectiveness, efficiency, and impact.

An ROI analysis that answers these questions is based on expected savings and revenue increases compared to the dollar cost of all expenditures on the new system. The measures are set by generally accepted or legally mandated accounting standards and practices that apply to the particular government organization. The costs and savings or revenue might be projected over a multi-year time span to show a payback period or to estimate the present value of future returns.

The ROI analysis that will answer these types of questions considers how much the investment contributes to achieving program goals and producing the desired results. It considers direct, indirect, and opportunity costs. The indirect costs include such things as training and administration over time. An opportunity cost could be the loss of return or revenue you would have received had you chosen a different alternative. The measurement of returns will be expanded beyond cost savings to include levels of performance relative to program or project goals.

The ROI that tackles this question requires information about the greatest possible value relative to its costs. Efficiency cannot be separated from effectiveness. It is usually expressed in terms of optimizing the value of a return for a given cost or input. Establishing that some particular result is the best of all possible results requires either examining many alternatives or simulating performance in some way that provides a valid picture of what is possible. This can be done for some (but not all) kinds of systems with sufficient resources and data. However to do so can substantially increase the cost and complexity of the analysis.

The analysis that answers impact questions will be concerned with the larger social and economic benefits and costs of a project. To define and measure variables that represent social costs or benefits requires more than the typical economic or accounting frameworks. These measures are based on either the specific program results desired by an agency or on general social benefits and improved quality of life. Though not impossible, the breadth and complexity of this kind of ROI analysis is rarely found in IT investment planning.

Investment decisions in the public sector, whether they involve IT or not, necessarily take place in a context of political and policy influences. No matter how solid or technically sophisticated an ROI analysis may be, it will not likely be the sole determinant of an investment decision. When deciding how to prepare and present an ROI analysis, therefore, it is best to take into account all the potential risks that influence the decision process. Undertaking an ROI analysis should include attention to the risk factors identified below.

• Public exposure to failure
• Divided authority over decisions
• Multiple stakeholders
• Year-to-year budget cycles
• Highly regulated procurement processes

• Complex program networks
• Misalignment of (or conflicting) internal goals
• Lack of leadership support

• Impact on existing process
• Fear of changing work assignments

• Rapid changes in technology
• Interacting with parallel systems
• Scale and complexity

Most of the risk assessment issues listed here involve problems related to thinking beyond the boundaries of the project, measuring factors, or determining probabilities. Simply recognizing where uncertainty and potential damage may lie is half the battle. Careful risk analysis, based on the best available data and estimates, will surely assist in ROI analysis and improve planning, even if the amount or quality of data is less than ideal.

Considering these various risk factors can help shape the style, emphasis or presentation strategies employed to introduce the analysis into the decision making process. Such considerations as those listed here may also help in recruiting support for the conclusions of your ROI analysis and guiding how the analysis process is positioned when seeking that support.

There is no single "right" way to conduct a return on investment analysis. Nor is there a Consumer Reports for ROI products and services. In determining how to conduct your analysis, the best advice is to focus on the strategic objectives of the analysis along with the goals and business processes of the proposed project. This focus will help guide decisions about the resources and methods to use to conduct a sound and valuable ROI analysis.