Chapter Two: Methods of ROI Analysis for IT
Measuring costs and returns
Financial metrics and government accounting
What is measurable within a government accounting frame is set by the generally accepted and/or legally mandated accounting standards and practices that apply to the particular government organization. Definitions and structures for financial data are standardized and designed to serve specific government purposes. The Office of the New York State Comptroller describes the state framework in these terms:
"The purpose of classifying accounts is to provide a standard format for recording and reporting financial transactions which allows comparisons to be made with others municipalities or other financial periods."
"The classification system serves as a basis for budgeting, accounting, and reporting as well as for administrative control purposes, accountability to the Office of the State Comptroller and the general public, cost accounting, and the compilation of financial statistical data on the state level.11"
This framework organizes the accounting data about costs/expenditures and revenues according to general organizational programs, functions or divisions, and by objects of expenditure (e.g., salaries, equipment, etc.). This is sometimes called a function-object accounting or budgeting system. A typical function-object accounting system for a state agency has a highly detailed and structured way of defining what kind of financial information is to be recorded and how it is to be organized.
The strengths of a government accounting framework for analysis of costs and benefits are also its weaknesses, namely standardization and rigid structures and rules. Standards and rules make such a frame-work useful for control of finances, generating standardized reports, and overall fiscal accountability. And if the existing categories of accounts happen to provide the particular cost and return data needed for ROI analysis, the frame-work would be readily available and useful. However this is seldom the case, particularly for IT investments. The full value of an IT investment is typically a mixture of many types of expenditures and other costs, the accurate accounting of which is not possible in the traditional accounting system. These systems will usually record the costs of equipment and personnel attached directly and full-time to IT activities. But they seldom record the changes in personnel costs for those who use the technology or the investments in training and learning to operate new systems. Maintenance and supplies may appear in a range of separate accounts, and many managerial and analytical costs are very difficult to estimate or prorate accurately.
Measuring costs and cost-effectiveness
In its most basic economic sense, a cost is whatever you have to give up to get something else. And a return is anything good that you get as a result, whether that good is measured in financial terms or otherwise. This is, of course, a much broader view of costs and returns than is found in most accounting systems. For example, an accounting system would not likely include as part of the cost of a new computer system the amount of time devoted to the purchase decision. Nor would the accounting system record the benefits of increased staff morale from a more functional or reliable system.
A full consideration of costs requires attention both to opportunities and to indirect costs. An opportunity costframework takes into account what alternative actions or returns would be missed or foregone as a result of a particular investment decision. The path of any IT project necessarily excludes other paths not taken. If opportunities along those foregone paths can be identified and valued, they can become part of the cost calculation. If I choose to implement a new procurement processing system, for example, the staff time spent training for implementing the new system is time unavailable for training in some other area of work skills. If a value can be assigned, this lost opportunity may be considered part of the overall cost calculation. Indirect costs (sometime referred to as imputed costs) are those that are not incurred or measured directly, but are calculated or estimated from other measures. For example, the cost of using existing network cabling for a project could be calculated from the amount of new traffic generated or from some other prorating factor. Similarly, apartment renters do not pay real property taxes directly, but the renter’s portion of that tax can be calculated as an indirect component of rent.
These costs are often "off the books" in the sense that typical public accounting and financial management systems do not provide this kind of cost analysis. There are no standard accounting practices to guide the measurement of these costs. An agency may have an existing standard indirect cost factor that is used in budgeting for Federal grants or contracts (e.g., some fixed percentage of personnel expenditures). But these standard factors are based on averages over many projects and are not particularly useful measures for any one particular project. Opportunity costs are even more problematic and are often not included, leading to possibly significant underestimation of overall costs. Underestimating true costs can make an investment’s returns look more attractive than they may really be.
Assessing effectiveness requires identifying the outputs of the project and its implementation in business processes or program terms. That means identifying meaningful units of output that can be related to the cost estimates. For example, the effectiveness of a new system to process business permit applications over the Web could be measured in terms of increased numbers of permits processed during a given time. Lower costs per processed permit would be a useful cost/effectiveness indicator. Producing such a cost/ effectiveness measure requires detailed data about permit processing and a way of assigning costs on a per permit basis. Without some enhancement, most government accounting systems do not provide the basic data necessary for such a calculation. It may be necessary to do detailed data collection on a sample of permit transactions, for example, to establish a baseline unit cost figure. A business process model could provide the necessary data if so designed. Even if such a unit cost measure is available, it may have problems, such as the distorted assumption that all permits are equally important or costly.
Efficiency measures
Efficiency is a way of describing the effectiveness of a project or system in relationship to costs (or other inputs). Efficiency cannot be separated from effectiveness, since using resources and failing to achieve a desired outcome can be little more than waste; you don’t save money by building half a bridge. Efficiency is usually expressed in terms of optimizing the value of a return for a given cost or input, or alternatively minimizing the cost for a given value of result. It is possible, of course, to improve efficiency without necessarily achieving an optimum. As long as it is possible to compare cost/effectiveness or return ratios for alternative systems or methods it is possible to make judgments about efficiency. To demonstrate an optimum result or projection, it is necessary to have a simulation or optimization calculation to provide the data. Some operations research and simulations, such as linear program- ming or workflow simulations, can generate this type of calculation if the necessary models and data are available. However most IT projects do not have the data or analytical resources to include these methods in an ROI analysis.
Impact measures
The identification of variables that make up a social cost or benefit calculation along with their definitions is broader than both the economic and accounting frames. They are based either on the specific program results desired by an agency or on general social benefits and improved quality of life. These impact measures can come from several sources.
Operational data. Some useful measures of social or broader economic outcomes may be available in the data collected during ordinary program operations. Providers of services for homeless persons, for example, routinely collect data about programs and activities of clients that could be indicators of impact on their quality of life or progress toward independence. Similarly, law enforcement agencies routinely collect crime statistics that can be useful indicators of neighborhood climate or quality of life.
Social and economic statistics.Government agencies collect statistics on social and economic indicators that are relevant to the overall social and economic status of their jurisdiction. These range from the enormous resources of the US Census Bureau to state and regional planning agencies, and include market research statistics available from private sources.
Special studies and evaluations. For large, high cost projects separate data collection activities, such as surveys or field research, may be used to collect and analyze data about social and economic impacts of a project. These efforts may be expensive and time-consuming, but may also be the only way to obtain data about particular outcomes. The case example in Appendix C on Social Return on Investment provides some examples of how this may be done.
How time perspectives change the measurements
Time has important effects on the measurements and calculations. Both the costs and returns of an IT investment extend over time. Moreover, the costs may be incurred long before the returns are realized. So when estimating the money value of costs and returns, it may be necessary to take into account the effect of the time perspective on the value of money. That is, a dollar in hand today is worth more than the promise of a dollar in hand a year from now. How much less that future dollar is worth, known as the discount, depends on the current time value of money, which is usually called the interest or discount rate. Thus if the discount rate is 5 percent per year, then the promise of a dollar a year from now is worth only 95 cents today. The farther out in the future, or the larger the discount rate, the less the present value of the promise of that future dollar. Calculations based on the discounted value of future returns are useful to estimate what size of return is needed to justify the costs of an investment. The typical methods used in this kind of financial analysis are called Net Present Value and Internal Rate of Return calculations. Both methods use an initial cost and some assumed figures for a future stream of returns, along with a reasonable assumption for the interest (or discount) rate. The Net Present Value (NPV) calculation estimates the value today of returns coming in the future, less the cost of the investment. If the NPV result is less than the cost, then the investment does not appear to yield a net gain. The Internal Rate of Return (IRR) calculation yields the same sort of result by a different path. Using the same basic information as the NPV calculation the IRR calculation yields the rate of return for the investment assuming a break even, or NPV of zero. If the IRR is less than the current interest rate, this indicates that putting the investment in an interest bearing bank account would yield a bigger financial payoff than investing in the project. An example of NPV and IRR calculations are shown in Table 3 (page 26). This example is based on a hypothetical investment analyzed over a six year period. It shows that when the time value of future returns is taken into account, the total returns must exceed the total cost by a substantial margin to generate a positive net present value or internal rate of return.12
Table 3. Net Present Value Calculation
13
|
Year
|
Costs
|
Returns
|
|
1 |
$ 100,000 |
$ 15,000 |
|
2 |
$ 5,000 |
$ 20,000 |
|
3 |
$ 4,000 |
$ 25,000 |
|
4 |
$ 4,000 |
$ 25,000 |
|
5 |
$ 4,000 |
$ 30,000 |
|
6 |
$ 3,000 |
$ 35,000 |
|
Total |
$ 120,000 |
$ 150,000 |
|
Discount Rate
|
3.0%
|
|
Net Present Value
|
$ 13,297
|
|
Internal Rate of Return
|
6.0%
|
The problem of choosing the project or investment life cycle
Notice that any calculation of this sort requires specifying a time-frame within which the value of the project or investment will be assessed. This time frame is sometimes referred to as the life cycle of the project. Any IT investment is assumed to have a limited useful life, beyond which the returns are not considered, or when the expected returns from some new technology will lead to replacement of the previous investment. For IT investments, the rapid change in technology makes the choice of a reasonable life cycle or planning framework a difficult but necessary part of the analysis.
11
Office of the New York State Comptroller. Accounting and Reporting Manual. Albany, NY (no date), p.17.
12
The discount rate shown in these calculations was chosen for illustration only.
13
Calculates the net present value of an investment by using a discount rate (
rate
) and a series of future payments and income (
values
) in each period (
i
) for
n
periods.
