Implications for Public ROI Assessment
Our approach to public ROI assessment looks at two main ways by which IT investments return value to the public: by improving the value of the government itself from the perspective of the citizens, and by delivering benefits directly to persons, groups, or the public at large. This case shows an investment that has generated substantial public value in the first sense, but less clearly so in the second, at least at the present time. It is clear that this particular IT investment has increased the value of the Austrian Federal Government from the point of view of the public. As a result of the investment, government is much more capable in financial management, in making wise use of resources, and in providing information for financial transparency and accountability. While difficult to quantify, these returns are meaningful and form an important part of the overall value proposition. In a sense, government can be thought of as a tool that societies invent and fashion to create value in terms of services and other cultural and political activities. An investment that improves the “tool” in some significant way can be said to return value, even if the direct connection to a return at the level of the individual citizen can not be measured. Since the investment in this Austrian case is so substantial and the returns so diffused throughout the government, more attention to this issue seems appropriate.
Beyond the financial metrics for efficiency and savings readily available in this case, it would be useful to consider how the increment in this “asset” value of the government can be described. The question rests in part on how to describe what the overall asset value of a government could be. The price of government services (i.e., taxes and fees) does not provide much guidance, since much of the services one “purchases” in this way are not priced via a market and are public goods in the economic sense. If the Austrian Government reduced taxes each year by the exact amount saved, the return to the public in money terms would be clear, but this is neither likely nor possible; the value of a “saving” that is simply applied to some alternative use is not a true cost reduction. A sufficiently sophisticated accounting system could identify the added value or savings from the “infrastructure” value of the ERP in supporting improved services in various government areas, but that sort of data is not available in this case. Moreover, the necessary baseline data on service costs and quality were not collected at the beginning of the project.
Alternatively, surveys of the population could reveal some insight into how the internal improvements are valued by citizens. This could yield estimates of value qualitative terms, such as a rating of government performance or “excellence” on some arbitrary scale. Since the ERP system has the capability to add value across the government, such a survey could elicit opinions about the performance of government across many programs and service types. In some service areas more objective measures of performance may be available, such as air quality indicators for environmental protection agencies, test scores for education agencies, or crime statistics for law enforcement. However, government agency actions are usually just one factor among many affecting these indicators, making the assessment quite problematic.
It is also quite expensive and conceptually difficult to assemble all the necessary data to assess an investment with such broad reach in government. It was not necessary for the government decision makers in this case to ask for or seek public value data to justify the investment. At this point in the development of the new system, the internal efficiency case has been sufficient. In one sense, therefore, an opportunity was lost to collect baseline data and attempt to track broader returns in terms of service quality or quality of life indicators. For further development of these systems, however, faith in the efficacy of the ERP system as enabling performance improvement generally may require more data. The Austrian e-government strategy is a comprehensive one, of which the underlying ERP capability is a potentially important part. It would be useful, therefore, to have baseline data about service levels, quality indicators, costs, and public opinion data for areas of claimed improvements. These would support the ongoing investment in this and other potentially valuable new IT tools.