The range and complexity of government information technology (IT) investments makes assessing their returns a daunting prospect. Projects can range from systemwide transformations, to improving financial transparency, to more efficient dog licensing. The returns may be large or small, obvious or obscure, and can run from a few minutes saved in a routine transaction to improving the trust and legitimacy of an entire government. In spite of the difficulty in assessment, however, knowledge about public returns can be vital to fully informed and justified IT investment decisions.

Assessing these returns remains a core problem in IT planning and decision making. That problem results from shortcomings in the available methods and models for assessing public returns, what we call public return on investment (ROI). In looking over these existing methods and models we saw three significant shortcomings:

• Incomplete analysis of public value, resulting in too narrow a scope of what can be considered returns to the public.
• Lack of systematic attention to how government IT investments generate results of value from the point of view of the public.
• Weak or absent methods for tailoring a public ROI assessment to the specific context and goals of a government IT investment..

Existing methods also deal with risks to government IT investment, but primarily from the point of view of technology development and implementation. The risks that involve the public beneficiaries of the investments merit more attention.

This white paper presents a public ROI assessment framework that addresses these shortcomings. We call it a public value framework to emphasize the point of view of the public, not the government, as the basis for the assessment. The framework describes how to identify and assess public value through the kinds of activities shown below in Figure 1.

The framework's strategy is simple in concept, but complex in application: connect what happens in the government (on the left) with the impacts on stakeholders in the public domain (to the right), then report and apply the results. The activities on the left identify the potential value mechanisms and outcome goals. Those are linked to stakeholder interests, impacts, and risks to the right. The curved arrows indicate that in practice the process would seldom be linear, requiring reflection and backtracking to adjust for learning and new information. The full paper presents a detailed version of this schematic, showing the links among these activities.

Figure 1. Public Value Framework

The public value proposition takes center stage. This value proposition must be broadly conceived to do justice to the scope of government and how it affects individuals, groups, and both public and private organizations. This framework presents a new and more comprehensive way of describing public value, based on six kinds of impacts government IT can have on the interests of public stakeholders:

• Financial — impacts on current oranticipated income, asset values, liabilities, entitlements, and other aspects of wealth or risks to any of the above.
• Political — impacts on personal or corporate influence on government actions or policy, role in political affairs, or influence in political parties or prospects for current of future public office.
• Social — impacts on family or communityrelationships, social mobility, status, and identity.
• Strategic — impacts on economic or political advantage or opportunities, goals, resources for innovation or planning.
• Ideological — impacts on beliefs, moral or ethical commitments, alignment of government actions or policies or social outcomes with beliefs, or moral or ethical positions.
• Stewardship — impacts on the public's view of government officials as faithful stewards or guardians of the value of the government itself in terms of public trust, integrity, and legitimacy.

Expanding the view of stakeholder interests in this way brings into focus two distinct but equally important types of public value: the delivery of benefits directly to citizens and enhancing the value of government itself as a public asset. An IT investment that makes government more transparent, more just, and a better steward has added public value, a non-financial but nonetheless important return. This framework describes how to include both in public value assessments.

The framework also identifies the basic ways government IT investments link to public value. The simplest link results from an IT investment embedded directly in a service delivery process (Figure 2:A) in a way that enhances its value to citizens or officials. The total value may be a composite of several separate measures: cost savings, quality, enhanced trust. An indirect link (Figure 2:B) can result from infrastructure improvements to business processes.

Figure 2. Direct and Indirect Links to Public Value

Figure 3. Mix of Direct and Indirect Links to Public Value

This link may be more difficult to trace and can involve more risk. Enterprise Resource Planning (ERP) software implementations in our case studies are excellent examples of this kind of investment. Value can also result from a mix of direct and indirect links (Figure 3) when new IT changes the environment.

Identifying these links is just part of the story. Each link can involve several value- generating mechanisms. The framework describes how to trace these mechanisms working through specific business processes to produce different kinds of value. The value-generating mechanisms are

• Increases in efficiency — obtaining increased outputs or goal attainment with the same resources, or obtaining the same outputs or goals with lower resource consumption. In our Austrian and Pennsylvania case studies, for example, new ERP systems helped achieve substantial efficiencies in financial management.
• Increases in effectiveness — increasing the quality and/or quantity of the desirable thing. Our case study of Service New Brunswick, for example, reports how an online registry for land data can contribute to improvements in property tax administration.
• Enablement — providing means or allowing otherwise infeasible or prohibited desirable activity, or preventing or reducing undesirable events or outcomes. In our Washington State Digital Archives case study, for example, putting birth and marriage records online enabled research by local historians and genealogists.
• Intrinsic enhancements — changing the environment or circumstances of a stakeholder in ways that are valued for their own sake. For example, our Israel case showed how enhanced financial accounting and reporting in the Merkava ERP system opened government financial decision making to greater transparency.

An IT investment project can deliver public value through any or all of these mechanisms.

The framework is deliberately presented at a moderate level of generality to make it most widely useful. Every government IT project will have its own unique goals, value propositions, and stakeholders. So this framework can be used to plan and guide a public value assessment, in combination with measurements, analysis tools, and reporting techniques chosen for the specific situation. To aid in this regard, the paper presents an overview of more detailed and highly specified assessment methods that can be used in conjunction with this framework. This includes a more detailed flow chart for the assessment process, a summary of several ROI models and methods, and suggestions for analysis and reporting of results.