The benefit figures claimed in this analysis appear to be based on a rather optimistic scenario. The biggest return is cost avoidance due to diminished Federal penalties—a very large decrease between the 2001 and 2002 fiscal years. These penalty levels were very high for SFY 2001 and were expected to drop by over 90 percent in a single year due to the introduction of improved information technology. It is not clear from the available documentation how realistic these penalty reduction estimates were. A footnote to the ROI analysis report for the SFY 2001 project description reads:

"Avoidance benefits include $440,992 food stamp penalties, $5,600,000,000 potential Medicaid related losses, and $29,592,824 TANF penalties. Funding for the TANF penalties will be needed in SFY 2002 ($14,796,412) and in SFY 2003 ($14,796,412) plus Federal Match for food stamps and Medicaid in the amount of $1,264,384. There is additional potential for sanctions due to food stamp error rates. The amount of these sanctions is unknown."

The similar footnote in the FY2002 description contains essentially the same estimates for all the other savings, but the potential Medicaid related losses drop from $5.6 billion to $500 million. This suggests that estimates of this sort are subject to considerable uncertainty and may not be the best basis for an investment decision without additional supporting data.

Such optimistic estimates to justify a project proposal are not unusual. In order to win the resource competition, it is tempting for agencies to assume the worst scenario for not implementing the proposed project, compared to the best case prediction for completing the project. That way agencies can show very dramatic and persuasive returns for reviewers. Decision makers have to find a reasonable balancing point between the two extremes. Related information needed to make more reasonable assumptions may not be available in the proposals, if it is not required. Hence, the evaluation process is usually problematic and critical. In this Iowa Health and Human service case, reviewers might need more detailed information about the process of savings calculation in order to make an accurate judgment. That is, the proposers of a new investment may deliberately skew their calculations to make a stronger case than they could otherwise justify. Reviewers may not be able to detect such deliberate exaggerations or unreasonable assumptions unless they have full information about how calculations were made. ROI calculations are products of social, political, and economic interest that are often in conflict with each other.¹⁷ The complexity behind the numbers and calculation processes should always be part of the overall decision making process.

Resources: