The selection of method will depend to some degree on the time frame and the scale of the project and the ROI analysis. Time frame refers to both the time perspective for the analysis and the overall time period of the project to be included in the analysis framework.

Anticipating the future. ROI analysis is commonly used prospectively. The results of the analysis are intended to inform a decision about a future IT investment. For a prospective analysis, estimates of anticipated cost and performance are based on assumptions about the future that may involve considerable uncertainty. The results of the analysis will depend, to a large degree, on how accurate those assumptions turn out to be.

Learning from the past. A retrospective ROI analysis will show actual performance data about the IT project’s costs and returns. The longer timeline and complexity of larger projects can lead to substantially different results. The analyst must take these possibilities into account when evaluating the results of pilot projects. These concerns become part of the risk analysis discussed in a later section.

Importance of scale. Even when there are actual data from such a pilot project, the problem of the accuracy of assumptions remains. There is no guarantee that costs or returns from a small pilot project will accurately predict what happens in a larger effort. The scale of the larger effort can itself lead to different results.

In the case study described in Appendix A (page 28), the analysis combines the two perspectives. The developers created a pilot project, which was a small part of a larger project they planned to undertake. They then used the data gathered from a retrospective analysis of the pilot project to estimate the costs and returns that would result from the full development.