The best source of data about the costs and returns for this new Web architecture came directly from the staff’s experience. The Center staff needed an estimate of costs and returns for the XML conversion that they could trust as a valid representation of what they could expect in a full-fledged conversion, without actually implementing that conversion. To do this, they chose to develop a few relatively small new Web-based applications as pilot projects using the new methods and architecture. The first was a decision making guide to help organizations design electronic information access programs. The project was called "Gateways." The experience from those efforts provided the necessary information about costs and returns to inform the larger decision.

Technology Unit staff members responsible for Web site development and maintenance consisted of three full-time personnel and two half-time graduate assistants. All were individually interviewed weekly for a period of four months. During the interviews, members were asked to recall the work they had done the previous week. They were asked to identify:

• whether tasks were done in HTML or in XML,
• the amount of time spent in production ("doing" the work) versus learning the new application in order to do the work,
• the benefits of working with XML, and
• the drawbacks.

The data for the study also included information about the operation of the Web site and related activities, such as impacts on routine site maintenance, changes in content development procedures, and coordinating work on this activity with other technical tasks.

Weekly interviews were a low cost and relatively unobtrusive data collection method. They provided enough data so that it was not necessary to observe work directly or have the staff record their work times in activity logs. This method fit one purpose of the case: to be a useful example for others contemplating ROI analyses, employing methods that would be useful in a wide range of settings without requiring highly specialized training or high costs. The staff involved in the work agreed to the interviews and prepared for them as a regular weekly activity.