Tying a Sensible Knot: A Practical Guide to State-Local Information Systems:

Chapter 3. Best Practices

Choose a capable pilot site

Many system implementations are initiated with pilot tests that bring the system into the field to evaluate and refine design, performance, and integration with other systems and activities. The pilot site (there may be more than one) is a critical organization Œ one that is willing to undergo on-the-spot evaluation and identify and work on the inevitable problems that pilots are created to uncover and resolve. The pilot site provides the system developers with a way to evaluate the initial release of the system in a controlled environment and, if necessary, make any changes before releasing the system to all users. Many of the projects we reviewed involve one or more local pilot sites to test and refine their systems. In some cases, the pilot site was an integral part of the development team, building local needs and practical limitations into the initial design and then serving as the initial implementation site.

Sometimes pilots are promoted as a way to get special attention and early implementation for a new system. While this is true, it is only half the story. In return for these benefits, the pilot organization has to commit staff, space, and other resources to a process whose goal is to find and fix problems. Although it has clear benefits, piloting can be frustrating, time-consuming, and disruptive.

A number of considerations go into the selection of a pilot site. Here are few of the more important ones:

Representativeness. Will the site(s) you are considering give realistic results for guiding broader implementation? If there is a great deal of local variation, you may need several pilot sites that represent broad categories of local conditions.
Organizational capacity to carry out the pilot.
Does the pilot site have the staff, space, equipment and other resources needed to carry out the pilot without adversely affecting its ongoing operations? Do the pilot site staff have realistic expectations about their roles and the amount and kind of assistance they can expect from developers or consultants?
Leadership commitment to the project and to the special demands of pilot status. Do senior managers in the pilot site believe in the goals of the project? Are they ready and willing to deal with unexpected problems? Will they give their staff the support they need to carry out both pilot activities and regular operations? Will they act as liaisons to local officials in other departments who may be affected by the pilot or eventual operation?
Geographic accessibility. Is the pilot located in a relatively convenient place so travel costs and time can be minimized for the project staff who need to be on-site and for the pilot staff who need to travel for training or other project activities? Try to avoid a pilot site whose location discourages on-site technical assistance, monitoring, and evaluation. For example, does it always require an overnight stay or several hours of driving, or more than one mode of transportation to get there?

SALESNET: This is a make-or-break relationship

The technology staff at the NYS Office of Real Property Services knew that pilot testing their SALESNET system would be critical to its successful implementation. Through surveys and discussions with county officials and information technology directors throughout the state, the SALESNET team chose a county that seemed ready and willing to test the Internet-based sales reporting system.

Before long though, it became clear that not all the primary stakeholders in the county had a strong commitment to the project, and some of the secondary stakeholders were reluctant. Local officials expressed concern over control issues and political concerns about having a central database on the state level. The SALESNET developers were wise enough to determine that this county was not ideal as a test site, and they started over to explore testing in another county. They knew that full cooperation in the original site was unlikely. Despite the added trouble, energy, and staff investment, a new county was chosen as the pilot site. Through the SALESNET project we learned that a pilot site must be chosen carefully, using the following criteria:

primary stakeholders must be committed to, not just interested in, testing
peripheral stakeholders must have an interest in the test
there must be an abiding interest in the project that will last the duration of the pilot period
the political climate must be open enough to permit some realignment of information and responsibility

Social Services Imaging Project: What to look for in a pilot site

The Department of Social Services SSIS staff (Social Services Information Systems) believed strongly in the importance of selecting a good pilot site for their Local DSS Imaging Project. Selecting a good site would add value to their project by providing useful feedback about the fledgling system and by letting other sites see and hear about the benefits of the project. The problem was, which site should be selected?

The team did its homework to ensure that the pilot site would contribute to the success of the project. After considering the criteria for the selection of a site, and the various pros and cons associated with each site, the LDSS team made its decision. The selection of their pilot site was based on three important factors:

What site had the best technology environment to act as a pilot?
What site had strong leadership and good relations with employees?
And finally, what site would provide the most positive impact for the project once the piloting work was complete?

The implementation of the system at the pilot site yielded solid information for the project team about the system and about the implementation process itself. The illumination of both technical shortcomings associated with the system and process issues related to implementation provided the team with vital information that could not have been discovered otherwise. Due to the success achieved at this pilot site and other early participating sites, additional counties have expressed a desire to participate in the project.

< Previous |

| Next >