This section presents some exemplary practices used in eleven state-local government information systems projects in NYS. In each section we discuss one kind of practice that contributes to successful state-local information systems, outlining some specific things that project participants can do to help reach their goals. Each best practice narrative is followed by examples of various approaches used in these projects to achieve the desired result. The examples are not prescriptions. They are intended to demonstrate how good managers adapted these concepts to the specific needs of their projects.

Each state-local system project requires a somewhat different mix of these practices to guide it to a successful conclusion. These practices are presented in a logical order of first consideration. However, we stress that these are practices, not steps. A traditional way of thinking about a project is that a number of steps need to be completed in order to reach the project’s goals. If that kind of thinking could be captured in a picture, it might look like this chart:

While this kind of thinking is useful and important for managing activities, we urge you to think of these best practices, not as steps, but as ongoing areas of attention that exist throughout the project. The level of intensity that any one practice commands at any point in time will vary. For example, the amount of attention you give to defining the project purpose and scope will be very high early in the project and then take a back seat to other considerations - but it will not disappear. The first definitions of purpose and scope will be revised and refined as you and your partners learn more about the problem you are solving and the resources at your disposal. Even after the purpose and scope seem fine-tuned, there are likely to be new participants or new audiences who need to understand and accept it. This kind of thinking would look more like the chart below.

Keep these differences in mind as you move through this chapter.

Projects are initiated in a variety of ways. Some result from policy changes, new legal mandates, or changes in elected office holders. Others emerge from grassroots discussions about the need to change, advance, or simplify a process, take advantage of a new technology, or factor in a new information requirement. Regardless of the motivating factor, a well-defined project purpose and scope are instrumental to success. Defining purpose and scope means resisting the lure of the "end all" project and relying instead on realistic incrementalism. We would all like to create the system that addresses all of the information and service delivery needs of state and local agencies. However, to be realistic and successful these needs must be identified, discussed, categorized, and prioritized. The huge range of programmatic issues must be culled for a project purpose and scope that are consistent with those priorities. Further, these needs must be analyzed against the resources that are likely to be available. Ideally, the selected purpose and scope not only attack current problems, but lay a foundation or build capacity to deal with future ones.

State and local participants must work together to identify the information and service delivery needs of a particular program area. Participants from the eleven state-local projects in this study used professional meetings, association conferences, and regular meetings with state agency regional representatives to carry out the needed discussions. State and local participants both saw these meetings as opportunities to discuss program needs and to establish working groups able and willing to participate in a project to address shared goals.

Resources are a key factor in decisions about project scope. The projects we reviewed were typically volunteer efforts at the local level and subject to restricted funding at the state level. Even when both state and local participants are convinced of the overall value of a specific project purpose, they are often unable to bring enough resources to the table to support a broad scope of work. Therefore, collectively prioritizing needs and collaboratively working toward a scope that is appropriate for the available resources serves all participants well. Communication skills, creative funding, and effectively managing existing resources all figure prominently in this stage of the project activities.

Probation Automation: Focus on core functions

The criminal justice community recognizes that the county-level Probation function is a critical piece of the public safety puzzle, but one that has typically been left behind in the technological advances that have become standard in the rest of the community. While larger counties with MIS departments have managed to support their Probation offices fairly well, small and medium- sized operations (50 of the 57 in the state) can offer little to help Probation officers do their jobs. In order to identify the best focus for this project, a design team consisting of both state and local officials examined the current functions performed by all 57 County Probation Departments and New York City. The team interviewed Division of Probation and Corrections Alternatives senior staff to identify the functions of the Probation Departments. Based on these interviews the functions were categorized into three levels:

• Level I encompassed mission critical probation core functions (such as criminal court investigations and supervision; family court intake, investigation and supervision; and basic administrative functions like restitution collections).
• Level II covers alternatives to incarceration such as community service and house detention.
• Level III functions involve external treatment providers such as mental health and drug and alcohol treatment services.

An analysis of Probation functions across the entire state revealed that all local departments perform Level I activities, but Level II and III activities are more specialized and not evenly distributed. This categorization was reviewed and affirmed by those interviewed and continues to be reviewed in the project activities. Since the project’s purpose is to provide one standard support system, the scope of effort was limited to support Level I functions. All participants agree that this focus will generate the greatest return on investment in the system.

Projects often begin with open-minded brainstorming sessions aimed at garnering as many solutions as possible. It is important in this early phase of idea generation to be relatively unburdened by real-world restraints in order to maximize the number of ideas and potential solutions produced. Once ideas are generated, the team must choose from among the possible solutions and evaluate each using such factors as alignment with project purpose, cost, benefit, skill level required, time requirements, and ability to integrate with other systems.

The following steps very briefly outline a procedure which can help your team establish and stick with realistic expectations:

• Prioritize project goals
• Identify resources - funds, time,people, technologies
• Consider time constraints - legal requirements, timing for maximum impact, budget cycles, elections
• Generate a wide range of potential solutions
• Choose those solutions which can best support project goals while staying within resource and time limits
• Identify measurable performance factors within those solutions
• Map out an implementation plan; assign responsibilities and chart project milestones on a timeline
• Create a budget
• Monitor and manage the project over time
• Discuss progress with the team regularly and adjust the project plan as needed

Immunization Information: Compelling purpose is a strong incentive
The NYS Immunization Information System (NYSIIS) project operates under state law and a grant from the US Centers for Disease Control (CDC). It comprises four demonstration projects around the state involving the voluntary participation of counties, physicians, and other health care providers. The demonstrations are testing the feasibility of a statewide registry for tracking and monitoring the immunization of children. The demonstration sites have wide discretion in how they set up their systems and forge partnerships with both public and private participants. The State Health Department hired a system integrator to work with each site to design a system that suits the needs and capabilities of that community. The ultimate goal is to increase the rate of fully immunized children in New York State through an electronic recordkeeping process that enables health care providers to track and recall children to ensure that they are age-appropriately immunized, and allows public health officials to assess the immunization status and issues in their communities.

The project is a difficult one for several reasons: it deals with highly confidential information, it needs to be integrated into the existing systems and practices of thousands of physicians and other practitioners, and none of the players is required to participate. The demonstration sites need to sell the project to health care providers and heath agencies in an environment where busy practitioners with existing (mostly proprietary) business systems have few dollars to spend on new systems and little interest in systems that deal with a single issue like immunization. The project leaders at both the state and local levels have worked hard to bring all parties to the table and address these issues in each site, but one factor stands out as a reason why participants become and stay engaged in this project: The purpose, (healthier children) and the project focus, (ensuring full immunization against communicable diseases) are clear, compelling, and widely endorsed. Since no other state has tackled this problem in a way that will serve New York’s needs, the participants look on themselves as pioneers and have a real stake in its success.

In virtually every project interview, we heard about the need for solid, consistent, positive leadership. Leadership was viewed as setting the stage for a project and ensuring timely and meaningful completion. Yet, the personal nature of leadership and our individual desire to be effective leaders often leads us to deal with it in abstract terms. The projects we studied, however, showed how specific leadership traits help produce successful outcomes. A successful project leader:

• Is able to span the psychological and political distance between state and local governments. The project leader should possess an understanding of both state and local needs and capabilities - and be able to balance them. This balance is crucial to the success of state-local projects since buy-in and cooperation are two of the fundamental aspects of a successful project.
• Has a good understanding of local operations. Since these systems actually run at the local level, a project leader needs to appreciate the reality of local operating conditions. Some successful leaders had experience as both state and local officials, others spent time in their careers working in field or regional offices of state agencies, and still others who did not have these kinds of work experience made it their business to understand local needs and operations from the local point of view.
• Enjoys the confidence and support of top-level executives. Getting and keeping top leadership support is the best way to keep a project on the front burner. It is essential to maintaining re- sources and to competing well against other government goals. Support from state level leaders gives local participants confidence in the project. Support from local level leaders helps ensure full participation and joint problem solving. Successful project leaders delivered on realistic expectations and kept their top executives well-informed and enthusiastic by communicating in terms that executive leaders value: return on investment, partnerships, options, early warnings and so on.
• Is an excellent communicator. Project Leaders must be able to articulate project goals, explain how they will be achieved, and show how the goals will benefit all the stake- holders. They need to speak the languages of different audiences and provide the right information in the right format to meet those different needs. The leader needs to know when to use formal presentations, newsletter articles, fact sheets, briefing papers, and other methods of communication. Perhaps more important, the project leader needs to be a good listener and adept at encouraging others to communicate their needs and ideas.
• Is a resourceful manager of people, time, and money. The project leader is responsible for the effective use of project resources. Often he or she is also the person who identifies and encourages others to commit staff, money, or time to a project. Since it is unlikely that a project will be fully funded from one source, project leaders need to be entrepreneurial, inventive, and resourceful.
• Is flexible and willing to seize opportunities. Successful project leaders have a clear vision of where they want to go, but are quite willing to try a variety of ways to get there. In some cases, they started with one kind of approach, but later modified, or even abandoned it, in order to solve problems or take advantage of a different perspective. They also had the ability to know when the time was right to act, even when they had imperfect information or scarce resources. They recognized when key factors in the environment were ripe for change and capitalized on them in order to move their projects forward.

Automated Dog Licensing: Leadership communication makes a difference
A project leader can make the difference in the enthusiastic adoption of a new information system, especially one that’s entirely voluntary on the part of local participants. In New York’s development of a system for the Electronic Transfer of Dog Licenses, the project leader was by all accounts an excellent communicator and salesperson.
She established a monthly division newsletter called "Dog Tales" to keep the Town Clerks informed, and, according to one project participant, the newsletter "helped to generate support, interest, and participation in the project." Everyone potentially affected by the new system received a copy, so there were no communication gaps while development was underway. One local official said that sometimes in dealing with the state, local governments find out about procedural changes after the fact. This project was different, he said, because the leader asked for local advice all along the way, and worked hard to make sure that the project would allow local governments to use familiar technology. Another participant said that the leader was honest and open, assuring everyone that there was no "hidden agenda" - only an attempt to save money while ensuring access to quality data.

Annual Financial Reports: Leaders bring out the best in each participant
In a project to automate the filing of local annual financial reports, project leaders at the Office of the State Comptroller (OSC) worked hard to bring out the strengths of each participant. Time and again the local participants commented that the work they were asked to do was perfectly suited to their abilities. This meant that local participants were answering questions concerning functionality and were commenting on design characteristics rather than being concerned with the technical system design and software considerations. This demonstrated a leadership attribute that is essential to the successful implementation of an intergovernmental project - understanding the roles and capabilities of each participant, appreciating the limits of their time and energy, and then involving them in the areas where they are most needed. The work accomplished outside of the state-local meetings was focused on coordinating state agency efforts to apply technology to implement all that had been discussed with the local participants. Hence, when OSC and the local representatives reconvened, the technical considerations of previous proposals had been worked out, new products were on the table for discussion, and the meeting could again focus on the reactions and concerns of the local participants.
OSC project leaders also ensured a very professional work-oriented atmosphere. Meetings were held in off-site facilities that were appropriate for the kind of work being conducted. Trained facilitators were used in all meetings. Food and refreshments were provided at each meeting. Local officials felt these were tangible ways of showing that OSC cared enough to take the time to do the project "right" and to treat them as equals. Local participants commented that this level of preparation demonstrated not only concern on the part of OSC but also proved the project had considerable top management support, something which gave them even more confidence that the project would proceed to a successful conclusion.

The success of any intergovernmental information systems project generally depends on three factors working together: technology, management, and policy. If any of these areas are ignored in staffing a project team, the project is likely to have either short or long term problems or both. Without individuals capable of handling project management functions (timelines, workplans, budgets, recruiting) you run the risk of poor coordination, and wasted time and effort. If a project lacks adequately skilled technology personnel, it is likely that deadlines will be missed and applications may fail or contain crucial flaws that render the system inferior to the old way of doing business. Teams that do not include well-informed program and policy staff, especially those engaged in direct service functions, are likely to miss the boat on substantive service goals.

Moreover, the project team needs both state and local membership and the roles assigned to each person should take advantage of that individual’s organizational location and professional background and skills. A survey we conducted as part of our study showed clearly that all participants had greater confidence in success when local officials played active roles as lead or co-designers. Local officials have the experience to understand the daily operational needs of any new project. They understand the street-level realities. As such, the early, active, and ongoing involvement of local government partners adds considerable value and ensures more complete success. It is also important to establish at the outset any limitations, such as travel time and costs, on local agency ability to participate.

Social Services Imaging Project:Clarify the nature of participation
Establishing a project team with the right mix of program skills and technical understanding of the potential uses of imaging technology was a critical step in the Local Social Services District Imaging Project. To encourage local involvement, the State Department of Social Services (DSS) sent a letter to each County Social Services Commissioner inviting the Local Social Services Office to participate in the project. In order to assist the Commissioner in this decision, the letter included detailed information about the desired nature of local involvement. Selected counties were invited to participate in one of two groups: those counties which had been working with the State DSS on imaging related tasks were invited to participate in the pilot activities, while those who had begun to evaluate the role of imaging in their business processes or information access methods were invited to sit on an advisory board.
The invitation letter included the following project details:
This approach resulted in the formation of a project team which was fully aware of the roles and commitment they had taken on, as well as a team with the necessary programmatic and technology skills to carry out the work. It also generated the necessary top management understanding and support for the project.

As a group the project team will need to set objectives, garner the support of many organizations, plan, design, test, implement, promote, and evaluate the system. Team members should be selected both for the skills they can contribute toward project completion and for the commitment they bring on behalf of their organizations.It is most likely that different team members will need to be involved at different levels of intensity. One approach is to assemble both a core working group and an advisory group. These can then be supplemented when necessary by outside consult- ants or contractors.

Using stakeholder analyses and similar tools at the inception of a project can greatly enhance your ability to identify and consider all the parties that may need to be represented on the project team and ensure they have a direct or representative voice in its development.

Electronic Death Certificate: Prepare the way with a business case
The Electronic Death Certificate Project faced a formidable initial challenge - convincing and securing the participation of all the parties involved in the finalization of a death record. This group of critical players includes physicians, medical examiners, funeral home directors, and local government vital statistics registrars. Beyond this core group, many state and federal agencies, the courts, and insurance companies have a keen interest in the timely and legal verification of deaths. Technology-based improvements in the current paper-bound process needed the support and participation of all parties involved.
Despite the variety of stakeholders and the complexity of their interactions, the project initiators at the State Health Department and the New York City Registrar of Vital Statistics were successful in getting all parities to sit down at the table and begin mapping out their requirements. They accomplished this by first making a compelling business case that the project would benefit each participant. The case was documented in white papers that were widely distributed prior to any formal meetings. For example, the business case for NYC Registrars showed how a networked system would reduce time, cost, and errors. Each borough in NYC used to have a satellite office for vital records where business related to the people living in the borough could be conducted. Due to budget cuts these offices were closed several years ago. Now when funeral directors need an official signature on a burial permit they must travel to the central Registrar in Manhattan to obtain it. Even under ideal conditions, this trip can take several hours. When you consider there are more than 70,000 deaths a year in New York City and that most people live outside Manhattan, this makes for a considerable workload for all the funeral directors in the area. The white paper showed how network technology, including digital signatures, could allow them to obtain the needed authorizations without leaving their businesses.
These papers, with their focus on benefits to stakeholders, generated a great deal of interest in the project and willingness to attend the initial planning meeting. At that session, participants were already well-informed and able to focus on key issues such as protecting confidentiality and adopting electronic signatures that would enable them to achieve these benefits.

"A successful project team needs policy, management, and technology experts with both state and local perspectives."

Reasons for developing information systems vary from one project to the next. Some projects are designed to take advantage of new technology, others to improve delivery of services to customers, and still others to improve business processes or reduce costs. Although the reasons differ, the need to "sell" the project to decision makers is universal. This is a particularly important consideration for local governments, which often have to work with very small budgets and much competition for limited resources. Moreover, in an intergovernmental project, the "selling" process needs to be a coordinated effort that involves individuals from both the state and local levels. Very often the project objectives and expected benefits are not uniformly understood by all of the project stakeholders. A good way to establish common understanding, market the project to decision makers, and generate consistent support is to articulate a shared vision at the beginning of the project. This vision (written down and used consistently in important project documents and events) communicates to all stakeholders important information about why the project is being undertaken, what the expected goals are, and how the realization of these goals will benefit the various stakeholders.

There is always some cost involved in automation or new information systems, and some of those costs must be borne at the local level. Local investments need to be tied to local benefits. It is easy to show how a new road or sewer system will benefit a community. It is much more difficult to show how a new information system will do the same. Since local authorities need to make trade-offs among competing demands for resources, they, like all other investors, put their money (or time, or staff, or good will) where there is the greatest potential for real returns.

Aging Services System:Establish a common vision
When planning the Aging Network Client Based Service Management System Project (CBS), the NYS Office of the Aging (SOFA) together with the 59 Area Agencies on Aging (AAAs) recognized that for the project to be successful it had to be "sold" to key decision makers at both state and local levels of government. SOFA worked with the 59 AAAs to create a clear vision for the project: "The goal of the CBS project is to restructure the Aging services network of New York State into an integrated, automated environment that supports the independence of older persons living in the State."

The project team members shared a clear and common understanding of the need for the project and its goals. These members in turn were able to communicate this information to the decision makers in their organizations to allow them to understand the benefits and goals of the project and thus gain their support. The project team identified key state and local stakeholders whose support was critical to the success of the project, including program management and technical staff and county level administrators, and developed an approach to communicating with these individuals about the project vision. They explained to the administrators how the CBS system would improve coordination of service delivery to clients by building on available family supports and ensuring that the service delivery system provided only essential services and programs thereby ultimately increasing the efficiency and effectiveness of local Aging offices.

When the choice is between a new road and an information system, it is often tempting for the city council or village authorities to vote for the road and bypass the new system. No matter how improved or elegant the new system may be, it must compete with projects whose benefits are more tangible and whose success is easier to measure. Local officials told us that if they are consulted in the earliest stages of a system design, they can give advice that will make the system more attractive to local decision makers and help make the case for local investment. Securing top management participation in a project up-front can be difficult and usually adds time to the startup phase of projects, but it goes a long way to ensure successful project completion.

Annual Financial Reports: Focus on fundamentals
The Annual Financial Reports project team enjoyed a simplified task of selling their project to top management since the project was designed to simplify a core function at the Office of the State Comptroller (OSC) and a fundamental legal requirement of local participants. Not all projects have this luxury, but the closer a project comes to supporting the "core" functions of both parties in the intergovernmental arena, the more support it is likely to get from top executives.

However, in this project, the team did not stop at the focus on "core" needs. They also encouraged top-level state managers to attend the meetings with local participants so they were informed first hand and stayed aware of the project goals and progress. How did they get top management to attend? By starting early and emphasizing the "core" nature of the project. The project leaders took advantage of the fact that OSC was in the midst of several quality improvement exercises and thus, everyone was more aware of the need "to get involved." This made project progress much easier since it was not a constant battle to get top management attention. The participation by OSC decision makers also sent clear messages to local participants that this project was important and would be supported and completed by the state agency - something local governments have come to doubt from many past projects.

"Local investment needs to be tied to local benefits."

Regardless of size and type of application, a project team operating in today’s intergovernmental environment is faced with a multitude of stakeholder relations issues. Project stakeholders have not only a desire but a genuine need to know what is taking place within a project. Good communication practices ensure that all stakeholders (both those actively involved and those who will eventually be affected) are continuously and adequately informed. In addition to communicating with stakeholders during the initial stages of project development, it is important to continue to communicate throughout the entire process to make sure that everyone is aware of and given opportunity to comment on and participate in the project activity. Just as important are good working relationships that encourage stakeholders to participate actively in giving and receiving information. Many techniques may be used to establish and maintain good communications: status meetings, distribution of printed and electronic project materials, formal presentations, and so on. The techniques selected should be based on the particulars of the project and the following factors:

• Who are the project stakeholders?
• What type of information do they require at what level of detail?
• What type and level of information is needed from them?
• How frequently do they want or need information?
• How frequently is information needed from them?
• What is the most useful way for them to receive information?
• What type of feedback mechanisms are necessary to encourage them to respond and react?
• What tools need to be used to continuously monitor the effectiveness of communication?

By answering these questions, the selection of communication techniques becomes a much easier task that generates more reliable and useful results. For project team members, communications tend to be quite frequent and detailed involving e-mail and phone messages, status reports, flow diagrams, and face-to-face and electronic meetings. For the broader community of stakeholders, they are more likely to involve briefing sheets, newsletter articles, and presentations at conferences. The important thing to remember is to keep information flowing continuously to keep everyone focused on the project goals and aware of the progress being made.

Automated Dog Licensing:Networks connect people as well as computers

There were plenty of benefits in the Electronic Transfer of Dog Licenses Project that made work easier and faster, and improved the quality of the data. One unexpected benefit, according to the team leader was the extent of "networking with municipal licensing agents and the software vendors." This networking was not accidental, however, but the result of consistent and careful communication that connected all the stakeholders in the project and provided opportunities for them to become invested from the very earliest stages of system development. The municipal clerks told us that they were members of the team whose opinions clearly mattered.

The team leader knew that the state sent notices occasionally to the municipal license issuers, so she used the mailing as a vehicle for a newsletter that kept all municipal agents informed about changes in the licensing process. She set certain standards for the newsletters. Consistent with the theme of the project, the newsletters aimed at simplicity. They had to:

• contain real information of interest to the municipal officials
• be short - only one page front and back
• be clear and concise, not detailed - other vehicles were used for intricate instructions
• have new information, not old recycled information

Once the system was implemented, a brochure was developed to explain the system and how it works, as well as the benefits for local licensing agents. The brochure was designed to answer one question posed by a municipal clerk: What do I need to know about this new way of reporting dog license information?

Throughout the project the state team also stayed in touch with the Clerks and vendors of municipal software packages by phone. The first calls to vendors about adapting to this new system met with a lukewarm response, but the team, particularly the team leader, was persistent. Development team members attended the statewide municipal clerks meeting where they organized a special session for vendors which eventually led several to adapt their products to support the new system.

Probation Automation: Frequent, timely interaction among peers

The Probation Automation Team consists of seven County Probation Directors or their designees as well as both program and technical staff of the NYS Division of Criminal Justice Services (DCJS) and the NYS Division of Probation and Correctional Alternatives (DPCA). This diverse team has been actively involved in every phase of the project. Communication among the members follows a familiar pattern applied across a number of tasks. Whether gathering information or making decisions, the group begins with personal visits. These have included structured "walk-throughs" of local department operations, planning meetings, vendor demonstrations, and so on. After each series of face-to-face sessions, a written document is produced by the state project members, representing their understanding of the business process, project scope, timeline, or other items under review or development. Usually, within a short period of time, the document is given to the local members to obtain their feedback and clarification. Revisions are then made and the documents become part of the written record of project results. These written records, having been developed and approved by all members of the team, form the foundation for a mutually agreed upon set of project principles, goals, and achievements. There is no confusion about what they represent or how each one fits into the larger scheme of the project plan.

Annual Financial Reports: Clear, convincing, and continuous communication

The NYS Office of the State Comptroller (OSC) knew that it faced a difficult task when it began planning its automated electronic annual report filing system. Because of the large number of local governments that would be participating in the project and the range of technical knowledge, available resources and existing systems that the local governments operated, there were many factors for OSC to consider. OSC further recognized that it would be necessary to not only enlist the support of all project stakeholders, but also to establish good communication practices for sending and receiving vital information about the project. OSC addressed this problem in two ways: first by letting the stakeholders know that the project was a collaborative effort, and that their input and participation truly mattered, and second by implementing efficient and effective practices for communicating with them. Since this was a voluntary option and not mandated, it was necessary to market the project well to ensure that all local governments who desired to file electronically were aware of the project. Some of OSC's techniques included:

• Establishment of a formal project vision and statement of scope to inform stakeholders about the project purpose and gather support
• Formation of a local government Advisory Committee to act as an oversight body to review project status and discuss goals and strategies
• Consistent and regular use of note takers, facilitators, meeting agendas, "parking lots", and action lists to make effective use of meeting time and ensure that all meetings were run in a professional atmosphere.
• Prompt action on outstanding issues and reports.
• Continuous marketing using surveys, newsletter articles, speeches at conferences, training sessions, and direct mailings.

OSC’s efforts resulted in a communication environment that encouraged participation and yielded prompt results. Stakeholders knew their input was important because they could readily see how it affected the project development and they were fully informed of what was happening with the project at all times.

The traditional ways to finance government information systems initiatives, prevalent in the 1970s and 80s, consisted of two main types: (1) direct appropriations from state legislatures that were used to cover both state and local costs or (2) federal funding that usually matched state funds by a formula. Some critical systems projects financed partly by the federal government benefited from "enhanced" federal funding - sometimes as much as 90 percent - as long as states abided by certain schedules and other rules. While these methods are still in use today, it is much more likely that a state-local information system effort will be financed by a "package" of resources that comprises some combination of cash appropriations, some grant funds (either federal or foundation), some "in-kind" resources (public and private), and a lot of redeployed human effort.

Since these resources go well beyond the usual budget categories that finance and budget office staff are familiar with, the project manager or the senior members of the project management team are often responsible for putting this package together. Creative financing entails not only the usual budget management skills, but the ability to convince others to contribute resources, the ability to identify grant opportunities and write successful grant applications, and the ability to recognize and balance the constraints and rules that multiple funding sources can impose on a project plan. Since the full project budget may not come from a dedicated new fund, it is more important than ever that the source and amount of available resources be well understood and carefully managed. Creative financing also means carefully thought out investment of the resources available. Think about ways to cover actual expenses that also "leverage" other resources.

Probation Automation: A little bit means a lot

Seven county Probation Directors are members of the design team for the Probation Automation project. Each is strongly committed to the goals of the project and willing to absorb the work it takes to be fully involved in every phase of the project. However, these local officials have great difficulty finding money in local budgets to pay the expenses of traveling to Albany or other local sites to attend planning sessions, or conduct process reviews. The design phase of this project is supported by a limited amount of federal funds. One of the ways these funds are used, is to pay the travel expenses that allow the County Directors to participate in all these activities. It makes sense to use project funds to cover these travel expenses (which are often small) because these expenditures leverage much more valuable and expensive resources represented by the time and expertise of the County Directors themselves.

In addition, a number of state staff in both the Division of Criminal Justice Services and the Division of Probation and Correctional Alternatives are participating in this project. Most of them are also assigned to other activities and divide their time and attention between those responsibilities and the Probation project. DCJS decided, however, that the project director should be devoted to the project 100% and not be diverted by other competing demands. As a result, he is able to focus full time on the needs of this project, organizing the work in such a way that the part-time involvement of all the others generates maximum value for the project as a whole.

Aging Services System: A financial fabric of many threads

The Aging Services community deals primarily in discretionary relationships. There is no mandate that aging services be provided. They operate through resources that are annually appropriated to support aging network activities. To use their words, "we live by faith." This is possibly one reason why the community has developed a keen appreciation for the value and necessity of building effective partnerships and identifying and securing multiple sources of funding.

Garnering financial support and building partnerships with a broad range of public and private sector participants has allowed the State Office for the Aging (SOFA) to move forward in its efforts to integrate and enhance service delivery to elderly New Yorkers. Several streams of funding support this highly interdependent effort. A grant from the U.S. Department of Commerce is supporting the implementation of Internet connectivity for New York’s 59 Area Agencies on Aging (AAA). Together with matching funds and private sector contributions, this grant provides for remote connectivity to local client databases. A partnership with New York City and a number of private vendors in the development of a pilot system has allowed SOFA and the AAAs to move forward despite limited staff and funding.

In an effort to attract partners from the private sector to participate in the project SOFA advertised the project on its World Wide Web page. As a result, both NYNEX and Cabletron provided services and materials. SOFA sees building partnerships with public and private sector organizations as an ongoing effort. Maintaining current partnerships and building new ones helps this project team stretch the value of the "hard" dollar investments being made by the state, local, and non-profit agencies.

Electronic Voter Registration: Financing from the ground up
Electronic Voter Registration in New York State has been a grass roots project with local officials taking the lead to implement a new process to expand voter registration, provide more accurate and timely data, give faster service, and expand the time period during which Election Boards register voters. This is not a mandated program. Neither is it an optional state-sponsored program that the state tried to sell to localities. It is a voluntary effort among the participants, led by local government sponsors.

The Monroe County Board of Elections was the sponsor of a grant proposal for staffing and consultant services that was funded through the New York State Local Government Records Management Improvement Fund administered by the State Archives and Records Administration (SARA). Total state funding for the project was $180,000. The staff position was located at the New York State Forum for Information Resources Management, an agency-supported organization associated with the State University devoted to effective use of information technology in state and local government. The Forum also contributed additional professional and administrative time to the project and managed a consultant contract. All local staff worked on a volunteer arrangement, as did state staff from the Departments of Health and Motor Vehicles. Together, the paid staff and volunteers managed an effort under the aegis of the American National Standards Institute (ANSI) to define the electronic data inter- change transaction set for voter registration that is now the standard for the nation and is currently being adopted throughout New York State.

"Projects are usually financed through an informal package of resources including appropriations, grants, and in-kind contributions."

The manager of a state-local system project needs tools to manage people, time, relationships, partnerships, ideas, conflicts, resources, information, and processes. He or she needs a range of techniques to manage multiple streams of formal and informal communication and activity. Most of the successful techniques we observed were based on a keen understanding of the project’s goals and common sense adaptation of both traditional and newly popular management techniques. The state-local project manager must have a considerable number of management techniques in his or her tool kit. Among these, the most important is the ability to select the right tool for the job at hand.

A number of tools are useful for establishing common understanding, and getting support and buy-in for proposed project activities. Starting a project with a set of formal assumptions about what is expected of each participant, the pace at which the project will progress, the limitations of existing resources or systems, helps avoid misunderstandings and dispels unrealistic expectations. Visual tools such as conceptual and project workflow diagrams and timelines become living guides to the project that evolve over time. Starting each meeting with a review of the workflow diagrams and timeline helps participants see progress and focus their energies on the work ahead.

Tools that are more oriented to task management are necessary at the functional level of project activities. Preparing for a group facilitation exercise, preparing a site for a pilot installation, or preparing for a public showing of progress or results all require detail-oriented task management techniques.

The following are a few of the more popular techniques in use in the projects we studied:

• Team meetings. Team meetings provide a forum for the project team to meet as a group to plan, and discuss issues, problems, or activities that affect the project as a whole. Thanks to teleconferencing and videoconferencing, meetings can be held face-to-face and electronically.
• Facilitated group meetings. For some activities it is very useful to have a professional facilitator take over the responsibilities of managing group dynamics so every member of the project team can make a substantive contribution to the discussion or decision at hand. This is particularly helpful when many voices need to be heard in the process of reaching consensus on an important decision or action.
• Committees and specialized work groups. Not every agency or individual is necessary in every step of the process. Instead, select representative members of user groups, agency types, or other stakeholders and organize them into working groups. A committee or work group is typically formed to perform a specialized task or activity, and then report its results to the larger project team. This allows work to go on in parallel on several fronts. It also helps focus the limited time of individuals on the areas where they have the most expertise.
• Status reports. Status reports provide information about current project activity and can be distributed in electronic or paper form. Most project participants we interviewed wanted regular status reports, even when there were no major changes or milestones to report. It was more important to have a steady flow of useful information than to have a sporadic one limited to big news.
• E-mail. E-mail is a very versatile communications mechanism. It can be used for one-to-one as well as group communications. E- mail distribution and discussion lists can be used to inform team members about important project activities or events, or to facilitate discussion of problems and accomplishments.
• Visual project management tools. Graphical project planning, scheduling, and reporting tools (e.g., GANT, CPM, PERT) provide useful techniques for visually communicating project information. These pictures are often the best way to show how different streams of activity interact with or depend on one another. There are software packages that create these pictures, but the most complicated is not necessarily the best, and sometimes a simple hand drawn sketch says all that is needed.
• Quality management techniques. Many of the tools and methods of the quality movement are very effective in state-local systems efforts. They are especially useful in setting goals and solving problems. Don’t worry about finding a complete set of brand-name quality management tools. The generic versions work just as well.
• Checklists. The humble checklist gets a great deal of use in planning specific activities, products and events. It puts all the pieces of work in one place along with due dates and work assignments for all to see, discuss, and understand.

Real Property System: User groups for various platform options

The New York State Office of Real Property Services (ORPS) needed a good way to effectively manage and communicate with a large and diverse user community when it began planning for its Real Property System (RPS) Version 4 project. The local government user community for the system was dispersed all across New York State and comprised individuals with different levels of technical knowledge and available resources. In addition, no two local governments seemed to be using the same hardware and software configuration. The RPS team understood that to meet the needs of all stakeholders, it would be necessary to work with several different groups that represented the various interests in this diverse user community. The solution was to create user groups for each of the major technical platforms then in use.

The RPS team held formal meetings with representatives from each of the user groups to discuss the project and how it would affect agencies using their platform type. The representatives participated on a purely voluntary basis, according to their interest and availability. After each meeting, the representatives relayed the meeting minutes to all members of the user group using e-mail or memos. This process ensured that everyone in the user groups was kept informed of all discussions held and decisions reached during the meetings. This method of management proved to be very successful. The staff at ORPS credit the success of this effort as part of the Agency’s new focus on the customer, and said that this practice will be continued in future system development efforts.

Probation Automation: A picture is worth a thousand words

The goal of the Probation Automation Project is to develop a computerized system to support the informational needs and business practices of approximately fifty small to mid-size County Probation Departments. In order to succeed, the project team needed to first identify and then stay focused on those functional areas which were common to all of these different departments. The various tasks being performed at the local offices were identified and then categorized as Levels I, II, and III with Level I representing common core functions. The team represented their findings in a diagram using concentric circles - the center circle specified Level I Probation Core Functions, the middle circle specified Level II functions needed for Alternative to Incarceration Programs, and the outer- most circle specified Level III functions performed by external treatment providers. The diagram was used by the project team to confirm their mutually developed understanding of the functional areas to be addressed by the new system. It was also used to convey their focus and reasoning to the wider community of stakeholders. The diagram became the focal point for many subsequent discussions, allowing the team to emphasize both the precise areas the project was meant to address and the larger context within which these core functions lie.

Any project can benefit from a systematic review of similar efforts in other places. Since private and public sector organizations in this country and others often conduct similar programs, there are nearly always models from which to learn. Academic researchers and non-profit organizations may also have solved a problem, or at least developed part of the solution. There is a lot to learn from success stories and even more to be learned from cases where things didn’t always go as planned. Although most organizations and individuals are more likely to report their successful models rather than their failures.

Best practices research is an organized attempt to learn from the experience of others. It aims at identifying the best possible set of solutions for a given problem. The advantage of best practices research is that it minimizes the possibility of repeating known mistakes and helps planning teams identify all components of a problem.

Models can be found on the World Wide Web, in library online catalogs and CD-ROM databases, from commercial information vendors and at conference sessions and vendor displays. Interviewing experts can yield good results as can posting questions on Internet-based discussion groups. Site visits and technology demonstrations can provide firsthand experience with a system already in use.

"Best practices research is an organized attempt to learn from the successes and failures of others."

Immunization Information: Piece together a model from partial solutions

The NYS Immunization Information System project is designed to construct and manage a number of very large regional databases that catalog immunization records from thousands of providers, including public, private, and non-profit health care professionals and agencies. To collect this invaluable public health data means involving no less than every provider of immunizations in the state. No existing models emerged from a review by the State Health Department of existing state or federal government efforts. However, the US Centers for Disease Control (CDC) had developed a working paper around the issue including a key factor: the specific data elements that would be needed for an effective immunization registry. A contract systems integrator was able to identify technology components that might serve the needs of this unique project as well as offer IT methodologies to help ensure that the project sites were considering all the various management, technology, and policy implications associated with such an ambitious project. While the systems integrators were not directly responsible for system design they were able to aid the teams in identifying and evaluating existing technologies which helped ensure technology awareness and system integration needs. The project team began to construct its plans based on these two partial foundations: from the CDC paper they were able to compare their thinking with other public health experts, and from the experiences and expertise that the systems integrator offered, they could see some of the likely technical options and operational considerations.

Although there were no working immunization registries from which to model their project, the team learned that there was great value in these pockets of expert judgment. They pulled these partial models together and were able to recognize key data and technology factors that gave them a significant head start on this important and ambitious new initiative.

Aging Services System: Good models are sometimes close to home

The Aging Client Based Management System (CBS) project team undertook a comprehensive search for models to guide the development of an information system to integrate and enhance service delivery to older New Yorkers. To avoid "reinventing the wheel," the project team established both technical- and content-focused interdivisional teams and undertook an exhaustive search for applicable models in other state and local units on aging. Professional associations, personal contacts, and formal channels were used to support this search. The effort successfully identified a model, secured a primary partner on the project - and ended closer to home than anyone expected. A project was already underway in New York City to develop a system to support integrated needs assessment and service authorization for the elderly and it served many of the needs of this new statewide effort.

The new partnership between SOFA and the New York City Department for the Aging is resulting in the development of a pilot system which will be tested in eleven Area Agencies on Aging (AAA) as well as in NYC. SOFA, working together with the AAAs and the NYC Department for the Aging, identified a set of common core business processes that could be the focus of the pilot system. SOFA and NYC are working with a subcontractor on the development of the system. Representatives from the eleven AAAs as well as staff from the NYC Department of the Aging have been participating in needs analysis efforts and are being trained on the use of the pilot so that they may effectively evaluate its use and the level of customization that may be required to support their local conditions.

A system that successfully supports both the service delivery role of local governments and the information requirements of the state usually results from a clear understanding of the dependencies and requirements that govern the business processes that link them together.

In many cases when an organization takes on a project to improve a complex business process, those involved in the process are brought together for the first time. Most work processes have evolved over long periods of time and reflect the idiosyncrasies or preferences of individuals or of program and policy changes. Often, no one knows the whole story or the basis behind particular tasks or sub-tasks.

Project teams often find that a significant amount of the improvement they expect from a new system, actually comes from understanding and improving these processes. Subsequently, automation can often add further improvement. Several approaches were used by the eleven project teams to ensure that the participants understood and improved the complete business process involved in their systems efforts.

Bringing state-local project participants together in an effort to map or outline the business process under consideration for improvement is critical to the success of an intergovernmental project. The information flows and process dependencies within and among organizations are complicated and seldom under the complete control of any one organization. Participants in these efforts certainly begin to understand the related processes in place in the other agencies, but in many cases they also come to better understand their own processes.Once participants are fully aware of the business process under consideration, the basis or rationale behind particular steps, and the effect of those steps on the overall process, they are then able to begin reviewing the process for improvement opportunities. If a particular step in the process results from a statutory requirement, then the step cannot be removed and the freedom to modify that step is limited. However, if a particular task is based on agency practice which has evolved over time and the rationale is no longer relevant, then the task may be considered for removal or refinement. Redundant and inefficient steps can be identified and removed. After this process improvement effort, a system design can go forward with greater likelihood of sucess.

Probation Automation: On-site "walk-throughs"

The Probation Automation Team conducted extensive "walk- throughs" of the seven local probation departments represented on the design team. They used a standard protocol to document the business practices of each department to determine whether their practices were similar enough to warrant a standardized process that could be supported by one new system. The Directors from all seven local agencies plus state staff (both technical and program) from the Division of Criminal Justice Services and the Division of Probation and Correctional Alternatives took part in each site visit. Each local department received a written summary of the documentation for review, comments, and revision. The next step was a "cross-walk" among the seven write-ups to identify both similarities and differences among these local operations. The result was a flow chart and process element narratives that will be used to design the new system. These documents not only established the feasibility of standard business processes, they also turned out to be quite useful in other settings, such as orientation for new staff and input to other business process improvement activities like document imaging.

SALESNET: Every player at the table

In preparation for developing an automated system to record and report property sales, the NYS Office of Real Property Services (ORPS) organized a highly representative statewide Steering Committee which acted as both an advisory group and a communication conduit. In organizing the committee, the ORPS team invited representatives of the Bar Association, the municipal assessors, county clerks, title companies, county directors of real property services, and county IT directors. Through the contacts on the Steering Committee, stakeholders in the counties who deal with property sales and transfers were surveyed about their readiness for adopting an automated system. The survey results helped the developers choose the pilot site and it helped provide a realistic user perspective.

The team also operated in parallel with an overall core process improvement project underway at the Office of Real Property Services. The core process improvement effort produces maps that explain what users are doing in transactions that affect real property. One group of representative clerks, assessors, and lawyers were brought together in Orange County to determine every step of the sales process, including what happens in the prelude to a sale. A data flow diagram that resulted showed the process from beginning to end. This helped the development team focus on users needs in designing a friendly, functional system - and one that would make sense in the context of the overall core processes that support real property transactions all around the state.

"Often, much of the improvement we expect from a new system actually comes from understanding and improving business processes."

Understanding that the solution to one problem is not always the solution to another even highly-related problem, helps a project team avoid mismatching technology solutions to problems. Further, understanding that not all problems have or require a technology solution helps ensure that an appropriate match between technology and the task at hand is made. Over-doing as well as underutilizing technology are both risks to state-local system initiatives.

Project teams often look to the technology lessons from their last project and try to apply them in the current project. Or they become interested in a new technology that seems to be barreling through the marketplace with powerful new features. They sometimes fail to give full consideration to the work processes and overall business context in which the system must operate. Consideration must also be given to user capabilities and the organizational and staffing limitations of the agencies that will be implementing, using, and maintaining the system to deliver services. Technical awareness activities that introduce a variety of technical tools to the project team are often helpful. These can consist of literature reviews, searches on the World Wide Web, vendor presentations, or attendance at technology exhibitions and conferences. Prototyping is an excellent, relatively low- cost way to test the "fit" between a technology and the environment it which it must work.

In most cases, incremental system building using appropriate standards, prototyping, piloting, and evaluation techniques, is a good idea. This approach allows for additions to system functionality as well as for the integration of new technologies over time.

Hunting and Fishing Licenses: Making hard choices

When looking at the information requirements of its automated hunting and fishing licensing system, the New York State Department of Environmental Conservation (DEC) recognized that the technology options that could be used to build the system were somewhat limited, due to the rather unique way that the Department sells and processes licenses as "accountables." One of the solutions proposed to DEC involved the use of specialized point-of- sale hardware that was impressive, but also proprietary, expensive, and difficult to integrate with local systems. The appeal of investing in this solution was tempting - DEC would be able to use dedicated, state-of-the-art technology to automate the licensing process and build a system with high integrity. The downside of the solution however outweighed this appeal. Local issuing agents did not want another piece of hardware or a stand-alone system for licenses. They believed they would be much better served by a solution that integrated with technology already in use in their offices. DEC decided that those considerations were more important and began to investigate other tools to get the job done. DEC’s ability to weigh long- term goals against the immediate appeal of the specialized hardware will eventually result in a system that will better support the needs of the local governments by satisfying their requirements for simplicity and integration with existing business activities.

Social Services Imaging Project: Tools of the trade

The tools that are used by tradesmen to perform carpentry, plumbing, electrical work, and masonry are all different because the tasks they perform are different. In the world of system development, the same principle holds true, if you want to do the job right you need to use the right tools.

The New York State Department of Social Services (DSS) understood and subscribed to this philosophy when it was considering design options for its local district imaging system. On the surface, the system appeared to be a simple imaging application that would be used to collect and archive forms and information relevant to social services cases. However, the team knew that the system would need to do much more than that. It would also need to be a decision support tool that would be used by local DSS caseworkers to make client decisions based on historical information. As a result, the team knew it would need to consider technologies that would accommodate the integration of images, workflow, and case information stored as "objects."

By understanding how case workers work and selecting the right tools, the team achieved successful results. The imaging technologies they selected met the initial requirements of the system design by supporting the collection and sharing of case-relevant information among local DSS staff. Information that previously had to be sent to different offices through the mail, could now be shared electronically over computer networks to support rapid and effective case decision making.

"Before choosing technology, consider work processes, user capabilities, organizational factors, and existing systems."

In our world of rapidly evolving technologies, there are many different options. Before making selections, it is a good idea to investigate current technical standards and to develop an awareness of what products support the standards. If you purchase technology that does not support current or developing standards, chances are good it will not be able to integrate with other products.

Industry standards exist for almost every type of hardware, software, and communications technology, including such things as data organization and access (e.g. database structure, query languages), data interchange (e.g. Electronic Data Interchange, encryption), networking services (e.g. data communications, network management, e-mail), and document imaging (e.g. scanning, imaging, workflow). In some cases, these standards are developed through the efforts of a formal national or international committee. In other cases, because of market share, a certain vendor’s approach becomes the de facto standard.

Standards enable interoperability and electronic messaging among system components. They also offer vendor independence and scalability - when you use a common standard, you will be able to choose among different products that adhere to the standard and will be able to scale up to larger systems when the need arises. You can become familiar with the appropriate standards for any given application through discussions with experienced colleagues, talking to vendors, reading trade journals and other literature, and by searching the Internet. New York State has established preferred standards for many technologies through the efforts of working groups sponsored by the Governor’s Task Force on Information Resource Management.

Social Services Imaging Project: Images of success

When considering technical options for the development of its Local DSS Imaging project, the New York State Department of Social Services was faced with a dilemma: try to build a system that would run on as few platforms as possible to reduce support and maintenance problems or try to accommodate the diverse hardware and software base of its large user community. The team understood the inherent difficulties of trying to develop and maintain different versions of a system that would run on multiple platforms at the local level - the numerous idiosyncrasies of different software, operating systems, and hardware would make it extremely difficult to design and support the system. However, the team also understood that local agencies need to integrate new systems into their existing computing environments and would have a tough time convincing their management to accept a system that was not compatible with existing hardware and software. The problem was how to build a system that would take advantage of the best technology while at the same time avoid the possibility that it would become an orphan in the local government’s larger systems environment.

As a solution to the problem, the team decided to develop a system that would be compliant with as many mainstream hardware and software standards as possible. This would ensure that the system components would integrate with one another, and greatly increase the potential longevity of the system by providing an upgrade path that would allow for the integration of evolving technologies that used the same industry standards. Local users were happy with this solution because they knew that they would not be receiving a system built with outdated or proprietary components. For those not yet following the mainstream, this was also an incentive to begin to move toward standard technologies that would enable them to adopt new tools and enhancements more readily in the future.

Electronic Voter Registration: The right standard at the right price

The Electronic Voter Registration project team was faced with a challenging task: the seamless communication of information among disparate computer systems employed by the State Board of Election, the local Boards of Election, and the State Departments of Health and Motor Vehicles. While employing Electronic Data Interchange to handle the application-related interface issues, the project team originally expected to use Value Added Networks (VANS) to actually transport the transaction records from one computer to another. Under this traditional method, companies lease lines into a central site supported by a for-profit VAN. The VAN transfers records from a client which is sending EDI transactions to the intended recipient. Both the client and recipient must be members of the VAN and the service charges are considerable. During the project, the team wondered whether the standard TCP/IP protocol of the Internet might be employed as a replacement for VANs. The team was able to demonstrate that such a system could be developed using electronic mail and the MIME protocol along with standard encryption methods. The result was a workable solution which used emerging world-wide technical standards to accomplish their goals at reduced costs.

"Technical standards offer interoperability, scalability, and vendor independence."

Data standards help different participants speak the same language. They usually include at least two features: an agreed upon definition of the meaning of a term and an agreed upon format for how the term will be represented in the system. For example, the term "application date" might be defined to mean the date on which an applicant for services submitted a signed application form. The agreed upon format for "application date" might be defined as an eight-digit number consisting of 2 digits for day, 2 digits for month, and four digits for year, in that order. Data standards can be more complex and include information related to business rules for how the data is used and even data models that show the relationships among data elements.

Standard data definitions and formats organized in a common data dictionary are an essential prerequisite for effective information sharing among government organizations and between the government and private firms. While the development of these standards is often a time-consuming and difficult process, it is an effort well worth making since common data definitions form the core of any integrated system. A standard data definition offers these necessary components for successful intergovernmental implementations:

• Provide a common language for information sharing. Since a primary goal of any intergovernmental application is to facilitate the exchange of information among interdependent organizations, it is important that all groups "speak the same language."
• Help ensure that the data sets will be described accurately. Data dictionaries can serve as a guideline for describing data completely and accurately. Well understood definitions are an important tool for an organization’s internal documentation as well as for data exchange.
• Facilitate automation. Once the data is standardized, more sophisticated software can be developed for creating, collecting, processing, and searching the data.
• Allow for both central and distributed storage of data. In some cases, it may be desirable to store and integrate the information collected at multiple origins into a single database. Sometimes, original information collection is accomplished with a variety of local software packages. Integration of information which originates from different sources can only be accomplished when data elements are commonly defined. Conversely, data defined in the standard way can be stored at distributed locations with confidence that the meaning and integrity of the information is consistent from place to place.
• Support information exchange. To enable the exchange of information among disparate computer systems, the data being transferred must conform to strict messaging formats. Often, this exchange is facilitated through Electronic Data Interchange (EDI).EDI uses a set of national and international standards to define message formats and the data elements within these messages. EDI standards have been developed and approved by the American National Standards Institute (ANSI) and the United Nations/Electronic Document Interchange for Administration, Commerce and Transportation (UN/EDIFACT).

Automated Dog Licensing: Keep it simple

Standard data definitions were a guiding principle of the Electronic Transfer of Dog Licenses Project from its inception. The NYS Agriculture and Markets staff knew that in order to make the licenses electronic, the process would have to work with many vendors’ systems. The development team looked at the information and asked "What do we really need?" Despite the fact that several interest groups wanted additional information collected at the point of issuing a license, the Ag & Markets team stood firm against data redundancy and "nice to know" information. They had one aspiration - to keep it simple.

After consulting with users and vendors, the development team defined the basic data needs with which all vendor systems would have to comply. They eliminated all fields that were not used regularly. Vendors realized that in order to compete with other software companies, they needed to meet state data requirements. Any automated licensing system that complied presented benefits to municipalities.

Today, each approved vendor system converts the information collected to a file that can either be e-mailed via the Internet or sent to Ag & Markets on a floppy disk. Each municipality has an authorization code that provides validity for the data communication when records are sent electronically. The basic information that now must be collected boils down to just a few important, standard elements describing the license, the owner and the dogs.

Each municipality that has adopted the system transmits information in the same order, using the same codes, and using fields that have common data definitions. Keeping it simple has eliminated paper- work, mailing, and postage costs, and has ensured more accurate data that helps protect communities against the spread of rabies.

Electronic Voter Registration: Local needs become national norms

Not every state-local project requires the creation of a new national data standard, but New York State’s Electronic Voter Registration project did produce standards for the transmission of voter registration information that will be used all across the country. This project was started by local officials in the Monroe County Board of Elections who applied for and received a grant from the State Archives to hire a technical expert housed at the NYS Forum for Information Resource Management. The expert, using New York State needs as a guide, chaired a national ANSI working group to select EDI protocols, encryption protocols, software, platform, and Internet capabilities that will allow for the standardized electronic transfer of voter registration records.

The State of New York has 57 counties, 62 cities, 932 towns, 554 villages, 707 school districts and 646 independent special purpose units working with each other, with citizens and businesses, and with about 100 state agencies and authorities. A project being conducted in this environment must deal with huge variations in financial, technical, and managerial resources, and seek to minimize the uncertainty of this environment on the project. A system that supports information exchange, transaction processing, or decision support between just two organizations is a challenge. A system which is integrated into the work processes of 50 or 100 or 1000 organizations is orders of magnitude more difficult.

Understanding the range of conditions under which both state and local organizations operate is key to ensuring that the system is designed to integrate with their business environments. The particular business process being addressed must be analyzed and understood by all participants. In most cases, state-local information systems projects are focused on standard business processes such as issuing a license, determining eligibility for a benefit, or recording a property transaction or vital record. However, these standard business processes are conducted throughout the state in very non-standard environments. Projects therefore need to focus on both the business process and the ability of individual organizations to adopt an information system to support that process. Tools such as data dictionaries, and process and workflow analysis help identify ways that different organizations can and should participate. Organizations unable to implement a sophisticated automated system in the short term can begin to transition their work environment by focusing simply on the new or improved business process. An organization that needs to retain its reliance on paper processing can still improve its performance and consistency by adopting the set of standard data definitions that are built into the computerized system. In this way, each organization can begin to integrate the useful elements of the new system into its own environment, within its own operational and resource constraints.

There are excellent resources available to help develop profiles of the various local entities. There are numerous state associations affiliated with counties, cities, towns, villages and virtually any other designation applicable to local governments. Often these associations have conducted extensive surveys of their members and at the very least have a working knowledge of the conditions their members face. The Office of the State Comptroller, Division of Municipal Affairs also has extensive information available on the state of local government in NYS.

In developing a working profile of the local participants, a project team should be able to better define the scope and possible solutions much earlier in the project cycle. For instance, if the majority of potential local users lack a fairly new, modem-equipped computer, then an Internet/ Intranet solution may simply have to wait, or a more incremental approach adopted. On the other hand if the majority of potential system users have already established Internet access, then it may make sense to pursue a system that takes advantage of this established resource. Our interviews revealed that many agencies are moving toward providing an Internet type option for their service but will continue to maintain paper and computer disk systems as well to meet the needs of all local governments. This may seem less than ideal but it is a realistic way to deal with so much local diversity.

Aging Services System: Identify and build on the common ground

The State Office for the Aging (SOFA) works with 59 Area Agencies on Aging (AAA) located throughout the state.The federal government has implemented new reporting requirements for these agencies. SOFA recognized that the range of available technical infrastructures within the AAAs located throughout the state precluded their ability to implement an information system to support these new requirements. SOFA needed to identify a way to ensure that each agency was capable of responding to the new reporting requirements. Recognizing the variability of the local environments, yet the generally similar approach used in the process of analyzing eligibility and authorizing services, SOFA engaged in a process of identifying a set of standard data elements to support both the federal reporting requirements and the local needs assessment processes. SOFA engaged in a collaborative process with over 200 representatives from the AAAs, to identify the necessary elements and to develop an approach for the implementation of these elements into the assessment process. The follow-on project being conducted by the project team is to pilot an information system that will support integrated case management in the AAAs so that as these agencies may transition to the information system as they develop capability to support the necessary technical infrastructure.

Automated Dog Licensing: Build on the existing base

The nightmare of proliferating hardware and dozens of incompatible software packages is what municipalities fear when state agencies dictate the adoption of new systems. With each new information system, local officials fear the cost of new hardware and the time necessary for training before the new system is fully functional. The NYS Department of Agriculture and Markets calmed the fears of Town Clerks when it developed an automated dog licensing system that was designed with compatibility in mind. The system can be used with existing hardware and commercial software packages that Town Clerks already have installed for other uses. The developers ensured this compatibility by working closely with the users and the vendors who would be supplying software. The new system was cost effective and required minimal training because of the smooth way it adapted to local conditions. Now Town Clerks have fewer weekly paper reports to submit, data errors are minimized, Dog Control Officers have more current information, and without adopting entirely new technology tools.

Immunization Information: Tailor made systems with a common purpose

The NYS Immunization Information System project consists of four regional demonstrations to define, collect, monitor, and report information about immunizations for children age two and under. Each demonstration adheres to a set of common program and functional goals, but each is free to design a system that suits the needs and capabilities of the users and agencies in its region.

• The Upper Hudson demonstration will use a Frame Relay System with a regional server connected to health care providers throughout the region.
• The Central New York demonstration chose to build its system on the foundation of the existing Central New York Regional Perinatal Data System.
• In the Finger Lakes demonstration, newborn data will come from the NYS Birth Certificate database and data on older children will be gathered from billing claims and outreach workers.
• In western New York, the Chautauqua demonstration will expand the existing immunization tracking system now in use by western New York county health department clinics.

Each demonstration site has its own design team advisory committee and is working with a system integrator hired by the NYS Health Department to assist them in matching the system to both statewide goals and local conditions.

Prototyping your system as you develop it offers an excellent way for the project team and customers to see the design-in-progress and help refine and improve the system as it evolves.

The development environment for state-local information system projects is typically complex, due to the large number of project stakeholders, the need to consider numerous system requirements of each, and the demand from all stakeholders for rapid development and deployment. Creating systems in this kind of environment demands a system development method (SDM) that lends itself to rapid design and development. Often, the SDM best suited to this environment is prototyping.

Prototyping differs from the classic system development methods in that it allows for the building of the system to begin much earlier in the development process, and allows customers to see and influence the system as it is being built. The philosophy behind prototyping is that system development is more effective when customers are partners in the design process. The prototype makes tangible all the ideas that both designers and customers usually try to communicate to one another in words. The prototype makes it possible for both to see and understand the needs, functionality, and limitations of the design and to alter it as needed. Most of the projects we reviewed used prototyping to develop their systems and reported satisfaction with the results.

Aging Services System: Prototype your way to a final design

When considering what type of system development method to use for its Aging Network Client Based Service Management System, the NYS Office for the Aging (SOFA) identified two criteria that the method would need to accommodate: (1) it would need to support a rapid development cycle, and (2) it would need to facilitate convenient modification of the system as it was being developed. These criteria were not selected by whim. Because of the large number of project stakeholders, which consisted of local administrators in all of the State’s 57 counties plus New York City, and some complex system requirements that included the integration of several different kinds of technologies, the project team realized that it would need to use a development method that would allow for a working version of the system to be developed very early so it could be shown to the various project stakeholders and modified according to their needs. Based on these factors, the team selected prototyping as the system development method that would be used.

The prototyping method produced positive results. The early system demonstrations showed the county level administrators the potential of the system and how it could support locally administered, cost effective, flexible services, while simultaneously supporting clients™ most important needs. These early demonstrations of the system created an enthusiastic response among local governments and generated positive interest in the system among the user community.

Social Services Imaging Project: Generate bigger and better ideas

The NYS Department of Social Services (DSS) found that prototyping method provided more than just a way to rapidly and effectively develop a computer system. When working on its local DSS imaging project, the team used prototyping to build a system that was more comprehensive than the original design.

The developers used the prototyping method to demonstrate the system as it was being built. As each iteration of the system was developed and presented to the full team, suggestions were made for modifications and enhancements. As the system grew and began to make more sense to the team members, suggestions were made to accommodate an increasingly larger number of processes and system features. In short, the increased exposure to and use of imaging technology acted as a stimulant for expanding the scope of the entire project. For example, the initial goal of the system was to construct an imaging system that would capture information vital to a DSS client case. The use of prototyping also encouraged the evaluation and expansion of related business processes. As the prototype version of the system progressed through various iterations, the focus of the project team moved to other scenarios such as "electronic case foldering" (i.e., how to electronically share all case information among geographically dispersed DSS employees) that offered expanded benefits to both clients and caseworkers.

"With prototyping, customers become partners in the design process."

Many system implementations are initiated with pilot tests that bring the system into the field to evaluate and refine design, performance, and integration with other systems and activities. The pilot site (there may be more than one) is a critical organization Πone that is willing to undergo on-the-spot evaluation and identify and work on the inevitable problems that pilots are created to uncover and resolve. The pilot site provides the system developers with a way to evaluate the initial release of the system in a controlled environment and, if necessary, make any changes before releasing the system to all users. Many of the projects we reviewed involve one or more local pilot sites to test and refine their systems. In some cases, the pilot site was an integral part of the development team, building local needs and practical limitations into the initial design and then serving as the initial implementation site.

Sometimes pilots are promoted as a way to get special attention and early implementation for a new system. While this is true, it is only half the story. In return for these benefits, the pilot organization has to commit staff, space, and other resources to a process whose goal is to find and fix problems. Although it has clear benefits, piloting can be frustrating, time-consuming, and disruptive.

A number of considerations go into the selection of a pilot site. Here are few of the more important ones:

• Representativeness. Will the site(s) you are considering give realistic results for guiding broader implementation? If there is a great deal of local variation, you may need several pilot sites that represent broad categories of local conditions.
• Organizational capacity to carry out the pilot.
  Does the pilot site have the staff, space, equipment and other resources needed to carry out the pilot without adversely affecting its ongoing operations? Do the pilot site staff have realistic expectations about their roles and the amount and kind of assistance they can expect from developers or consultants?
• Leadership commitment to the project and to the special demands of pilot status. Do senior managers in the pilot site believe in the goals of the project? Are they ready and willing to deal with unexpected problems? Will they give their staff the support they need to carry out both pilot activities and regular operations? Will they act as liaisons to local officials in other departments who may be affected by the pilot or eventual operation?
• Geographic accessibility. Is the pilot located in a relatively convenient place so travel costs and time can be minimized for the project staff who need to be on-site and for the pilot staff who need to travel for training or other project activities? Try to avoid a pilot site whose location discourages on-site technical assistance, monitoring, and evaluation. For example, does it always require an overnight stay or several hours of driving, or more than one mode of transportation to get there?

SALESNET: This is a make-or-break relationship

The technology staff at the NYS Office of Real Property Services knew that pilot testing their SALESNET system would be critical to its successful implementation. Through surveys and discussions with county officials and information technology directors throughout the state, the SALESNET team chose a county that seemed ready and willing to test the Internet-based sales reporting system.

Before long though, it became clear that not all the primary stakeholders in the county had a strong commitment to the project, and some of the secondary stakeholders were reluctant. Local officials expressed concern over control issues and political concerns about having a central database on the state level. The SALESNET developers were wise enough to determine that this county was not ideal as a test site, and they started over to explore testing in another county. They knew that full cooperation in the original site was unlikely. Despite the added trouble, energy, and staff investment, a new county was chosen as the pilot site. Through the SALESNET project we learned that a pilot site must be chosen carefully, using the following criteria:

• primary stakeholders must be committed to, not just interested in, testing
• peripheral stakeholders must have an interest in the test
• there must be an abiding interest in the project that will last the duration of the pilot period
• the political climate must be open enough to permit some realignment of information and responsibility

Social Services Imaging Project: What to look for in a pilot site

The Department of Social Services SSIS staff (Social Services Information Systems) believed strongly in the importance of selecting a good pilot site for their Local DSS Imaging Project. Selecting a good site would add value to their project by providing useful feedback about the fledgling system and by letting other sites see and hear about the benefits of the project. The problem was, which site should be selected?

The team did its homework to ensure that the pilot site would contribute to the success of the project. After considering the criteria for the selection of a site, and the various pros and cons associated with each site, the LDSS team made its decision. The selection of their pilot site was based on three important factors:

• What site had the best technology environment to act as a pilot?
• What site had strong leadership and good relations with employees?
• And finally, what site would provide the most positive impact for the project once the piloting work was complete?

The implementation of the system at the pilot site yielded solid information for the project team about the system and about the implementation process itself. The illumination of both technical shortcomings associated with the system and process issues related to implementation provided the team with vital information that could not have been discovered otherwise. Due to the success achieved at this pilot site and other early participating sites, additional counties have expressed a desire to participate in the project.

We’ve all heard the phrase, "Don’t reinvent the wheel." If the technology you need has already been developed and is available for you to use, then you shouldn’t waste time and resources recreating it. Another increasingly common phrase is, "Outsource it." Depending on the nature of your project and the availability of resources, it can make good sense to pay an expert to build the system for you, so you can concentrate on the work that needs your specialized expertise.

Managing organizational interdependencies and new partnerships, setting data standards, and facilitating group decision making are just a few of the challenges to state-local government project teams. These processes require the programmatic and contextual knowledge that government officials possess. Technical expertise to support the implementation of a new network, a new database engine, or a more intuitive graphical user interface is not the exclusive knowledge of government officials. A number of the projects in the eleven reviewed in this effort had no technical expertise on the project team. Either the resources were not available in the participating agencies at all or they were not available to these projects. In some cases, technical expertise was available on the teams but in limited quantity.

To overcome the resource limitations and to maintain focus on the programmatic challenges, a number of the teams operated as systems integrators rather than as system developers. Project teams identified portions of the plan that could be outsourced to technical specialists and then managed those relationships. This hybrid approach allowed for substantial time savings. In many cases the project participants recognized the value of various technical approaches to implementation of the system, however, the necessary technical expertise was not available on the team. Rather than investing in developing those skills first, and then designing and developing the system, the team focused on business process issues and basic design and handed off the detailed design and hands-on development work to vendors. In a number of cases, vendors were able, due to their comprehensive knowledge of the technology and the use of an iterative prototyping approach, to contribute to the design efforts as well. It is important to remember, however, that agency staff will need to develop the skills to maintain the system unless an ongoing maintenance relationship with vendors is part of the overall plan.

Immunization Information: Experts put the technology pieces together

The immunization registry project is organized into four regional demonstrations areas, covering large numbers of individuals and needs. In considering the resources necessary to effectively design and develop a system, participants agreed that neither the state nor the localities had enough technical staff to do the system development work in addition to other project activities and existing assignments. As a result, an approach was selected that freed the project team from the burden of technical development and allowed them to focus on identifying and meeting disparate local needs within a set of statewide goals.

The project team chose to focus on the local and state program needs and on acquiring a general understanding of available technologies to address these needs. They became familiar with a number of potential technology solutions and began to understand what kind of system would best meet their needs. They then turned to the private sector for technical experts to implement the technology pieces. A private firm was brought in as the systems integrator and was charged with subcontracting the necessary technical development to appropriately skilled vendors. The knowledge gained in the process of self- education about needs and possible technical approaches gave the project team greater confidence when working with the integrator and helped them educate the vendor as well as themselves.

Automated Dog Licensing: Selling to the vendors

When planning for the NYS Department of Agriculture and Markets Dog Licensing system, the project team was acutely aware of the time and staff constraints that all participants faced. To meet the project schedule, they decided to look to the vendor community for technical solutions while they concentrated on the data and operational issues. Knowing that many municipalities use commercial packages designed to support the usual functions of Town, City, and Village Clerks, the team created high- level system specifications and issued an RFP looking for companies interested in adding dog licensing to their packages. This approach would allow several vendors to build and market versions of the system as long as each provided a mechanism for the data to be sent from their system to the Ag & Markets centralized database. Several vendors responded to the RFP and the competitive aspect of multiple vendors vying for a limited market ensured that all would have to build quality products.

The NYS Department of Agriculture and Markets talked extensively with vendors to ensure that the requirements for automated dog licensing could work with local systems already in place. This helped many localities, especially small ones, save money and time while allowing them to make useful connections to a new statewide system. Some local officials told us that being able to use existing systems to meet the requirements of automated licensing made all the difference in their enthusiasm and willing compliance with the new program. This approach also generated unexpected side benefits when vendors offered suggestions for making both the centralized system and the field system design more efficient.

Real Property System: Buy some and build some

Version 4 of the software to support Real Property Services in NYS is being developed with relational database technologies. The team reviewed the technology and application marketplace, as well as the systems and software located in the local agencies before deciding on the appropriate platform and approach to use in developing the new release of this software.

The challenge faced by the Office of Real Property Services (ORPS) in implementing this development approach was the fact that few, if any, of their MIS staff had experience in these technologies. The question facing the project team was this: Do we develop in the technologies we know or do we work with the technologies that better meet the needs of the users? Clearly, the right choice was to match the technology to user needs, but how could they do that with no in house technical expertise? To address this challenge ORPS technical staff worked with vendors to build components of the system while in parallel building capability in the agency to handle these technologies themselves. As ORPS staff became more knowledgeable about the relational technologies, they were increasingly able to participate in the design and development activities. The ongoing maintenance and enhancement of the system is expected to be the responsibility of the newly trained ORPS MIS staff.

Mastering a new computer system can be a tricky business for even the most proficient users. Of course, the ideal system design is so elegant and simple to use that little training is necessary. This simplicity is seldom possible, however. Complex, interconnecting systems may not permit the ease of use that a single purpose system does. For example, a system that connects health care providers with local governments and insurers demands different training strategies than one that involves only the account clerks in a single finance office. Both need excellent training, but the integrity of the system and its information is far more vulnerable to error in the first than in the second. In both cases, the users are not technical experts, but are professionals in other fields who must use the system to accomplish some part of their responsibilities. Training needs to demonstrate not only how the system works, but how it fits in this larger picture. "User-friendly" training is crucial, but "friendly" is often in the eye of the beholder; that is, what is friendly to the development team may not seem so easy to the user. The user’s needs and reactions should be the litmus test for the ease or difficulty of the system, and training should be developed around their needs.

When any user adopts a new information system, there is always some anxiety. The process of adopting a new system can be made much less painful by offering well-designed, user-oriented training sessions and reference materials. A thorough training program can help users be more confident in the system and allow them to approach the work more enthusiastically. It is often a good idea to offer training at various points in the system development process. Train those who will evaluate prototypes early in the process, give general orientation sessions to all participants in advance of implementation, and train thoroughly at the point of roll-out in each organization.

Training can take many forms from formal classes to written help materials, and it’s important to recognize that different staff members have different preferences and varying strengths in acquiring new knowledge. It’s helpful to present the same content several different ways to appeal to the different learning styles represented among employees. One person may be an excellent listener and can learn most easily through a lecture or by hearing a trainer talk about how to navigate through a new system. The person in the next seat could be a visual learner and would much prefer seeing the functions of a new system mapped out in geometric shapes with colors differentiating various options available. Others learn by doing and benefit most from hands-on exercises. Whatever the format, thoughtful user training conveys information and relieves anxiety - both critical to successful implementation.

Automated Dog Licensing: Get help from your friends

The NYS Department of Agriculture and Markets, initiated peer tutoring for the Electronic Transfer of Dog Licenses. Town Clerks who have adopted the automated dog license process volunteer to train their colleagues from other towns in issuing licenses electronically.They are even willing to travel to other municipalities to do on-site one-on-one training. The experienced users also use the Municipal Clerks Association to spread the word about the automated process of issuing dog licenses. In our discussions with the local representatives for this project, they cited this volunteer training program as a real plus that encourages new users to adopt a system that is strictly voluntary. Good training is to the point, meets personal users needs, and comes from credible sources such as experienced users. Excellent training is customized to meet users immediate and long-term needs. The individualized training given by Town Clerks involved in the Automated Dog Licensing information system meets all of these criteria.

Aging Services System: Develop skills as you develop the system

Through the Aging Network Client Based Services Project, the State Office for the Aging (SOFA) is building a large and complex system to manage services to support the independence of elderly people. The project has started training sessions for the system even though the beta testing phase is not yet complete. Through the cooperation of the City of New York Department of Aging, a primary partner on the project, twenty-two people from twelve New York counties came together to learn about the system in preparation for the second round of beta testing. This group will explore the system capabilities and give feedback to the developers.

In preparation for the adoption of the new information system, SOFA sponsored a statewide interactive teleconference with 59 Area Agencies on Aging; 600 people participated. This training session was offered to alert practitioners to the information content that will be available on the system. The comprehensive information collection will allow case workers to develop complete care plans for aging individuals. This content training is just as important as the technology "how to" training. The case workers need to know what information and tools the system will contain in order to use it effectively to support integrated case management.

When the new information system is implemented SOFA will conduct regional training sessions and additional teleconferencing training in order to introduce the system to users around the state.

"Training needs to demonstrate not only how the system works, but how it fits into the larger picture."

Ideally all new systems work just as they were designed. Realistically, you can usually count on the implementation of a new system to result in unanticipated problems.Computer users become comfortable in their familiar software surroundings, and taking on a new system is the intellectual equivalent of physically moving to a new home or new office. Users need help adapting to a new system and making it feel as easy and comfortable to use as their old way of doing business.

The time period surrounding implementation is a critical one for user support. Offering immediate, appropriate support at this point in time will relieve anxiety and will encourage willing and effective users. But don’t stop there. There are always new users and most systems continue to add or change features throughout their life cycle. User support needs to be continually updated and continuously available.

The projects we studied made good use of a number of user support formats and strategies including:

• awareness meetings to introduce the system goals and features
• system "maps" presented graphically and in color
• easy to follow one-page "cheat sheets" for common activities
• full-scale documentation manuals
• online help features
• a staffed help desk
• individual tutoring or peer tutoring
• small group training, review, and support sessions
• designated learning time for initial training
• videotaped step-by-step instruction
• frequently-asked questions and answers brochures
• a "help" newsletter

Annual Financial Reports: Willing help from familiar faces

The Electronic Annual Financial Reporting project is designed to meet the needs of a very exacting community - accountants and auditors - and many of the project requirements had to meet national standards. As one local participant put it, "When was the last time the IRS allowed you creativity in your filings?" The reports that were being automated reflect directly on the municipalities and have considerable legal bearing on local governments and the state as well. Thus, it was crucial that the system work well and maintain accuracy from the start. Meeting the needs of the local officials as the primary customers, was essential for this project to succeed.

OSC maintained a consistent project team throughout the process. This commitment to keeping "familiar faces" in the room was cited by local participants as a key to the success of the project. The local participants observed that by seeing these same faces at each meeting and having contact with these same individuals throughout the project they felt confident that expert, consistent help would be available.

OSC has a technical support group for this project. They answer the usual questions about specific system problems. However, local officials say the real benefit that this technical support group has provided is their willingness to answer related technology questions that local users may have. While OSC is not in the business of all-purpose technical consulting, this willingness to take extra time to listen to and help resolve other technical concerns has built a solid trusting relationship and helped local organizations become more technically adept overall.

Social Services Imaging Project: More than a help desk

Evaluating a pilot system is a significant undertaking for any agency and the DSS Local District Imaging project was no exception. The local Social Services offices which participated in the pilot phase of the project depended on support from the project team for many purposes. Of particular value was the support that the pilot counties received in preparing to participate in the pilot efforts. Thanks to group discussions about techniques to use to overcome local technology and management problems, the pilot site staff were well prepared for the integration of the new technology as well as for changes in workflow to support the improved business process. The local participants found that they turned regularly to the project team for support for many things including their efforts to secure and maintain local support, to establish effective working relationships with local MIS staff, and to overcome resistance to changes to the workflow of the agency.

"User support doesn’t stop with implementation - there are always new users and new features to adopt."

Performance-based accountability is a real issue for managers and agency directors, and systematic evaluation and review activities can help administrators and IT staff determine if their systems are achieving their intended goals. Every system should begin with clear purpose and goals. After it has been up and running for a period of time (say six months to a year), it makes sense to revisit them in a formal way to see if the system is performing as intended.

The system evaluation assesses how well the information system is working to support the purpose and goals of the project. Moreover, a solid evaluation of your work establishes credibility that goes a long way in establishing the support you need for the next project. A comprehensive evaluation plan is attractive to funders, policy makers, and taxpayers alike. A project without an evaluation plan is like a stage performance without any audience reaction. Program managers need to know if the system is making a difference. Designers need to know how well the system is working in order to modify and adjust it. Often a new information system is also the impetus for improvements in business processes and an evaluation helps measure these as well.

An evaluation can sometimes be as simple as a self-administered customer survey, phone interview, or focus group with users. It can also be a more formal and elaborate program review, cost-benefit study, or other analysis conducted by evaluation experts. The method needs to be matched to the goals the project was designed to achieve. Each method has its advantages and disadvantages with resources and time crunches being major considerations. Whatever format is used should address the following kinds of questions:

• Service outcomes: how well does the system meet pre-defined customer needs?
• Programmatic outcomes: how well does this system contribute to integrated service delivery or other service system goals?
• Operational outcomes: how well does the system meet time-savings, streamlining, and other operational improvement goals?
• Financial outcomes: how well does the system meet cost-savings or revenue goals?
• Return on investment results: Considering what it cost to create and operate, how cost-effective is this investment?

The answers to these questions lead to decisions about changes, improvements, refinements, and lessons for future initiatives.

Immunization Information: Measuring up to a set of ideals

The NYS Immunization Information System (NYSIIS) project has a number of measurable goals that lend themselves to formal evaluation. One set focuses on the functional requirements of an ideal immunization information system. Each of the four regional demonstrations now underway has great latitude to choose methods and features for its systems, but all are trying to measure up to these functional ideals:

• Each child’s immunization record must contain a unique identifier that could be reconstructed by the parent from information they have readily at hand.
• The system will use the National Immunization data standards to assure compatibility and usability of data across different regions
• A registry must be able to identify all immunizations due for every child in the registry and generate reports of immunizations due
• The system will capture only necessary data elements and automatically generate reports and reminders based on that data set
• Local systems and users, with appropriate safeguards, will be able to update and query regional systems
• Immunization coverage level reports will be available for children at various ages from birth to 24 months of age.

The second set of measurable goals is focused on the public health outcomes associated with immunization:

• By 1997, have at least 120,000 immunization records of children under age 5 (i.e., 15% of such children outside New York City) captured in the NYSIIS
• By 2000, have at least 400,000 immunization records of children under age 5 (i.e., 50% of such children outside New York City) captured in the NYSIIS

To monitor progress over time, these questions are being asked about each regional effort starting in 1997:

• How many immunizers are electronically exchanging immuniza- tion records with a regional IIS server?
• How many individuals have their immunization records in a regional IIS server?
• Of the immunization records captured, what percent are accurate and complete?

These and other evaluation activities are reported and discussed by all the participants in annual working group meetings where each site shares its progress and problems with other public health officials from around the state.

Annual Financial Reports: Looking back for future success

The NYS Office of the State Comptroller (OSC) wanted to design its automated annual reporting system to be useful to localities for as long as possible. OSC recognized that the longevity of the system would depend not only on how well it was initially designed, but on how well it could continue to meet local needs. This meant that communication channels with the customers would need to be maintained, and that periodic reviews of the system would have to take place so that changing environmental factors and functional requirements could be accommodated. OSC let its local customers know that it is committed to the longevity of the system by identifying and correcting problems as soon as they become visible. Customers were encouraged to keep in contact with OSC staff, and to report any problems right away. In turn, OSC keeps in constant communication with system users by informing them of system changes and by demonstrating by their responses to requests that they are listening. The result? A system that is not only accepted but appreciated by its users, with the potential for being around for a long time.