Traditional prototypes provide programmers and program staff the ability to explore concepts and test assumptions. What prototypes and pilots do not provide is a way to look beyond the technical application to the social and organizational challenges. Expanding the traditional concept of prototyping to the comprehensive prototyping model within a Testbed approach allows for the exploration of technical, organizational, and policy implications.

In this project, comprehensive prototyping within the Testbed model enabled the teams to grow into the project by balancing capability with support. If a team had lower capability at the start (based on the four dimensions of readiness, confidence, communication, and knowledge outlined at the start of this chapter), they may have needed more outside support from CTG staff in the beginning. As they progressed through the Testbed and their capabilities increased, the need for support dropped off. The Testbed framework provided each team with additional support:

• through knowledge gained in training and the workshops;
• by alleviating project management responsibilities through the external framework of the Testbed model and the CTG teams; and
• by providing an environment where the teams could meet and work on their individual projects.

As seen throughout this report, a project such as implementing XML for Web site management impacts organizations, workflow processes, technical resources, and willingness to innovate. What at first may appear to be a purely technical task (XML) is seen on closer examination to involve much more. Due to this complexity, traditional training and system development approaches may not be sufficient to carry the project through to successful implementation.

The Testbed model, which can be implemented within a single agency, is specifically designed to fill the gaps left by the traditional methods and provide the additional levels of needed support. Training, for example, is delivered in the context of a specific product development or process improvement. More importantly, the training is delivered across organizational units and encompasses technical and business process areas (so-called “hard skills” and “soft skills”). Project teams are given time and support to apply what was learned in the training directly to the prototypes and business cases they develop. Executive support is procured at the start and nurtured throughout the process.

The Testbed teams provided the most convincing validation of the value of the approach, primarily in their prototypes and business cases, but also in their reflections on the process. Some saw value in the cross-organizational team makeup of the Testbed:

“A real benefit in just going through this entire exercise was the team. In a lot of ways, it was a real eye opener for [the other units] in terms of what we’ve got to go through ... what it takes for us to do this.”

Others stressed how the Testbed created the work environment that they needed to make their case: “[The Tested] certainly convinced me that technically it’s doable and technically it’s a great thing. We had suspected, but I wanted to actually prove that; I wanted proof of that theory.” That proof of concept enabled the teams to take the next step forward: “Without this project, I don’t think that we would’ve had the confidence to put a business case forward ... I think it gave us the confidence; it gave us the tools.”

The Testbed environment also allowed the teams to investigate all aspects of the innovation more thoroughly, “to look at the whole thing, to take it and break it down step by step” as one participant put it. The outcome, according to this person, was that “we changed a lot, we really did. We thought our problem was one thing and in reality it was something different.” Another participant saw benefits beyond this one project: “I think the business case development was a good exercise, not just for this particular focused area but for anything that’s coming down the road; those were good skills to learn.”

And even though the Testbed focused on a prototype development, not a full-fledged system implementation, that did not diminish the impact of the work. As one team member remarked, “I think we actually went above and beyond what we thought we were going to do ... And so just by looking at [the prototype] and just like seeing what we could do with it, I think it was just such a positive; it was just such a great feeling. And I think everybody on our team was really happy with it.” Or, as another said: “I’m always amazed in looking back at how much we did accomplish in such a short time.”