The range and complexity of government IT investments makes assessing their returns a daunting task. Just like each of the project teams, the Electronic Commons Program administrators had to determine if in fact this project was money well spent. Was this a good use of the agency’s funds? Was there a way to leverage this investment to help spawn new initiatives? Did we create a cohort that could be used as a resource for future grantees?

Creating the framework for assessing these questions is in fact a challenge in and of itself. The measures will need to show an increase in efficiency, increase in effectiveness, an enablement of some kind within the organizations or stakeholders, and intrinsic enhancements. Achievement of each of these measures was seen in the discussion and engagement at the November Reflection Workshop, as well as the final reports from the grantees. Based on the grantees own evaluation of their projects, each of these factors were able to be measured through their program results.

The USDA Forest Service

Northeastern Area News Notes
November 2006

Each project team created a different product. For projects whose primary goal was delivery of educational lectures, the impact was relatively clearly defined by the number of people they were able to reach, as well as the extent of the geographic area from which their students originated. As you can see from the examples below, the impact of these types of projects, especially when accounting for the small size of the grant, was significant. With a small investment, the project teams were able to reach hundreds of interested individuals coming from a wide geographic area spanning close to 30 U.S. states. In addition to counting their students, some teams also conducted post-Webinar surveys to gain feedback from their students as to the method of delivery as well as the content of their Webinar.

Measuring impact was more difficult for those projects with a primary goal of creating a Web site to serve as an educational resource. In these projects the typical debate about the value of a “hit count” as an indicator of success took place. Simply counting the number of visitors to the site is not generally seen as a testament to the usefulness of the Web site itself. The White Tail Deer project team resolved this issued by conducting an “informal survey” of a number of known experts in white tail deer management field. The experts were asked to evaluate the Web site for content. Of the 195 experts surveyed, 91% stated they would recommend the Web site to their colleagues. Other teams used less formal means that made sense in their context. For example, the Augusta Springs project team, who created their Web site to prepare students for their work at August Springs as volunteers, relied on observations from the natural resource educator, who noted that students coming in as volunteers “were better prepared for their roles than before the Web site became available.”

Regardless of the measurement approach, each project put forward stories to illustrate the impact that the small grant they received from the USDA Forest Service had on their efforts to share knowledge about natural resources management. Three of these stories from their final reports are presented below.

The Cooperative Weed Management Areas (CWMA) team held two in-person training sessions that they used to revise existing training materials and compress its 8-hour long session into a 2-hour training session. After revisions and careful feedback from their audience, the team was able to deliver four distance-training sessions to a total of 113 participants in the distance CWMA workshops representing 28 states and the District of Columbia. Many states had several participants, each representing different affiliations (universities, federal agencies, not-for-profits, etc.). In summary, this grant allowed the Midwest Invasive Plant Network to promote the creation of CWMAs throughout the eastern U.S. where several have already started to be organized. According to the team, this project will have ongoing ripple effects as more and more groups working on invasive plant species issues see the effectiveness of these CWMAs and decide to create their own.

The Electronic Commons Program grant had a tremendous impact on the students and teachers of the Nicollet Forest Education Network and surrounding communities. All together, 15 different projects involved 788 students in hands-on field days enabling them to learn about their environment through direct experience. Three thousand, one hundred and ninety one students (3,191) participated in the Preparatory Interactive Television (ITV) programming to prepare participating students and teachers for field day experiences. In addition, 1,060 students participated in ITV sharing sessions in which students who participated in the hands-on projects shared their experiences with their peers. One of the groups of students involved was asked to present their project, titled “The Effects of Zebra Mussels on the Water Quality of Lake Metonga” at the Wisconsin Association for Environmental Education Fall Conference. The grant also enabled the building of many valuable partnerships that will ensure that many of the grant projects continue after the funding ceases. The project has earned strong support of school administrators who amidst budget cuts have committed to continued support of these service learning projects.

Throughout the grant period, the Stewardship Network hosted ten Web casts focused on native plant restoration. Session topics ranged from logistics of restoration (seed collecting) and invasive plant control to the use of volunteers and equipment sharing programs in restoration. The Webinars were well attended with each attracting anywhere between 60 to 175 individual volunteer and professional land managers across North America. The total number of participants over the past year has reached more than 420, representing 29 states within the United States, 3 provinces of Canada and the state of Veracruz in Mexico. Each Web cast session attracted new participants as well as returning participants. Approximately 65 individuals have participated in more than one Web cast since December 2005.