Of the repositories selected for this study, five conducted comprehensive operations, consisting of data collection, analysis, storage, and access provision. That is, their repository function was integrated with a role as the major or exclusive originators of the information resources, or taking a substantial part in that data collection process. Some had large-scale data collection operations internal to the organization, as in the case of the Census Bureau, BLS, and NYSDOH. The others had a major role in administering or sponsoring the processes that resulted in acquisition of information. All of them had primary responsibility for providing access to these resources, including policies on confidentiality and use.

As comprehensive repositories, these organizations maintained a somewhat different set of relationships with users and those involved in supplying information. Much of the supply of information is from units within the organizations, other government agencies with which it has functional relationships, or contracted data collection by external firms or other government agencies. The intake of information is thus largely under the control of the repository or regulated by policy, especially for statutory collection and reporting requirements such as the decennial census, quarterly inflation indicators, crime statistics, or educational assessments. In addition to required information collection, these agencies can be proactive with respect to additional research programs that generate new information flows. They may conduct these studies with internal staff, or contract for the data collection through commercial organizations, other government agencies, or research organizations, such as universities.

The flow of information into and out of these repositories is largely regulated by the agency and its legal and policy framework. In the case of the NY State agencies, much of the information flows both into and out of the repositories is confidential and thus limited to specific, legally sanctioned users and uses. This also applies to some of the data in the FHWA, BLS and Census files. For the BLS, confidentiality is in some cases a temporary constraint, since certain economic statistics (e.g., inflation and employment indicators) have important financial and political implications. That information is embargoed until the regularly scheduled release time. Premature release is illegal and punishable.

Providing appropriate access is complicated by the mix of users, running from lay persons seeking small targeted items of information for personal use (e.g., parents seeking information about a school system), to policymakers working on national issues, to researchers seeking large data sets, ad hoc queries, or new sophisticated analyses. The ones in this group that serve the general public, primarily the BLS, and Census Bureau, had therefore invested heavily in interactive access capabilities and online analysis and query tools. These provide efficient ways of supporting large volumes of user interactions with a limited staff. They reported focusing professional staff resources more on responding to requests from policy makers and the research community. Their attention to the needs of their user community is also reflected in substantial investments in user support.

This group of comprehensive repositories also paid considerable attention to the problems of multiple data formats and migration. Some of these concerns are a direct result of these agency’s roles in long-term retention of government records and statistics. Even if the agency has direct control over the formats of data at the collection stage, the need to deal with both emerging new and obsolescent old formats remains. This is a particular problem for large agencies, such as these, that support many diverse Web sites, each dealing with a particular program or policy area. The current FHWA Web presence includes 40 separate Web sites, each with distinctive information content and format requirements. The FHWA’s Highway History Web site, for example, includes an html version of the first issue of Public Roads, (Vol. 1, No.1) from May 1918. There is the additional need to provide data in digital formats to other government agencies with different formatting requirements, such as in the case of the Census Bureau and the New York State agencies. In spite of the exemplary practices these agencies have developed to deal with multiple formats, the problems will most likely persist, due to the combination of technology change and increasing conversion to digital formats.