Providing access to many of the information resources involved in this research requires maintaining the various levels of security and confidentiality. In this research we did not concern ourselves with the aspects of information security required to protect any electronic repository from attacks or intrusions by malicious persons or organizations. These security concerns are generic to all electronic repositories. Instead, we concerned ourselves with practices to maintain various levels of privacy and security in relation to access by authorized users. This is a particular issue for repositories of electronic information that provide access for diverse or general populations of users, but must limit access or use according to some regulatory framework. For these repositories, the practices of interest deal with controlling the conditions of access as well as controlling use of information subsequent to access.

The most elaborate set of confidentiality and security provisions in our research were reported by Federal agencies, particularly the NCES. This agency’s repositories contain some data about individuals (students, teachers, etc.) that is protected by law. Yet the agency must provide some access to these data sets to fulfil its mission to support research and policy analysis for education. To do so NCES maintains both public use and restricted use files and a Disclosure Review Board. Before a data set can be released to a public use file, the agency’s Disclosure Review Board must review it and make a recommendation to the agency head (Commissioner). Data in public use files do not identify individuals; data in some restricted use files may have such identifiers when judged necessary for research. To obtain data from a restricted use file, the user must obtain a Restricted Use Data License from NCES. These licenses, which are legally binding, specify the conditions of use and access that must be maintained by the user. The NCES employs inspectors who perform unannounced inspections at user sites to ensure that the terms of the license are enforced. ⁵

A restricted use strategy is also employed by the Census Bureau and Bureau of Labor Statistics (BLS). They employ licensing procedures to control use of restricted or confidential information. But their procedures and regulations are not as detailed and elaborate as NCES. Some BLS data is time sensitive so procedures are in place to monitor its release according to these sensitivities. However, most BLS data are available only in aggregated form and not suitable for identifying individuals. Some census data is collected at the individual level and is confidential by law (13 USC). In one particularly innovative approach to preserve anonymity, the Census Bureau has developed techniques to create synthetic data at the individual level. The technique transforms data from real individual records into new artificial records that do not represent any real person or household, but retain the statistical characteristics of the original data. The synthetic data could then be released for research without violating confidentiality requirements.

A voluntary approach to controlling use is employed for part of the Urban Institute’s repositories, the Assessing the New Federalism and National Center for Charitable Statistics data sets. The Institute requires users of the public use files from these sources to register before gaining access. As a private organization, the Institute has no statutory authority to control external user’s actions, but can use their registration information to communicate with users if a problem arises concerning how data are used. For the files in the Institute’s Federal Justice Statistics Research Center, no registration is necessary, since there are no confidentiality requirements for accessing the crime and court files. The same applies to data sets on state welfare policies in the TANF Typologies database.

A different confidentiality issue is faced by the NYDCJS. The criminal histories in their repository are a potentially highly valuable research resource, unavailable elsewhere. Studies based on these histories could provide useful new insights into criminal behavior and aid in prevention and rehabilitation. However, the legal restrictions in place on the use of these histories prevent such research by outside researchers. The agency has tried, thus far unsuccessfully, to have legal restrictions changed to allow some research of this type. In this case the agency’s mission of public safety is aligned with the research interests of scholars. So the agency is in a position to advocate for both interests and attempt to establish a collaborative research relationship through changes in confidentiality policies.