Skip to main content
photo
 
APPENDIX A: Methodology

The overall objective of the reconnaissance study was to identify and summarize the state of international digital government research. Two main efforts contributed to this study; online Web searches and a traditional academic literature review.

The purpose of the online Web searches was to identify organizations, research projects, and events that focused on international digital government research. This phase relied heavily on the use of Internet search engines and a set of keywords commonly used within the domain of digital government/e-government/e-governance. Web searches were conducted in English over a 12-month period during 2007 and were iterative in nature. Examining the various search results allowed for the identification of other possible resources that initially did not appear in the Web searches. Successful search results were stored in a database along with basic identifying information such as the main URL and contact information.

Organizations identified during the Web searches were classified by type into one of the following six categories: think tanks, university research organizations, non-profit research organizations, government agencies, intergovernmental entities, and private corporations. In addition, organizations were tagged as to whether they conduct or sponsor international digital government research. World regions of interest to these organizations were also identified and recorded in the database. Additional information, if available, was collected for organizations that sponsor research. This information included their annual funding devoted to international and comparative research projects, the number and types of grants they provide, and their major publications and events they sponsor.

Research project data were also collected. Where it was available, information was collected regarding sponsor, the main topic or problem addressed, key collaborators, grant amount, project duration, main methodologies used, countries included in the study, and key findings.

Finally, information was collected regarding the sponsors and organizers of major international events that focus on digital government research and appear to welcome international work. Information collected included the topics presented and world regions of interest.

The second major effort that contributed to this study was a review of research in relevant journals and conferences published in English in print and online from 1994 through 2008. Several sources were used during this effort, including two published libraries specializing in digital government research and numerous online library databases. EndNotes, a bibliographic software package, was used to manage citations throughout this process.

The first specialized library used during the literature review was published by the North American Digital Government Society. The library was compiled and is periodically updated by scholars at the University of Washington. The references included in this library were published from 1994 through 2009. To collect these references, they systematically scanned traditional journals in information science, public administration, and public policy, as well as references from special issues on e-government in other journals. Research from general e-government conferences such as DEXA/EGOV, eChallenges, HICSS EGOV Track, and dg.o, in addition to topic-specific conferences (e.g. mGov and eVoting) were included in the library. References were included only if they were four or more pages in length and were peer- or editor-reviewed. The second library used was published by the Special Interest Group on Electronic Government at the Association for Information Systems, which was compiled by scholars at the Copenhagen Business School. The references obtained from both published libraries were combined and duplicates were removed, resulting in over 2,000 unique references. Our research team extracted an annotated bibliography for all references that included keywords similar to those used in the domain of international digital government/e-government/e-governance and isolated these references in a new EndNote library. The purpose of this step was to exclude references that were neither international nor comparative in scope.

In an attempt to ensure that the literature review was comprehensive and exhaustive, we then conducted searches on numerous library databases using keywords regularly used in the domain of digital government/e-government/e-governance. The library databases that were used in this process included: EBSCO Academic Search Premier, Emerald Fulltext, Social Sciences Abstracts, and Business Source Premier. This process identified several references from 1994 through 2008 that were not included in either of the two specialized libraries described above.

Each reference was then classified according to the type of study by two team members based on title and abstract. This strategy was employed to ensure inter-coder reliability. The types included benchmarks, comparative studies, best practice studies, fundamental issues studies, regional studies, and transnational studies. Both team members needed to agree on the classification in order for it to be classified successfully. The complete paper was obtained for further review in the event that the two classifications differed. The team members scanned the complete work and attempted to classify it once again, however, if the discrepancies remained, an expert in the field of digital government classified that particular reference.

To obtain a more detailed picture of this body of work, 20 percent of the 276 articles were selected at random from each of the research types listed above. To do this, the citations in each category were sorted by authors’ last name and numbered, and an online random number generator (www.graphpad.com/quickcalcs/randomN1.cfm) was used to decide which references would be reviewed thoroughly. In total, 59 of 276 references were sampled (19 benchmark studies, 16 comparative studies, 7 fundamental issues studies, 7 regional studies, 5 best practice studies, and 5 transnational studies). We read these articles carefully and used them to help us write the summaries and several were selected to highlight as illustrations of each type.