The NYC Administration for Children’s Services (ACS) initiative to test mobile technologies in child protective services was originally developed in response to Mayor Bloomberg's "Safeguarding our Children 2006 Action Plan." The Plan instructed ACS to "deploy handheld computers or tablet PCs to field office workers." In response to this, ACS's Division of Child Protection (DCP) worked in conjunction with ACS Management Information Services (MIS) to develop a plan for piloting testing the use of portable wireless technologies for child protective service (CPS) caseworkers. The first phase of the pilot took place in spring of 2006 and lasted for three months. This report presents the results of an assessment of the technology in the second, larger, extended pilot project

The original portable information technology pilot project was a direct result of the NYS Legislature’s Laws of 2006 where in Chapter 58 was a requirement to “conduct a pilot program in New York City, Westchester County, and Monroe County to test best practices in portable for child protective caseworkers…[to] allow such caseworkers to complete effectively tasks necessary for their investigations of allegations of child abuse and maltreatment from field locations.” In December 2006 OCFS submitted a report to the Governor, detailing the impact of the pilot program on caseworker efficiency, productivity and caseload and a recommendation for continued testing.

Based on this report, and in order to learn more about the use of connected laptops within CPS, New York City's Administration for Children's Services (NYC/ACS) decided to undertake an extended pilot test of wirelessly connected laptops deployed to caseworkers and supervisors in two NYC field offices. This extended pilot, utilizing funds remaining from the NYS Portable Information Technology Pilot project as well as city finds,  begun in July 2007 with data collection concluding in October 2007. 

The overall goal of the initiative was to provide CPS caseworkers with remote access to CONNECTIONS (the OCFS central child welfare information system) and other ACS applications to allow them to accomplish their reporting activities while outside of the office. Specifically, the program was to enable caseworkers to use time spent waiting for appointments or court appearances, which often involve several hours of waiting time, to complete their required case documentation. Overall, ACS has approximately 1,310 CPS staff in five boroughs which investigates approximately 70,000 reports of suspected child abuse and neglect a year.

The approach used in this assessment is based on results from the first mobile technology pilot in 2006, which showed that any technology use depends on the following main factors:

• The overall capabilities of the device
• How well the device fits in with the users’ normal work practices
• How well the device fits in with personal work preferences
• Nature of environment (physical and organizational) in which work is performed

Results from the first phase showed the perceived benefits of a laptop computer in reporting and documentation. Unlike the other technologies tested, laptop computers offered direct connectivity to the central database providing the potential for increased opportunities for where and when work is done, and access to information from the central database and other online sources while in the field. In comparison, the use of a dictation service with a cell phone has the potential to have data automatically entered into a central database. Figure 1 below describes two main tasks in which various technologies differ in their capabilities (1) when converting the users’ words into digital text and (2) when entering the text into the central CONNECTIONS database.

Figure 1 - What Can be Done In the Field

It was also found that to derive the most value from a mobile device, the following is required (1) mobile devices with access that provides real time entry into the central database application and (2) connectivity with ubiquitous access. If the two components are not present, the potential value of the mobile technology will not be obtained.

The first phase results indicated that requiring caseworkers to adopt a new technology and change the way they do their work proved to be a significant obstacle for many. Any technology must fit well with individual work practices to produce value. The advantage of laptop computers was that it required relatively small adjustments in work practices, being similar to an office PC, except for connection and logon procedures. But not all caseworkers are comfortable working in the same way and these individual skills and preferences, therefore shape the use. Some caseworkers avoided using technology during a home visit, preferring to focus on the family and rely on handwritten notes, which can limit the potential value of the technology.

Mobile technology’s effectiveness can be strongly influenced by that environment, as well as by organizational support and the goals for work to be done in the field. The work environment of CPS workers presents any potential challenges to an effective use of mobile technologies such as privacy concerns, timing issues, and personal security. The extent to which these potential challenges become obstacles to use depends on the personal circumstances of each CPS worker. Thus workers who rely on public transportation may use technology differently than those who have a dedicated car. For some, time between visits may be too short to complete work, whereas large blocks of time waiting in court can be very productive. Some individuals may need a quiet work space to enter notes; others may be more adept at working in public spaces. The constraints of the workers’ environment and the organizational support from superiors can shape use of the technology and the costs and benefits that result from that use.

The Center for Technology in Government (CTG) was asked to conduct the independent assessment of laptop use in the CPS work setting and the impact on CPS work. These four categories of investigation emerged from preliminary discussions with OCFS, ACS-MIS, and ACS-DCP:

• Efficiency/ Productivity - measured by factors such as changes in number and timeliness of documentation (i.e. progress notes, safety assessments), change in the number of cases closed and reports of ability to use time differently and/or more effectively.
• Types and Locations of Work - the types of work activities the laptop computers were used for and where they were being used most frequently. It also contains investigation of barriers/issues encountered by CPS workers in specific locations.
• Effect on Current Work Practices and Policies - how work practices changed with the introduction of technology and how policies and management practices may impede or promote the use of laptops.
• Overall Opinion and Satisfaction - effect of laptop use on workers’ overall job satisfaction, work-related stress levels, and satisfaction with using the laptop, including willingness to recommend the laptops to other CPS workers.

The Panasonic Toughbook W5 was selected for the extended pilot, based on the recommendations from the first phase of testing in 2006. Its predecessor, Panasonic Toughbook W4, was rated by the first phase’s participants most favorably on account of its weight, size, and battery life (see the appendix for a detailed device description). Per ACS request, each device was equipped with an internally mounted Verizon Wireless Wide Area Network (WWAN) card and the laptop memory was upgraded to 1GB of RAM.

Each laptop connected remotely to the internet, the ACS network, and CONNECTIONS via the built-in Verizon WWAN card for access to the Verizon cellular network (where available). Although the Manhattan and Staten Island area coverage per Verizon information was 100%, the actual connectivity varied considerably. A typical session using ACS Remote Access to use the ACS network and CONNECTIONS would last between 30-60 minutes before the user needed to reestablish a connection to the system.

Security provisions included deterring theft and ensuring data and network integrity. Locks were distributed to each pilot participant to secure the laptop to a fixed location. In addition, each laptop hard drive was encrypted using BeCrypt data security software. Finally, access to the ACS network and CONNECTIONS was protected by several layers of passwords designed to prevent unauthorized access to sensitive client data. No security-related problems or breaches were reported for the test period.

Figure 2 - CPS Connectivity to CONNECTIONS System During the Pilot

As shown in Figure 2, using the laptop required the user to execute four password-protected logons: the first provided access to the encrypted laptop itself; the second log-on provided access to the server at DOITT; the third log-on provided access to ACS’ remote access server; and the fourth provided access to the CONNECTIONS database. Although ACS MIS staff recognized that a multi-layer password protection would present usability problems for the pilot participants, they were unable to eliminate any of these layers prior to the beginning of the pilot due to requirements imposed by NYC’s Department of Information Technology and Telecommunications.

The assessment covered a four month period from mid June 2007 to late October 2007. There were four streams of data collection throughout this project. Surveys of pilot participants and data from the central OCFS CONNECTIONS database were used to assess productivity, user satisfaction and timeliness, and patterns and locations of laptop use. This data was supplemented by material from workshops with CPS workers in both field offices and telephone interviews with CPS supervisors and managers from both boroughs (a more detailed description of the methodology is in Appendix B). A baseline survey was distributed in June 2007 to all participating caseworkers. All participants underwent training and were given laptops by the end of July. Five workshops with Manhattan and Staten Island caseworkers were conducted at the end of August, approximately four weeks after the beginning of the pilot. In October, the research team conducted teleconferences with supervisors and managers from both boroughs and distributed the post survey to all participating caseworkers. CONNECTIONS data collection occurred in the pre-pilot period from April 29, 2007 – July 21, 2007 and during-pilot period July 29, 2007 to October 19, 2007.

One hundred and ninety ACS managers, supervisors, and caseworkers volunteered to use the laptops in the pilot test. Of those 190, 135 caseworkers and supervisors from the William Street and Staten Island field offices participated in the assessment. All part received training prior to deployment and all received the same model and configuration of laptop. Prior to receiving the laptops, the volunteers also received a survey to establish a baseline concerning their attitudes and work practices against which to compare a similar post test survey’s results. They did not, however, receive special instructions on how to employ the devices in their work.

The pilot design included two boroughs, with the expectation that different geographical and transportation characteristics might result in different use patterns. Manhattan field workers would face different obstacles when trying to use the laptop among high-rise buildings. In addition, population density and prevalence of public transportation for CPS work in Manhattan were likely to present different problems, compared to Staten Island’s more suburban landscape and more common use of cars by CPS staff.

The pilot participants in both field offices all volunteered and were assigned their laptops for the duration of the pilot period. Prior to receiving a laptop computer, each participant attended a three-hour orientation and training session, which introduced them to the device and provided training on connecting to the ACS and CONNECTIONS networks (copies of training manuals can be obtained by contacting ACS). In addition to the practical training, caseworkers were encouraged to contact the ACS help desk in case they experienced any problems with their device. To ensure comparability of data and work conditions, supervisors and managers whose caseworkers were issued a laptop were instructed not to modify any policies or work practices during the pilot period.

Of the laptop computers, 135 were deployed to caseworkers and supervisors from two ACS field offices: Manhattan (70) and Staten Island (65).¹ Of these 135 participants, 95 replied to both, the base and post pilot survey, creating a total population of 42 caseworkers and four supervisors for Manhattan and 46 caseworkers and three supervisors for Staten Island. Overall, the participant populations in both field offices exhibited a lot of similarities in terms of their CPS experience and general computer skills, while displaying noteworthy differences in the length of time spent working overtime and waiting in court. Also, as expected, the two field offices reported significantly different patterns of transportation use.

The pilot test group had a generally low level of CPS experience, averaging just under 4 years, with 57% of all caseworkers reporting CPS experience of two years or less. The participants also rated themselves relatively high on technology skills; the majority rated their skills as intermediate or higher in all surveyed categories ranging from general computer use to the use of CONNECTIONS. Most respondents did not report a need for future training, with the exception of about 39% needing training in CONNECTIONS.

Graph 1 - Level of Skills Within Assessment Population

Two noteworthy areas where Staten Island and Manhattan reported significant differences were in (1) the average length of weekly overtime and (2) the average length of court wait time. In both, the pre-pilot and pilot periods, Manhattan caseworkers reported using nine hours of overtime a week or less (average of 5.32 hours). At the same time, the majority of Staten Island caseworkers in both time periods reported the average length of overtime to be ten hours and more (average of 9.37 hours). Interestingly, the amount of reported overtime use decreased for both boroughs during the pilot period, which is consistent with caseworkers’ testimony during our workshops in which they indicated that the timing of the pilot coincided with a seasonal slow-down in incoming cases.

Graph 2 - Overtime Averages for Manhattan and Staten Island Field Offices

The average court waiting time reported and the number of court appearances showed a similar pattern: Staten Island had significantly more wait time in court compared to Manhattan and slightly more frequent appearances. Almost 70% of Manhattan caseworkers reported waiting in court five hours or less per typical court appearance, compared to 60% in Staten Island, who reported waiting five hours or more per typical court appearance.