The New York City Administration for Children’s Services’ (NYC ACS) strategy to test mobile technologies was originally developed in response to Mayor Bloomberg’s “Safeguarding our Children 2006 Action Plan.” Over the last two years, in conjunction with the NYS OCFS and the state legislature, NYC ACS provided funding to deploy and test the use of mobile technologies in Child Protective Services (CPS) work. During the weeks of July 16 through July 27, 2007, ACS deployed 190 Panasonic Toughbook to managers, caseworkers, and supervisors. Of the 190, 135 caseworkers and supervisors in two field offices – Manhattan and Staten Island – received laptops.

Following this deployment in July 2007, an initial assessment of the use of laptops in CPS work took place. The initial assessment examined how mobile technology affects CPS caseworker productivity, mobility, and satisfaction. This extended assessment examined similar areas over a longer period of time totaling ten months.

At the time of data collection, ACS had approximately 1,310 CPS staff in five boroughs which investigates approximately 70,000 reports of suspected child abuse and neglect a year. The overall goal of the initiative was to provide CPS caseworkers with remote access to CONNECTIONS (the OCFS central child welfare information system) and other ACS applications in order to allow caseworkers to complete reporting activities while outside of the office. Specifically, the goal was to enable caseworkers to use time spent waiting for appointments, in between appointments or during court appearances to complete their required case documentation.

NYC ACS provided internally mounted Verizon Wireless Wide Area Network (WWAN) cards and access to the city network went through several passwords (i.e., one log-on provided access to the server at NYC’s central IT office; another log-on provided access to ACS’ remote access server) designed to prevent unauthorized access to sensitive client data. During the initial assessment, access to the State network (i.e., the state central database) was through NYC ACS networks. After technical difficulties from this arrangement substantially slowed connections to the state’s central database, NYC allowed access to the state network through a virtual private network (VPN). This practice was consistent with other districts across the state. In addition, each laptop hard drive was encrypted using BeCrypt data security software.

Prior to receiving a laptop computer, each participant attended a three-hour orientation and training session, which introduced them to the device and provided training on connecting to NYC ACS and CONNECTIONS networks.

This profile is specific to NYC ACS and brings together the most comprehensive data on the two data collection periods as well as findings on use, mobility, productivity and satisfaction.

The overall objective of the laptops was to provide caseworkers with opportunities to work outside the office environment in new ways. This section reports on how participants used those opportunities in terms of 1) the type of work done, 2) locations of use, and 3) factors influencing their use. Additionally, this section reports on the major technical problems reported by the caseworkers. The data used in this analysis was collected soon after the laptops were deployed, as well as after an extended period of use. See Appendix A for a full description of the methods used.

During the first data collection period, survey respondents reported using the laptop during normal work hours, after hours, during commute times, and while working overtime. The laptops were used in case investigation and interventions, documentation and reporting, and court-related activities. Case documentation was the most frequent use, including inputting and updating notes, and completing safety assessments. Other work included reading and reviewing case histories or new cases, doing person searches or checking client histories, conducting clearances, email, accessing government or other Web sites, and looking up services for clients. During the second data collection period, respondents reported using the laptops in similar ways.

In the first data collection period, caseworkers reported the following benefits to laptop use: 1) access to information in the field, and 2) less frequent need to return to the office to access case information. For example, one caseworker stated, “It increases caseworkers’ opportunities to access services while in the field and affords the CPS worker more independence in acquiring clearances.” Similar trends were reported nearly one year after the laptops were deployed.

As part of the first data collection period, caseworkers were surveyed on where they used their laptop, as well as the average length of time they used it. Table 6 below represents findings from the first data collection period.

Table 6 – Location and Hours of Use per Week

	Overall (n)	Average length of use per week	Manhattan (n)	Staten Island (n)
Home	86 % (82)	4.47 hours	89 % (41)	84 % (41)
Court	44 % (42)	2.34 hours	44 % (20)	45 % (22)
Field	42 % (40)	2.33 hours	35 % (16)	49 % (24)
Office	6 % (6)	0.30 hours	0 % (0)	12 % (6)
Do not use at all	4 % (4)	--	2 % (1)	6 % (3)

As noted from the table above, the majority of caseworkers used the laptops from their home, followed by use in court and the field. The data gathered also indicate that caseworkers used the laptops from home for approximately four and a half hours a week. Several stated they were more productive at home due to fewer interruptions from their colleagues, while others stated that technical problems deemed them less effective than at the office as it took them longer to do the same amount of work. While no survey was used in the second data collection, teleconference participants reported using the laptops primarily from their homes after normal business hours. Supervisors also indicated that they use the laptops at home to review their cases on an as-needed basis (e.g., depending on workload).

In the first data collection, technical problems (i.e., connection problems) and work environment issues (i.e., privacy or perceived physical danger) were important factors that shaped the use of laptops in the field, particularly in clients’ homes. During the second data collection, caseworkers again reported using the laptops from the field and similar technical difficulties were reported. As well, many caseworkers noted it was nearly impossible to use the laptops when in clients’ homes or in client meetings. Several concerns were noted: slowness of connection, delayed connection, and impersonal interaction. Caseworkers explained the nature of casework documentation requires quick notes when visiting clients and using a laptop generally interferes with communication.

In the first data collection period, the amount of time caseworkers spent in court suggested that it was possibly an important location for mobile work. However, caseworkers reported privacy and connection problems at court as barriers to use. The second data collection indicates that caseworkers still have similar barriers when using the laptops in court. One caseworker stated their preference would be to have a dedicated area for caseworkers to use laptops while waiting to be called for cases. Respondents noted a dedicated area in family court in Manhattan, but reported difficulty hearing cases being called and therefore, did not feel comfortable using that area. Most caseworkers reported using the court computers instead (i.e., they are considered faster than the laptops). Family courts in Staten Island do not have dedicated areas and barriers such as inadequate privacy and confidentiality were reported.

In the first data collection, it was suggested that caseworkers will be able to use their laptops during their commutes to and from work, but it seemed that the nature of the commute (i.e., using various forms of public transportation or dealing with traffic) may not allow optimal conditions for laptop use. During the second data collection, some caseworkers reported using laptops successfully during their commutes. One caseworker mentioned the laptop was convenient for working on progress notes when on an above-ground train. The caseworker’s commute to and from work was about 45 minutes and this time frame allowed the laptop to be used to catch up on work.

During the second data collection period, caseworkers stated that having the laptop allows them to shift the time and place of where they work and to do work in non-traditional work locations. For example, one caseworker used the laptop in the park and in a coffee shop. The caseworker commented on doing this about once a week and self reported being mostly caught up on cases.

In the first data collection period, the most common noted technical problems were related to the wireless connection. Specifically caseworkers reported problems establishing and maintaining a wireless connection. The speed of the wireless connection was also reported as problematic. Additionally, a number of caseworkers reported problems related to the lack of privacy and confidentiality when using the laptop in the field. .

Participants in the first data collection were surveyed and 48% of participants rated the log-on process as “Very difficult” to “Extremely difficult,” compared to only 20% who described it as “Easy.” An additional 28% of the participants rated the log-on process as “Neither difficult nor easy.” While no surveys were used in the second data collection, interviewed caseworkers reported continued programs with the log-on process and described it as cumbersome and time consuming.

Overall, nearly a year after the laptops were deployed, caseworkers reported relatively few new technical challenges using the laptop in the field. The same issues were reported in the second data collection period, but were not considered major barriers to use. Most commonly, caseworkers continued to experience lengthy system boot up times, continuous disconnection from CONNECTIONS, and lagging wireless connection speeds. Several caseworkers from Staten Island reported wireless difficulties, but attributed this mostly to the lack of coverage in the court. One caseworker recommended experimenting with the use of printers in the field.

This analysis uses central database data to examine two core questions about possible technology impacts within the New York City ACS: 1) Are workers with laptops more productive with respect to case closings, safety submissions, and progress note reporting? and 2) Does laptop use have an effect on the timeliness of reporting? Additionally, this section presents the findings based on an analysis of the perceived usefulness of the laptops. See Appendix A or a full description of the methods used.

Case closing is one way to assess any changes in efficiency and productivity.Figure 13 below shows the volume of timely closing of cases (in 60 days or less) increased during the test period, up from 916 in the pre-test period to 1,527 during the test period. The number of cases closed that were over 60 days old decreased from 851 to 503 during the test period.

 

Figure 14 below shows the percentage (or proportion) of timely closing of cases (in 60 days or less) out of the total increased during the test period, up from 52% to 75%. It is important to note that in this county, the total number of cases available to be worked also increased from 2,090 in the pre-test period to 2,671 during the test period—a 28% increase. Caseworkers improved their percentage of timely case closings (in 60 days or less) while absorbing a 28% increase in cases available to be worked on. This represents is a marked increase in productivity. Also, the closing of cases after 60 days dropped, indicating that the prior ‘catching up’ effect was not present. Overall, this suggests that caseworkers, overall, complete a higher percentage of cases on time. The length of the test period during the second data collection was 293 days. However, a positive trend in cases closed was also seen during the initial assessment, which lasted 82 days.⁶

 

Another important indicator of a positive productivity trend is the change in the period when the laptops were taken from caseworkers for 76 days within the test period (for technical maintenance). Figure 14 above shows the percentage of timely closing of cases (in 60 days or less) out of total cases slightly decreased during the no laptop period, from 75% down to 67%. The percentage of cases closed in over 60 days old slightly increased from 25% to 33% during the test period.

The rate of completing safety assessments is another way to assess any changes in efficiency and productivity. Figure 15 below shows the volume of timely (in seven days or less) submission of safety assessments increased during the test period, from 1,259 in the pre-test period to 1,440 during the test period. The number of safety assessments submitted that were over seven days old increased slightly from 443 to 483 during the test period.

 

Figure 16 below shows that the percentage of timely (in seven days or less) submission of safety assessments as a percent of total cases changed little during the test period. In the second data collection, caseworkers maintained their already-high level of safety submission (approximately 75 percent) despite a 28% increase in caseload. Seventy-five percent of safety assessments submitted within 7 days leaves only 25% where improvements can be made. Therefore, the overall timely submission of safety assessments is already relatively high.

 

An indicator of timeliness is elapsed time, defined as the number of days between an event and the posting of documentation regarding that event in the central database system. Figure 17 and Figure 18 below show trends in the elapsed time between progress note entry and the related event. During the pre-test period, the majority of all progress notes were entered by the fifth day following the event. Figure 17 shows that the number of progress notes entered (i.e., volume) rose significantly during the test period from 33,738 in the pre-test period to 39,474 during the test period – a 17% increase. Figure 18 shows that the rate of progress note entry increased very little during the test period however, caseworkers were able to maintain that level of entry while increasing volume of notes by 17%. Therefore, productivity increased overall. This level of entry is consistent with findings from the initial assessment.⁷

 

 

During the first data collection period, participants were surveyed for their perceptions as to whether using a laptop made any difference in their CPS work. The survey included questions on five areas: timeliness of documentation, ability to work from court, ability to access case information, communication with supervisors, and services provided to clients. Survey respondents were asked to rate the difference on a five-point scale where “1” indicated “Much worse,” “3” represented “About the same,” and “5” was indicative of being “Much better.”

Overall, respondents from NYC ACS reported some positive impacts on their work resulting from laptop use. A summary of the findings is shown in Table 7 below. As the table shows, 67% of the respondents reported improvements in timeliness of documentation, and 78% reported improved ability to access case information. Ability to work in court improved for 49% of the respondents, and 33% reported improvements in ability to communicate with supervisors. Lastly, 29% percent reported improvements in service to clients. Only a few caseworkers reported negative impacts as a result of using the laptop to conduct CPS work.

Table 7 - Perceived Change Timeliness and Work Impacts – New York City

	Much worse (n)	Somewhat worse (n)	About the same (n)	Somewhat better (n)	Much better (n)
Timeliness of documentation	1 % (1)	0 % (0)	32 % (30)	48 % (45)	19 % (18)
Ability to access case information	2 % (2)	1 % (1)	19 % (18)	45 % (42)	33 % (31)
Communication with supervisors	0 % (0)	1 % (1)	66 % (61)	20 % (19)	13 % (12)
Service to clients	2 % (2)	0 % (0)	69 % (65)	17 % (16)	12 % (11)
Ability to do work in court	3 % (3)	3 % (3)	44 % (40)	28 % (25)	21 % (19)

Data gathered from the videoconferences during the second data collection support the general findings from the first data collection. While few caseworkers reported changes in the quality of service they provide to their clients, one caseworker stressed that the benefit of the laptop is to directly assist the caseworkers, not families. Many of the interviewed caseworkers valued the ability to lookup case histories regardless of time of day and location. One caseworker described their ability to do so as “a luxury”. Other caseworkers indicated the laptop is useful for catching up on progress notes and entering information in a timely manner. And one of the caseworkers from the first data collection period stated, “If I can't sleep at night because of all the stress that results from a build up of casework activities that are not completed, I can complete case documentation at home during the evening to reduce some of the work I will have to do the following day.”

Personal preference was a consistent theme during the first data collection period. For example, some caseworkers preferred to use the laptop at home after normal business hours, while others preferred to use the laptop to enter notes immediately following a visit. Caseworkers interviewed during the second data collection period reported no change in the way they communicate with their supervisors.

In the second data collection, caseworkers reported the laptop was valuable for entering notes into CONNECTIONS, as well as accessing information from CONNECTIONS while out of the office. To highlight the benefits of the laptops, one of the interviewed caseworkers conveyed a story about preparing for a court appearance at home the night before, then going to the office the next morning, printing the court report, and being ready.

In the first data collection period, survey respondents reported moderately high levels of satisfaction. The survey data showed that 67% of respondents reported being “Somewhat satisfied” or “Very satisfied;” compared to 18% of respondents who reported being “Somewhat dissatisfied” or “Very dissatisfied.” An additional 15% of respondents reported feeling “Neither satisfied nor dissatisfied.”

Despite moderately high levels of satisfaction during the first data collection, caseworkers reported various barriers to use. During the second data collection, satisfaction was again high and some barriers to use remained including lack of comprehensive policies on acceptable use and compensation, and the technical challenges previously described (i.e., connectivity, boot up time, and privacy). Some caseworkers expressed their frustration about inconsistent policies, and several stated they may stop using the laptop altogether if policy issues are not addressed. Caseworkers added that defining what is considered acceptable use and specifically informing supervisors on what should be expected from caseworkers is an important step to using laptops.

During the second data collection period, CPS caseworkers reported several incentives associated with using the laptops, including a desire to increase self satisfaction and a need to reduce work-related stress. Despite not being compensated, many caseworkers reported self satisfaction and the desire to keep up on their progress notes as the main drivers behind using the laptops from home. Furthermore, caseworkers reported high satisfaction with the ability to shift work tasks around during the day. One caseworker reiterated that since having a laptop, they are able to go home at a reasonable time, care for their family, and catch-up on work as needed. A number of the interviewed caseworkers reported not wanting a laptop initially, however, after participating in the project, they are happy to have experimented with mobile technology. Almost all caseworkers said they would recommend using a laptop to colleagues, however, a few would caution their fellow colleagues about a potential work-life balance tension.