Previous pilot and demonstration assessments have established a solid foundation of information to support a reasonably clear picture of the short term impacts of deploying and using laptops in CPS work. However, both OCFS and CTG recognized the need to learn more about the long term impacts and conditions necessary for statewide deployment.

The three LDSS chosen to participate in this assessment were: NYC ACS, Onondaga County, and Wayne County. These LDSS were selected based on the length of time caseworkers had to use laptops, geographical area, favorable policy and management setting, and connectivity capability. Two data collection periods, a pre-test period and a test period, were contrasted to illustrate changes from the first data collection period to this second data collection period. For information about the data collection periods and the data collection methodology, see Appendix A.

This extended assessment, therefore, builds upon the already strong foundation of knowledge generated in the previous assessments, but focuses more narrowly on three LDSS that have long term laptop experience. In order to learn more about how laptops are integrated into CPS work, the assessment examines findings on use, mobility, productivity, and satisfaction. This report concludes with observations that shed light on concrete strategies which can assist LDSS in maximizing all current and future mobile technology investments.

The evidence obtained from the previous evaluations has shown that mobile technology, most notably the use of laptop PCs in tandem with wireless connectivity, provide CPS caseworkers with increased capacity to enter documentation and access information from the state central database while out in the field, as well as assist caseworkers during investigation tasks. The multiple assessments also found evidence of improved but modest productivity gains including increased timeliness of documentation and case closings with the use of laptops.

One fundamental question in this extended assessment is whether or not its findings concur with or are different from the previous findings. The following categories present themes that emerged in this study with a description of any differences from the previous assessment.

The laptops provided caseworkers with opportunities to work outside the office environment in new ways. To understand what mobility meant for caseworkers in CPS, we examined how caseworkers were using laptops, where use occurred, shifts in work opportunities, and changes in communication patterns. Our goal was to gauge to what degree the laptops have become integrated within the daily work practices of CPS caseworkers.

How the laptops were used did not seem to change significantly over time. As in previous assessments, the full range of CPS-related work was completed using the laptops. The laptops were used for case investigations and interventions, documentation and reporting activities, as well as court-related activities. Case documentation was the most frequently mentioned use in both periods, including inputting and updating notes. Other work included court-related documents, safety assessments, reading and reviewing case histories, opening new cases, doing person searches, checking client histories, email, and accessing the Welfare Management System (WMS). In the second data collection period, one caseworker mentioned she no longer uses the laptop to look up driving directions because she received a GPS system.

Accessing and entering information while out of the office was an important feature of connected laptops. This enhanced capability was reported positively by participants in both data collection periods. For example, caseworkers reported enhanced information access and retrieval capability as well as increased data entry capability. However, it appears that accessing case information from the field is not a daily need for most caseworkers. Caseworkers did express that the laptop was very helpful in times ‘when they need it.’ The nature of casework in child protection work means that emergencies may not happen everyday, but when they do arise, the laptops are very beneficial. For example, caseworkers reported using the laptop to access information on the Sex Offender Registry and to make decisions as to whether or not a child could be placed at the suggested residence. This type of scenario may happen about once a month.

There are two main benefits of mobility: 1) increased flexibility regarding where and when CPS work is done, and 2) increased access to information while out of the office. Together, these two benefits potentially allow CPS caseworkers to shift when and where they work. In fact, in both data collection periods respondents in the three LDSS reported using the laptop during normal work hours, after work, on-call, and when working overtime. In the first data collection period, caseworkers alluded to patterns of use and shifts in work opportunities. However, in the extended assessment, the patterns and changes that emerged were clearer:

Non-traditional field locations. Caseworkers reported experimenting with non-traditional work locations away from the office. For example, in both data collection periods, caseworkers reported using their laptops in libraries, parks, hospitals, schools, and commercial coffee shops. In all three LDSS, during both data collection periods, respondents stated that they do not and will not bring laptops into clients’ homes. Some caseworkers said it was a formal policy, and others suggested it was an informal policy but suggested by supervisors or management. The reasons mentioned included that it interfered with relationship building or it was seen as a barrier between the caseworker and the client.

Shifts in when work is done. Laptops were originally conceived as enabling opportunities to do work in situations where caseworkers were previously unable to access the state central database. These situations included waiting times in court and in between field visits (i.e., client visits, schools, hospitals, etc.). However, the opportunities to do work during waiting times in court proved less attainable than anticipated. Many obstacles to working in court were identified, including poor connectivity, lack of confidential work areas, and overcrowding. Several caseworkers reported using the laptop while sitting in their cars, although it was noted that cold weather and location are important factors concerning the extent to which the laptop is used in a car.

Caseworkers, especially in a rural or geographically large LDSS, suggested travel time to and from the office was significantly reduced as a result of using the laptop. Many participants reported that they experimented with staying in the field instead of returning to the office between appointments, saving time and travel expenses. However, some caseworkers reported that they still return to the office, because they feel pressure to ‘be seen’ by supervisors.

On-call practices. On-call caseworkers usually remain at home while they wait for new cases or issues to happen during their shift. In both data collection periods, on-call caseworkers reported increased productivity and satisfaction using the laptops while on-call. Almost all reported saving time by not having to travel to the office to examine case records while on-call, as well as increased opportunities to complete documentation.

Using the laptop at home after work hours. During both data collection periods, using the laptop at home, mostly after work hours, was reported most frequently. Many caseworkers reported fewer interruptions at home than when working in the office and reported more satisfaction in their ability to get work done. Others reported they used the laptop at home to catch-up on casework, thus affording them ‘peace of mind.’ However, not all caseworkers used the laptops at home. Many cited personal reasons such as infringement on family time while others stated they stopped taking the laptop home after they were told by management not to use it while at home.

The laptop creates new communication channels. Prior to laptops being introduced, most caseworkers relied on cell phones while in the field (either personal cell phones or district issued cell phones). Laptops, however, enable the use of email while out of the office. Our findings from the first data collection period revealed that laptops modestly changed the way caseworkers communicated with supervisors or other caseworkers. Caseworkers reported calling supervisors about cases more frequently than emailing supervisors about new or existing cases while in the field. Some caseworkers did use email pretty regularly, just not for discussing cases with supervisors. In both data collection periods, caseworkers reported initiating cases while out in the field by accessing CONNECTIONS; still others reported that it was still necessary to call a supervisor to initiate an assigned case while not in the office.

Changes in communication channels were most apparent for on-call caseworkers. Prior practice before laptops involved on-call caseworkers receiving cases from the State Central Registry (SCR) by phone. For example, caseworkers would have to talk with a SCR caseworker by telephone and then write down the details of the case as it was read over the phone by the SCR caseworker. Overwhelmingly, in both data collection periods, caseworkers reported that this practice had largely changed, and on-call workers were now able to receive cases from the SCR through their laptops. Caseworkers reported time savings and satisfaction that this time consuming practice of listening to and writing down case information by SCR caseworkers had changed.

Any social, procedural, or management change in work environments may take a period of adjustment before employees are really able to change their habits and routines to match new working conditions. Therefore, it was expected that integrating laptops into CPS casework may take some time. In the second data collection period, we asked participants how long it took for the laptops to become a normal part of their daily routines. A significant number of respondents suggested the transformation was almost immediate, citing that laptops are very similar to their existing office workstations. Other respondents suggested it took them on average up to three months to feel comfortable with the technology, citing not being computer savvy and the difficulty of changing habits. However, several mentioned that once one became familiar with the technology, there was an additional learning curve with respect to using the laptops in the field: knowing when, where, and at what times one may need the laptop and how to incorporate the laptop into one’s daily work.

Mobility also implies integrating laptop use into normal work routines. For example, caseworkers reported using laptops while in the field. However, the majority of caseworkers carried the laptop in the field only when they knew they were going to use it. In areas where cars are used for field visits, laptops were often kept in the car. Other caseworkers reported that they leave their laptops at home, however those with docking stations stated that the need to bring the laptop back and forth from home to the office is frustrating. A very small number of caseworkers reported carrying the laptops with them at all times.

The findings presented in this section are based on the analysis of data extracted from CONNECTIONS. The data examined were separated into a pre-test period and a test period (see Appendix A for more information). In order to support comparisons of productivity that reflect as much as possible the effect of mobile technology, the pre-test and the test performance periods were conducted with as much similarity as possible. Therefore, the productivity data was collected for the same caseworkers, doing the same kinds of work as in the test period, and for the same number of days. There was, however, some caseworker turnover between the first data collection period and the second, as well as for both the pre-test and test periods.

During the first data collection period, an assessment of productivity was completed and subsequently, showed modest gains. Additionally, caseworkers had relatively high levels of satisfaction with the use of laptops in CPS work – generally between 65 and 80 percent of respondents were satisfied and 80 percent would recommend the use of laptops to colleagues. Those who reported less satisfaction with laptop use in the first data collection period, tended to do so because of connectivity issues and inconsistencies in some management and policy issues (i.e., working from home and compensation questions).

This assessment focuses on productivity improvements in the timeliness of documentation, including case closings, safety assessments, and progress notes:

• Timeliness of case closing: CPS workers are mandated to complete the investigation of a case within 60 days from its opening. Our measure of improvement in timeliness of case closing was therefore the number of cases closed within 60 days during the pre-test period compared to the test period.


• Timeliness of safety assessments: These assessments are to be completed (i.e., approved by a supervisor) within seven days of the opening of an investigation. Our measure of improvement in timeliness of safety assessments was the number of assessments completed within seven days in the pre-test period compared to the test period.


• Timeliness of progress notes: These notes are to be entered into the system as soon as possible following the event or activity to be documented. Timeliness would therefore be reflected in how many days elapse between a particular event date and the date the progress note conveying that event was entered. We examined the proportion of progress notes entered each day following the related event. This yielded a productivity improvement measure based on the proportion of notes entered closer to the event date.

Productivity could be affected by possible variation in the volume of open cases between the pre-test period and the test period, a factor which would be outside of the control of either the workers or the evaluators. Case volume is defined as the total number of cases available to be worked on during the pre-test period and the test period. Fortunately, there was very little change in overall case volume in two LDSS from pre-test to test periods. Onondaga experienced a 5% increase in the test period (from 2,674 cases pre-test to 2, 819 test) and Wayne experienced an 8% increase (975 cases pre-test to 1,060 test). However, in New York City, there was approximately a 28% increase (2,090 cases pre-test to 2,671 test).

The data extracted from CONNECTIONS during the extended assessment shows that each LDSS increased its rate of case closings in the first 60 days. New York City ACS experienced the largest increase in cases closed within the first 60 days and a reduction in cases closed after 60 days. Wayne and Onondaga experienced an increase in case closings in the first 60 days, but also reported increases in case closings after the first 60 days. However, since this pattern resulted with a simultaneous improvement in case closings within the first 60 days, these results can indicate improvements in both volume and timeliness of work. Another way to interpret this is by suggesting the increase in case closings after 60 days represents backlog reduction. Many caseworkers from both LDSS stated that they often used the laptop after hours at home, and on weekends to ‘catch-up’ on cases. New York City, in contrast, did not present evidence of backlog reduction. It appears, based on an increase in case closings for the first 60 days and a decrease of case closings after 60 days, that NYC ACS increased both timeliness and volume.

Improving the timeliness of safety assessments is another area where mobile technology may support improved performance. Therefore, the assessment includes an examination of the timeliness of safety assessments during the pre-test period and the test period. A safety assessment is considered timely if completed (i.e., approved by a supervisor) within seven days of opening of a case. In all three LDSS, the volume of safety assessments submitted within seven days increased. During the test period, safety assessments submitted past seven days increased for Wayne and Onondaga. This suggests that timeliness of safety assessment submissions closely follows patterns in case closings and that the same ‘catching up’ effect seen in the case closings is impacting safety assessment submissions. The catching up effect may be directing limited attention and resources toward case closings, instead of safety assessments.

Progress notes represent the narrative updates about case work, completed tasks, and communications throughout the course of an investigation. Progress notes, as a matter of good practice, are encouraged to be entered into the CONNECTIONS system as contemporaneously as possible (i.e., following the actual event date). Therefore, we looked at the lapsed time between the related event and the progress note entry into CONNECTIONS. Onondaga and New York City entered approximately 45-50 percent of all progress notes on the same day as the event and about 70 percent by the third day. Both LDSS entered approximately 80 percent of progress by the fifth day after an event. These patterns were almost identical (i.e., no shifts occurred) for the pre-test period and the test period. Wayne County participants entered approximately 26 percent of progress notes on the same day as the event and about 48-52 percent by the third day. About 55-59 percent of all progress notes were entered by the fifth day. Again, these patterns were almost identical for pre-test period and the test period. All three LDSS showed no overall increases in the proportion or progress notes entered within the first five days, nor any shifts in the timeliness of progress notes submitted across those five days. However, if entering progress notes by the third day is considered contemporaneous, then New York City and Onondaga only have approximately 30 percent of all notes where improvement can be made. Wayne County would have approximately 50 percent of all notes where improvement could be made.

We looked at various measures of satisfaction in relation to CPS work and job-related stress to assess how using the laptop impacts employee morale. The first assessment study revealed that caseworkers were satisfied with using laptops in CPS work and would likely recommend using a laptop to colleagues. However, those who were not satisfied generally reported connectivity issues, such as the lack of connectivity or connectivity problems (i.e., slow speeds or inconsistent signals) as reasons.

In the extended assessment, caseworkers and supervisors again reported their overwhelming satisfaction with having a laptop. However, there was more frustration reported with policy and management messages regarding laptop use.

In both periods, the major benefits reported from using the laptop included increased flexibility in when and where work was done, value for on-call and emergency situations, and better use of time while on-call. For example, caseworkers reported that when the laptop is really needed, it is available and beneficial. However, between the initial assessment and the extended assessment, the caseworkers in the extended assessment were less likely to report that the laptop had increased their flexibility of when and where work gets done. Most, as noted, used the laptop at home, versus using it while in the field or in court. Another example demonstrated the value of having laptops in emergency or on-call situations. Emergency situations may not happen on a daily basis, however caseworkers repeatedly reported the important value of a laptop and access to information when needed. In addition, when on-call, the laptop provides access to information at the worker’s home. This saves time traveling back and forth to the office. Another example of laptop convenience in extreme situations are child removals where a caseworker can do clearances (i.e., background checks) from the field to ascertain whether a child can be placed with referred individuals.

Child protective services in New York State is locally administered within Local Departments of Social Services (LDSS), usually in each county’s Department of Social Services (DSS) and in New York City. In a federated system, such as CPS work in New York State, policies and practices are developed and implemented by LDSS. This structure, common of intergovernmental programs, typically creates a diverse administrative environment across the state. While some of the variations in conditions are natural and unavoidable characteristics of locally administered programs, it also means that management has substantial leverage over the mix of strategies and adaptations to normal working policies that can positively or negatively impact how caseworkers ultimately use and take advantage of mobile technologies.

Understanding the policy, management, and organizational variation across NYS is important, as the overall productivity gain resulting from a large-scale deployment of mobile technology will need to consider the variability of conditions that exists across LDSS.

Policies and management practices were reported as issues affecting the following situations: a) mobility or non-traditional field work locations, and b) overtime and compensation while working at home. In both assessments, inconsistent policy and management practices were reported. Caseworkers noted they felt the full capability of the laptop was not being realized due to management, policy, and technical barriers.

Non-traditional field work locations: Mobility implies being able to do work while away from the office. As noted above, working at home was the most reported location for use of the laptop. In addition, caseworkers reported using the laptop while out in the field – including parks, libraries, while taking public transportation (i.e., ferry or train), while in cafes, while in parking lots, or while outside of a client’s home. Caseworkers reported that working in these non-traditional field work locations was difficult for several reasons – connectivity, their comfort level, and work place policies.

In the extended assessment, accounts pointing at the inconstancy of policies regarding how managers and supervisors treated mobility surfaced again. For example, some caseworkers reported that they were told by supervisors not to work in these locations – even though they had done so in the early months of laptop deployment, while other caseworkers reported that their supervisors encouraged working in various locations.

Overtime, compensation, and working from home: Caseworkers reported using the laptop most frequently while at home – generally after work hours. In the initial assessment, caseworkers reported using the laptops while at home in order to ‘catch-up’ on documentation. Each of the three LDSS initially stated that they would provide some type of compensation for time spent working from home after regular work hours with the mobile device. However, in the extended assessment, inconsistencies regarding overtime, compensation, and working from home policies were reported. Some respondents reported that they were provided compensation, others stated they were told they would not receive compensation. One LDSS provided up to four hours of compensatory time each week for using the laptop after work hours to catch up on documentation.