Over the last two years, NYS OCFS, the LDSS, and the state legislature provided funding to deploy and test the use of mobile technologies in Child Protective Services (CPS) work. In April 2007, Onondaga County Department of Social Services submitted a mobile technology proposal and was awarded funding to deploy laptops in CPS work. As a result, on November 19, 2007, 56 Dell Latitude D620 laptops were distributed to 40 caseworkers and 10 managers. Six laptops were also shared on a rotating basis among night service staff.

Following this deployment in November 2007, an initial assessment of the use of laptops in CPS work took place. The initial assessment examined how mobile technology affects CPS caseworker productivity, mobility, and satisfaction. This extended assessment examined similar questions over an eight-month period, longer than the time period in the initial assessment.

At the time of data collection, Onondaga County DSS had 56 CPS staff responsible for child protective services. Located in Central New York, Onondaga County has approximately 458,000 residents. The county has a land area of about 790 square miles and has one major city. Onondaga County DSS submitted a mobile technology proposal to OCFS stating that they wanted to learn if connected laptops would provide caseworkers with more opportunities to stay in the field (i.e., reducing travel time returning to the office), and to provide supervisors with more immediate information on case-related issues. In addition, Onondaga’s proposal also identified high caseloads and high numbers of overdue safety assessments as two important drivers for testing mobile technologies.

The district-provided external broadband cards were distributed and all access to the state network (i.e., to access the central database) was through a virtual private network (VPN) that secured the transmission to and from the laptop and the network. The district also provided a one hour and fifteen minute small group training session which covered: (1) orientation to the initiative, (2) orientation to the equipment, (3) local guidelines, and (4) initialization of individual IDs, and setup of broadband and VPN access. Each participant received a small training packet at the end of the session for future reference.

This profile is specific to Onondaga County and brings together the most comprehensive data on the two data collection periods as well as findings on use, mobility, productivity, and satisfaction.

The overall objective of the laptops was to provide caseworkers with opportunities to work outside the office environment in new ways. This section reports on how participants used those opportunities in terms of 1) the type of work done, 2) locations of use, and 3) factors influencing their use. Additionally, this section reports on the major technical problems reported by the caseworkers. The data used in this analysis was collected soon after the laptops were deployed, as well as after an extended period of use. See Appendix A for a full description of the methods used.

During the first data collection period, survey respondents reported using the laptop during normal work hours, after hours, during commute times, and while working overtime. In addition, the full range of CPS-related work was completed using the laptops. The laptops were used in case investigation and interventions, documentation and reporting, and court-related activities. Case documentation was the most frequent use identified by respondents, including entering and updating notes. Other work reported included reading and reviewing case histories, opening and closing cases, conducting clearances and safety assessments, preparing court petitions, using the Welfare Management System (WMS), and communicating via email. During the second data collection period, respondents reported using the laptops in similar ways as previously reported, and added uses such as looking up addresses and driving directions, and accessing information on the sex offender registry.

In the first data collection period, caseworkers reported the following benefits to laptop use: 1) access to information in the field was very important, and 2) needing to return to the office to access case information less frequently. For example, one caseworker reported the following: “It [the laptop] gives you more flexibility in when you enter your notes and you don't have to call anyone else or go back to the office if you need to look up information you may need in the field.” Similar trends were reported consistently during the second data collection period. One of the caseworkers interviewed indicated that they often use the laptop during hospital visits with clients in order to look up case histories.

While all the caseworkers reported they were comfortable using the laptop to conduct their work, the time it took to incorporate it into their daily routine varied from one caseworker to another.

As part of the first data collection period, caseworkers were surveyed on where they used their laptop, as well as the average length of time they used it. Table 4 below represents findings from the first data collection period.

Table 4 - Location and Hours of Laptop Use per Week

	Use of Laptop (n)	Average length of use per week
Field	24% (10)	0.70 Hours
Court	17% (7)	0.19 Hours
Home	63% (26)	3.07 Hours
Do not use at all	(0)	--

* Based on survey respondents who took the post survey n = 41. Total number of testers n = 69.

As noted from the table above, the majority of caseworkers used the laptops from their home, followed by use in the field. The data gathered also indicates that caseworkers used the laptops from home for approximately three hours a week. While no survey was used for the second data collection period, interviewed caseworkers still reported using the laptops primarily from their homes after normal working hours.

During initial planning for this project, the amount of time caseworkers reported spending in court suggested that the courts could potentially be an unexploited location for mobile work in many LDSS. However, respondents during the first data collection reported spending on average just under two days a month at court and waiting on average 1.5 hours during a court visit. A smaller percentage of respondents reported using the laptop in court. Findings from the second data collection period indicated that caseworkers still have problems using the laptops in courts. Several caseworkers suggested a dedicated area for laptop use is important and would likely increase use in court. Privacy issues and confidentiality were the main reasons reported for not using the laptop in court. Furthermore, a shortage of tables and power outlets created barriers to using the laptops in court.

During the second data collection period, caseworkers continued to report using the laptops from the field. Several caseworkers indicated using their laptops while parked in a secure parking lot and several reported using their laptops while at coffee shops or hospitals. Caseworkers also reported using laptops while on-call. Having the laptops during these periods allowed the caseworkers to access full case information as opposed to having to take extensive notes from the hotline.

Caseworkers reported during the second data collection period that using the laptops within clients’ homes was not encouraged by supervisors. Several mentioned a formal policy was enacted. One caseworker mentioned the need to make clients aware of the caseworker’s use of the laptop while sitting outside the client’s home in their car. For example, one caseworker was unfamiliar with a particular area and used the laptop to lookup an address and driving directions. At that time, the caseworker notified the client of the situation and asked permission to use the laptop in their car outside the client’s residence.

During the first data collection period, the most common noted technical problems were slow connections and loss of wireless connectivity. Additionally, most respondents reported that privacy was less problematic in the field, but some did experience privacy problems. Again, in the second data collection, caseworkers reported problems with the lack of privacy and confidentiality in court.

Participants were surveyed in the first data collection period about the ease of logging-on to the device. Overall, 39% said it was “Easy,” 50% rated it as “Neither difficult nor Easy,” and another 11% of survey respondents rated the log-on process as “Difficult.” Nearly a year after the laptops were deployed, caseworkers reported very few technical problems similar to those identified in the first data collection period. Most commonly, caseworkers continued to experience lengthy system boot up times, lagging wireless connection speeds, or cumbersome log-on processes. In the second data collection period, a few caseworkers suggested the ability to print from the field would be an added benefit that may increase use and increase their ability to fully exploit the potential for mobility and communication.

This analysis uses central database data to examine two core questions about possible technology impacts within the Onondaga County DSS: 1) Are workers with laptops more productive with respect to case closings, safety submissions, and progress note reporting? And 2) Does laptop use have an effect on the timeliness of reporting? Additionally, this section presents the findings based on an analysis of the perceived usefulness of the laptops. See Appendix A for a full description of the methods used.

Case closing is one way to assess any changes in efficiency and productivity. Figure 7 below shows that the volume of timely closing of cases (in 60 days or less) increased during the test period, up from 1,197 in the pre-test period to 1,422 during the test period. The number of cases closed that were over 60 days old increased from 833 to 954 during the test period.

 

Figure 8 below shows that the percentage of timely closing of cases (in 60 days or less) out of the total did not change during the test period. It is important to note that, in Onondaga County, the total number of cases available to be worked increased from 2,674 in the pre-test to 2,819 during the test period—a 5% increase. Since the proportion of timely case closings did not change despite an increase in cases, we can conclude that an increase in productivity occurred, albeit a modest one. The length of the test period during the second data collection was 256 days. However, a positive trend in cases closed was also seen during the initial assessment, which lasted 51 days. ⁴

 

The rate of completion of safety assessments is one way to assess changes in efficiency and productivity. Figure 9 below shows the volume of timely (within seven days or less) submission of safety assessments increased during the test period, up from 859 in the pre-test period to 1,020 during the test period. The number of safety assessments submitted that were over seven days old increased from 1,143 to 1,328 during the test period.

Figure 9 - Number of Onondaga County LSSD Safety Assessments Submitted Pre-Pilot and During Pilot

Figure 10 below shows that the percentage of timely submission of safety assessments (in seven days or less) as a percent of total cases changed very little during the test period. However, again in this county, the total number of cases available to be worked on increased from 2,674 in the pre-test to 2,819 during the test period—a 5% increase. In the second data collection, caseworkers maintained their level of submission (approximately 43%) despite a 5% increase in cases available to be worked on.

Figure 10 - Percentage of Onondaga County LDSS Safety Assessments Submitted Pre-Pilot and During Pilot

 

An indicator of timeliness is elapsed time, which is the number of days between an event and the posting of documentation regarding that event in the central database system.Figure 11 and Figure 12 below show trends in the elapsed time between progress note entry and the related event. Figure 11 shows that the number (or volume) of progress notes entered rose slightly during the test period from 40,876 in the pre-test period to 42,858 during the test period – a 5% increase. Figure 12shows that the rate of progress note entry changed very little during the test period but remained high overall. During both periods close to 80% of all progress notes were entered by the fifth day following the event. In the first data collection period, timeliness (or the rate of entry) slightly decreased during the test period, but was high overall.⁵

 

 

During the first data collection period, participants were surveyed to evaluate whether using a laptop made any difference in their CPS work. The survey included questions on five areas: timeliness of documentation, ability to work from court, ability to access case information, communication with supervisors, and services provided to clients. Survey respondents were asked to rate the difference on a five-point scale where “1” indicated “Much worse,” “3” represented “About the same,” and “5” was indicative of being “Much better.”

Overall, nearly 90% of survey respondents reported improvements in timeliness of documentation and 92% reported increased ability to access case information from the field. There were smaller proportions of respondents reporting improvements in their ability to work in court (25%), communicating with supervisors (23%), and providing service to clients (31%). None reported a negative impact.

Table 5 - Perceived Change Timeliness and Work Impacts – Onondaga County

	Much worse (n)	Somewhat worse (n)	About the same (n)	Somewhat better (n)	Much better (n)
Timeliness of documentation	0% (0)	0% (0)	11% (3)	52% (14)	37% (10)
Ability to do work in court	0% (0)	0% (0)	75% (18)	17% (4)	8% (2)
Ability to access case information	0% (0)	0% (0)	7% (2)	44% (12)	48% (13)
Communication with supervisors	0% (0)	0% (0)	77% (20)	23% (6)	0% (0)
Service to clients	0% (0)	0% (0)	69% (18)	27% (7)	4% (1)

This is somewhat inconsistent with the timeliness of documentation results obtained from the central database. Thus the self-reported (i.e., survey responses) positive impacts may be related more to the increased rate of case closing than the timeliness of progress notes.

Data gathered from the videoconferences in the second data collection period supports the general findings from the first data collection period. The majority of the caseworkers present in the videoconference reported a change in the way they conduct their work. Almost all stated they felt more organized as a result of having the laptop. Many commented that they were more caught up on their progress notes and have either eliminated or reduced their documentation backlog. One respondent noted from the first data collection period, “It [the laptop] allows me to catch up on progress notes and related work while at home, at my own speed, instead of having to be pressured to come into the office. It will also be effective while on night service.” Similar statements were made during the second data collection period. The caseworkers attributed the reduction in backlog to the introduction of the laptops.

Personal preference was a consistent theme during the first data collection period. For example, some caseworkers expressed the preference to use the laptop at home after normal business hours, while others expressed the preference to use the laptop to enter notes immediately following a visit. Caseworkers interviewed during the second data collection period reported no significant change in the way they communicate with their supervisors, many stated how they communicate is based on personal preference. One caseworker stated that prior to having a laptop, working after normal business hours from the office was rare, but the laptop allows the caseworker to continue working from home and also to attend to her family’s needs. Caseworkers indicated continued reliance on the use of cell-phones and in-person meetings to communicate with their supervisors. Supervisors reported using email to communicate with caseworkers while they were out of the office. Several supervisors suggested utilizing a chat-like feature as a way to enhance communication between supervisors and caseworkers.

Nearly one year after the laptops were deployed, there were mixed responses regarding the effect the laptops had on the quality of service caseworkers provide to their clients. While some caseworkers felt that they still perform the same set of tasks using the laptops, others indicated some improvements, but that was dependent on the particular case they were assigned. For example, one caseworker spoke about the use of the laptop to begin a critical action before returning to the office. Another caseworker valued the ability to lookup case histories while they were away from the office without having to constantly call a colleague or a supervisor.

In the first data collection period, survey respondents reported a high overall level of satisfaction. The survey data showed that 81% of respondents reported being “Somewhat satisfied” or “Very satisfied,” compared to 11% being “Somewhat dissatisfied” or “Very dissatisfied.” Additionally, 7% responded that they were “Neither dissatisfied/satisfied.” While no surveys were administered in the second data collection period, caseworkers in the videoconference reported similar and consistent sentiments regarding their overall satisfaction with the laptops.

Despite the overall high levels of satisfaction reported in both data collection periods, caseworkers reported issues during the second data collection period that they hoped would have been addressed by then. For example, in the first data collection period, participants indicated technical difficulties, inconsistent access to CONNECTIONS, lengthy boot up times, and issues related to login passwords as problematic. Nearly one year after deploying and using the laptops, similar observations were reported by caseworkers in the videoconference, where technical issues identical to those previously mentioned were cited. Additionally, frustration regarding the lack of comprehensive or consistent policies on acceptable use and compensation were identified. In the second data collection period, caseworkers indicated that acceptable use of the laptops is primarily dictated by individual supervisor preference. Some supervisors allowed caseworkers to work outside of the office, others preferred them working from the office. Caseworkers who were allowed to work from the field reported high levels of satisfaction. They attributed this satisfaction to their ability to access case information regardless of where they were. Other caseworkers indicated that their stress level would be reduced if they were allowed to work more from the field.

There was no policy change regarding the use of laptops from home for CPS work between the initial and extended assessments. Caseworkers could work from home if they obtained prior approval; noting that they are allowed up to four hours a week of overtime (issued as compensatory time). Other issues such as the lack of dedicated work areas in court, inability to print from the field, and inconsistent Wi-Fi connections were additional barriers cited.