The NYC Administration of Children Services (ACS) initiative to test mobile technologies in child protective services was initiated in response to Mayor Bloomberg's "Safeguarding our Children 2006 Action Plan" which included "deploy handheld computers or tablet PCs to field office workers." In response to this, ACS's Division of Child Protection (DCP) worked in conjunction with ACS Management Information Services (MIS) to implement, test, and evaluate portable wireless technologies for child protection service (CPS) caseworkers. The overall goal of the initiative was to provide remote access to CONNECTIONS and other ACS applications in order to determine if it allowed caseworkers to more effectively accomplish their work activities.

In total, twelve caseworkers (two from each of the five boroughs and two from the Office of Confidential Investigation) participated in the district’s initiative. Laptops or Blackberries were assigned to each of the 12 caseworkers for two-week periods. The entire pilot lasted 12 weeks (six two-week cycles) starting in May 2006 and ending on August 9, 2006.

NYC/ACS selected devices based on weight, size, battery life, and functionality, taking into account access limitations to CONNECTIONS. Four of the six technologies had wireless access to CONNECTIONS through a virtual private network that was channeled through the NYC/ACS system. One device (Blackberry) had access to email only and one device (tablet PC) did not have access to email or the CONNECTIONS database. The following were the technologies tested in the pilot:

Technologies with wireless access to CONNECTIONS:

• Lenovo/IBM X-40 ultra-light notebook
• Lenovo/IBM X-41 ultra-light tablet
• Panasonic Toughbook W4 ultra-light notebook
• Sony PCG-4F1L ultra-light widescreen notebook

Technologies without access to CONNECTIONS:

• Blackberry
• Fujitsu P1510 ultra-light widescreen Tablet PC. (This device lacks WWAN capability and was unable to access CONNECTIONS in the field. It was used as a demonstration device for its touch capable screen and unique small form-factor design).

To assess the impact of the pilot, NYC/ACS created and administered bi-weekly surveys and held bi- weekly meetings to gather input from all 12 field testers from May 2006 to August 2006. CTG created and administered a post-pilot survey and facilitated an information gathering session with eight of the 12 testers on August 9, 2006. Finally, several NYC/ACS caseworkers and program and IT staff attended the Final Assessment Workshop in Albany, New York on October 5, 2006.

The NYC/ACS pilot was nearing its end when CTG became a part of the initiative and much of the data was already collected. CTG did not take part in the pilot design, the selection of field testers, or the device rotation schedule.

A laptop that is directly connected to CONNECTIONS allows CPS caseworkers to perform almost all work activities as if they were sitting in their office. There is no need to change the way work is done, only where and when. What does matter, however, are the different sizes, weights, and characteristics of the devices that will affect choice and use. This section reports on how participants rated the characteristics of the technologies tested in the pilot, specifically size, weight, readability, durability, battery life, portability, and quality of wireless connection. All characteristics play a role in the overall perceived ratings although two characteristics emerged as leading indicators: wireless connectivity and the physical characteristics of the device.

Wireless Connection

A laptop or tablet’s usefulness is directly related to quality and reliability of wireless connectivity. With such a connection, a worker with a laptop can connect to CONNECTIONS and complete a broad range of work activities in more places and at flexible times. Without such a wireless connection or a synchronization process, the value of the laptop severely decreases.

Physical Characteristics of the Device

The size and weight of a laptop will determine if people use it. A large heavy device will not be used as much as a small and lightweight one. This is because of several factors, most importantly, the CPS worker’s comfort in the field and perception of safety.

Other important characteristics are readability of the screen, quality of the transmission, battery life, and the ease of logon. Readability of the screen can be affected by sunlight and the quality of the connection depends on location. CPS workers do not always have the ability to charge their computers or have access to outlets while in the field, therefore battery life is important. Finally, the multiple logons for authentication, to access the ACS server, then the central database, caused frustration and extended the time needed to boot-up and close down.

As shown in Table 2, the Panasonic rated highest for size and weight, with the Sony Vaio not far behind in both categories. The Lenovo X41 and the SONY Vaio rated significantly above average in readability and the Lenovo X41 rated best in quality of wireless connection. The Panasonic received the best battery life rating with the Sony Vaio somewhat lower. The Sony Vaio was significantly above average for portability and durability, and the Panasonic was rated as extremely durable. The Blackberry’s size received a low rating, with readability rated average and battery life rated below average.

Table 2 - Average Participants Ratings of Characteristics of the Mobile Technologies: ACS

Technical Characteristics	Blackberry (n)	Fujitsu (n)	Lenovo X40 (n)	Lenovo X41 (n)	Panasonic (n)	Sony Vaio (n)
size	2.29 (7)	2.50 (6)	3.86 (7)	4.50 (6)	2.83 (6)	2.67 (6)
weight	4.86 (7)	4.20 (6)	3.43 (7)	3.17 (6)	5.17 (6)	4.83 (6)
readability	3.67 (6)	3.83 (6)	4.86 (7)	5.00 (6)	4.83 (6)	5.00 (5)
quality of wireless connection	4.80 (5)	1.50 (6)	3.86 (7)	4.83 (6)	4.50 (6)	3.40 (5)
portability	---	---	3.86 (7)	3.50 (6)	4.67 (6)	5.00 (5)
battery life	3.20 (5)	3.80 (5)	4.00 (7)	4.17 (6)	4.83 (6)	4.60 (5)
durability	---	---	5.00 (7)	4.83 (6)	5.33 (6)	4.80 (5)

Notes: Size was rated on a 6 Point Scale (1 = “Too small” and 6 = “Too large”). Weight was rated on a 6 Point Scale (1 = “Too Heavy” and 6 = “Just right”). Readability, Quality of Wireless connection, Portability and Battery Life were rated on a 6 Point Scale (1 = “Poor” and 6 = “Excellent”).Durability was rated on a 6 Point Scale (1 = “Not at all Durable” and 6 = “Durable”).

Types of Activities

The results in Table 3 show the most frequent work activities performed with each of the technologies. Almost two-thirds of the caseworkers used the technologies for progress notes and safety assessments (65 percent). In addition, over half (55 percent) of the caseworkers added to their to-do lists and conducted searches for people, addresses, and cases, and half of caseworkers did investigation conclusions. However, many of the caseworkers (52 percent) did not use any of the technologies for email. Of those who did, 38 percent used the laptop and 10 percent used the Blackberry (email was the only function available on the Blackberry). The Fujitsu was reported as not being used for any of the most frequent work activities.

Participants liked devices with access to CONNECTIONS because of the ability to do reporting work in the field. One participant said, “It was like having my office in my car. I got so much done in between home visits.” Another caseworker described being with a family that needed other services. With mobile access she was able to search for the needed resources and provide the family contact information for additional help. Without the laptop, this task would have taken a day or more to find and deliver that information.

Table 3 - Percentage of Workers Using Mobile Technologies for Specific Activities: ACS

	Percentage of Caseworkers Who…
Work Activities	Used Blackberry and Fujitsu for work tasks (No link to CONNECTIONS)	Used the laptop for work tasks (No link to CONNECTIONS)	Did not use any mobile technology for work tasks	No.
progress notes	0%	65%	35%	40
safety assessment	0%	65%	35%	40
to do list	0%	55%	45%	40
searches: person, address, case, resource and staff	0%	53%	47%	40
investigation conclusion	0%	50%	50%	40
email	10% (Blackberry)	38%	52%	40
risk assessment profile	0%	45%	55%	40
event list	0%	33%	67%	40
review intake information	0%	25%	75%	40
adding/relating person	0%	20%	80%	40

Additional comments about how the devices were used:

• “My supervisor called me and gave me a new case. When I went to the address, they were not there because they were using that address as the school address. With the laptop, I was able to do a search on previous history and found another address. So I went to the new address that night instead of waiting until Monday. It took me about 20 minutes to boot up, and do the search (which is not long!). Before, when I got a new case, I would have had to go back to the office to find out the information about the case (unless my supervisor gave me some over the phone). Now it’s much quicker and efficient.”
• “One evening, I downloaded legal forms to my laptop, filled them out and then emailed them to my own email account. When I got to the office the next day, they were completed and sitting in my email. I printed them and was ready to go.”
• “Now I don’t get lost everyday! I use Mapquest and Hopstop to get directions to clients’homes. It saves a lot of time each day.”

Work Location

Before the devices were deployed, caseworkers predicted the places they would like to use the devices. These included in the car, in court, on the train, at home, on the bus, in the park, at a school or community center, in a client’s home, on a ferry or subway, and in the office. After using the devices, three places emerged as the top locations for actual use: at home, in court, and in a car.

Many caseworkers reported new-found flexibility in working at more convenient places and times. Those who worked at home expressed appreciation for the flexibility of not having to stay at or return to the office. One said, “I did a visit on a Friday evening and without a laptop I would have had to go to the office that night to write up the notes or it would have had to wait until Monday morning. With the laptop, I wrote up my notes that night and it went directly into CONNECTIONS. It made me feel so much better knowing that they were in the system and I could do it from home.”

In order to make time more productive, many caseworkers used the laptop while waiting in court. Previously, caseworkers would use this time to make phone calls and write notes. But with a laptop they could complete CONNECTIONS work activities.

Some caseworkers brought the laptop with them into the field everyday and used it in the car and in court. Others used it only in court because of the big blocks of time available there. Some workers chose not to use the laptops because of environmental constraints within the locations. One worker who rode public transportation found that it was too crowded for laptop use and often did not have sufficient blocks of time to work. In addition, uninterrupted wireless access was difficult to achieve primarily due to the density of the city’s landscape. This may be true for most forms of public transportation.

Caseworkers reported that using the laptop allowed them to be more efficient but did not add to the overall quality of their work. More specifically, in reference to progress notes, caseworkers stated that the quality of the note is the same if they enter it into a laptop (at a remote location) or at their pc in the office. It’s the ability to work at various locations and times that makes the largest impact on their work.

As shown in Table 4, caseworkers strongly agree that the use of the laptops provided time savings and agree that it helped in overall efficiency. The Sony Vaio received the highest score in overall time savings and the Panasonic rated highest in overall efficiency, though the ratings are very similar. One caseworker talked about the time it saved in driving back and forth to the office to get new cases. Case histories are read while in the field, now taking 15 minutes rather than the hour to get to the office and back out again.

Table 4 - Average Participant Ratings of Devices for Efficiency and Overall Quality: ACS

Efficiency Impacts	Blackberry (n)	Fujitsu (n)	Lenovo X40 (n)	Lenovo X41 (n)	Panasonic (n)	Sony Vaio (n)
time savings	---	---	5.43 (7)	5.50 (6)	5.33 (6)	5.60 (5)
overall quality of work	---	---	3.43 (7)	4.00 (6)	3.67 (6)	3.00 (5)
overall efficiency	---	---	4.86 (7)	5.00 (6)	5.17 (6)	4.80 (5)

Notes: Time Savings was rated on a 6 Point Scale (1 = “It added time” and 6 = “It saved time”). Overall quality was rated on a 6 Point Scale (1 = “Not at all enhanced” and 6 = “Very enhanced”). Overall efficiency was rated on a 6 Point Scale (1 = “Much Less Efficient” and 6 = “Much More Efficient”).

One caseworker said he was able catch up on backlogged progress notes, but having the device will not necessarily mean that he will not get backlogged again. NYC/ACS relaxed overtime policies for the caseworkers during the same period as the pilot test and instructed them to do as much as they could. The caseworkers used the laptops to catch up on old cases, but caseloads did not decrease because new ones were assigned. Some said due to the nature of their work, cases will get backed up again. But all went on to say that using the laptop does allow flexibility and would recommend its use.

Some caseworkers did not use the laptop regularly on field visits. They used it when they knew that blocks of time would be available. One caseworker used it during her commute on the train because it was a large block of time that she could devote to work. While others took the laptop with them all day, almost all of them did not bring the device into the client’s home. This was avoided for many reasons including interfering with establishing a rapport with the family and personal safety. Some felt that opening the laptop and typing in front of the family may be inappropriate and took their attention away from the environment. Workers who traveled by car stated that using it after a visit expedited their ability to get notes written and entered into CONNECTIONS.

Those who regularly took the laptop with them reported collateral benefits. Having access to the central OCFS application allowed them to catch up on email, to research and document collateral contacts, read case histories, document progress notes, and essentially stay in touch with their supervisor. But not all impacts were positive. Some of the caseworkers felt that while they were able to complete a backlog of progress notes, working more at home disturbed their balance between work and home life. The availability of the laptop allowed, even encouraged them to work much longer hours than they might have done otherwise. Another caseworker reported a supervisor encouraging them to work in off-hours because of the mobile device. In both cases, the availability of the laptop resulted in an increase in pressure on the caseworker to work during off-hours.

The flexibility in work time and location also involves personnel policies. One caseworker said, “ If I stayed at the office from 6 pm – 9pm to get my case documentation completed, I would get overtime. If I go home, feed my kids, get them to bed, and then work on the documentation from 9 pm to midnight, I don’t get overtime. I know right now we can get it, because of the crisis mode. But what will happen in the long term? If the policies do not change, there is no incentive to use the laptop.”

Finally, some felt that carrying the laptop with them posed a physical security risk, making them a potential target for theft or violence. If the device could not be easily concealed, then they did not bring it with them all day. Caseworkers identified high risk areas such as some clients’ homes, subways, or parks.

None of the caseworkers approached using the laptop in the same way but all were satisfied in the end and would recommend its use in child protective services. As shown in Table 5, the overall satisfaction was highest with the Panasonic. Also, the Lenovo X41 and the Panasonic tied as the technology that would be most recommended to co-workers. The lowest recommendation and satisfaction rating was with the Lenovo X40, and it was still above average. Thus, all technologies (with mobile access) received above average to significantly above average ratings in satisfaction and recommendation.

These rating show that caseworkers would generally encourage the continued use of laptops in CPS work. However two factors shape those ratings:

• Having mobile access or seamless data entry
• A small and lightweight device with long battery life

Table 5 - Average Participant Overall Satisfaction & Recommendations for Laptops: ACS

Overall Evaluation	Lenovo X40 (n)	Lenovo X41 (n)	Panasonic (n)	Sony Vaio (n)
overall satisfaction	4.29 (7)	4.83 (6)	5.00 (6)	4.60 (5)
recommendation of mobile technology	4.57 (7)	5.33 (6)	5.33 (6)	5.20 (5)

Notes: Overall Satisfaction was rated on a 6 Point Scale (1 = “Not at all Satisfied” and 7 = “Very Satisfied”). Recommendation of Mobile Technology was rated on a 6 Point Scale (1 = “Not at all Recommend” and 7 = “Strongly Recommend”).

The initiative in NYC/ACS tested only twelve users of laptops who connected to CONNECTIONS through ACS’s network. A large scale deployment of laptops may require an alternative set up.

Connectivity. A large scale deployment of laptops will require connectivity solutions that fit agency and statewide policies. This may include alternatives such as a synchronization process for when continuous connectivity is not possible.

Authentication. The logon and authentication procedures established for the pilot interfered with efficient access to the central database. The time needed to logon and the possible loss of wireless connection can inhibit effective use of the laptops.

Infrastructure. Currently the hardware, software, and connectivity infrastructure is designed for desktops. A mobile workforce may require enhanced network infrastructure, servers, or other hardware and software devices.

Hardware security. Data that can remain on portable devices poses new security risks. Encryption of all remotely stored data is essential even though data will be directly stored in CONNECTIONS as well. Use of laptops in the home environment may increase the risk of unauthorized access, damage, or theft.

Data Security. Provision for secure and reliable backup for all remotely stored data is essential. This may require user restrictions or protocols that control storing any sensitive data on laptops or other portable devices.

Technical Support. Methods for supporting mobile technologies are quite different than those for in- office technologies. It is more difficult to oversee and manage deployed equipment for such tasks as maintaining current anti-virus and operating system versions, as well as asset tracking and utilization verification and/or validation. This problem is compounded when there are multiple types of mobile technologies in use in the field.

Workforce support. The equipment may be in use in the field 24/7, requiring expanded hours and types of help and technical support, procedures when technical support is not available, and possible decentralization of some technical support functions or resources.

The Westchester County Department of Social Services tested two technologies at the same time: Santrax, a third-party telephonic dictation service (the system), and cellular telephones. The pilot project spanned thirteen weeks, July 1 to September 30, 2006. Approximately thirty-four child protective service employees and preventive services/foster care caseworkers participated in the pilot from two areas within the district: White Plains and Mt. Vernon. Additional employees from Peekskill began using the technology while the pilot was in progress, however the length of time for each of these participants varied. Twenty-eight responded to the baseline survey and 16 responded to the post- pilot survey, and about 14 were involved in the information gathering sessions and the Final Assessment Workshop, organized by CTG.

The primary technology tested was a telephonic dictation system, Santrax. A service that provides a way for the user to use a cellular or landline telephone to enter data and dictate narrative progress notes. Caseworkers using Santrax call into a dedicated number where they are asked to enter some demographic information about the case using number prompts, and then are instructed to dictate their narrative into the system. The voice recordings are then transcribed by persons that are contracted by Sandata. Within twenty-four hours, the caseworker can access the narrative text in digital form through a secure Santrax Web site. Caseworkers can then cut and paste the text from the Santrax Web site into CONNECTIONS. The secondary technology, the cellular phone, was key in enabling users to access the Santrax system in the field or otherwise out of the office. The full use of both technologies requires CPS employees to connect to a computer (either at their home, office, or by laptop) in order to complete the entry of their progress notes in CONNECTIONS. CTG evaluated both the telephonic dictation system and the cellular telephone separately, though they are complementary technologies.

A variety of technical and use characteristics are associated with each technology and impact individuals’ acceptance and use of the technology to do their work. Through surveys and workshops we gathered users’ ratings and opinions about cell phones including: size, weight, portability, battery life, readability, durability, and quality of cellular connection. We also asked about the specific characteristics of the system including: connectivity to the system, readability and accuracy of transcribed notes, and reliability of turn-a-round (i.e., the ability of the dictation service to produce digitized progress notes in 24 hours).

Participants told us that the most important characteristics overall were:

• Reliability of cellular connection
• Quick turn-around of digitized notes

Table 6 below provides a summary of the average participant’s ratings for cell phone technical characteristics. The size of the cell phone was appropriate for most individuals, as was the weight, portability, and durability of the device. The battery life and quality of wireless connection was rated closer to poor (a mean of 2.93 and 3.36 respectively on a 7-point scale). Many participants commented on how frequently the cellular service dropped calls or they encountered “no service” messages throughout their territories. The poor quality of the wireless connection was often cited as frustrating and bothersome when using the cell phone. One participant said, “I was in the middle of a removal and the call dropped on me, I had to go outside of the house and try to use it there. I was in a situation where I needed the assistance of a coworker, and I had no service.”

Table 6 - Average participant ratings of technical characteristics: Westchester

Technical Characteristics	Cell Phone (n)
size	4.57 (14)
weight	3.67 (15)
portability	3.64 (14)
battery life	2.93 (14)
readability	3.29 (14)
quality of wireless connection	3.36 (14)
durability	3.93 (14)

Notes: Size was rated on a 7-point scale (1 = “Too large” and 7 = “Too small”). Weight was rated on a 7-point scale (1 = “Too heavy” and 7 = “Just right”). Portability, Battery Life, Readability, and Quality of wireless connection were rated on a 7-point scale (1 = “Poor” and 7 = “Excellent”). Durability was rated on a 7-point scale (1 = “Not at all durable” and 7 = “Durable”).

Quick turn around of digitized notes is extremely important to child protective service work. One participant emphasized the importance of a reliable dictation system because of time pressures in completing work stating, “We have deadlines, where we have to do certain things in a certain amount of time, which includes dealing with paperwork.” Early during the pilot period the Local District experienced uneven turn around of digitized notes within 24 hours. One participant said she dictated seven case notes on a Friday, but on the following Monday the notes still were not digitized (though the voice recording was there). She had to spend that morning typing the same progress notes directly into CONNECTIONS. Initially, these service glitches impacted some caseworkers’ ability to meet deadlines, while the effect also dampened enthusiasm for using the new technology. The Local District immediately took action and worked with the third party service to ensure delivery of digitized notes in 24 hours. The vendor implemented an email notification mechanism that informed the implementation team when recorded notes approached the 24-hour limit without being transcribed. Caseworkers expressed that by the end of the test period digitized notes were coming back reliably within the time period.

The system’s transcription accuracy was high. The Local District took the imitative to sample dictated voice recordings and compare them to the transcribed notes. The transcription error rate was less than three percent. In addition, many participants acknowledged that overall, the transcription was accurate. Approximately eight participants used the Santrax system to dictate progress notes in Spanish and had them transcribed and digitized in English. Some commented that this worked well for them, while others had more difficulty with reliable transcriptions because of their accents.

Caseworkers’ ability to read the digitized notes was very important. In the beginning, readability of the digitized notes from the dictation service took some getting used to. The dictation service transcribers had difficulty with punctuation, paragraph divisions, and initially refused to transcribe obscenities included in notes by caseworkers to provide an accurate account of the client’s statements. Initially, caseworkers found that the service provided notes in one continuous narrative and the worker had to break into the appropriate sections. This added additional time to editing progress notes before cutting and pasting them into CONNECTIONS. The Local District worked with the vendor throughout the pilot and the system was refined to include a separation in the digitized notes which highlighted multiple dictation sessions for a given caseworker on a that day and to ensure the obscenities were included.

Different use characteristics are associated with the two technologies. Table 7 below summarizes the average participant ratings for usability in the post-pilot survey. The telephonic dictation service and the cell phone were rated as relatively easy to use and participants were relatively comfortable with the technology after they used it for some time (ratings averaging near 5 on a 7-point scale). Overall, many commented that accessing the Santrax system by phone and the secure Web site was easy and straight forward.

Table 7 - Average participant ratings of mobile technologies use characteristics: Westchester

Interaction Characteristics	Santrax (n)	Cell Phone (n)
ease of use	4.94 (16)	5.27 (15)
comfort with technology	4.75 (16)	5.38 (13)
quicker data entry mode	3.75 (16)	2.40 (10)
encountered technical problems	5.00 (13)	4.43 (14)

Notes: Ease of Use, and Comfort with Technology were rated on a 7-point scale (1 = “Very difficult” and 7 = “Very easy”). Quicker Data Entry, and Technical Problems were rated on a 7-point scale (1 = “Strongly Disagree” and 7 = “Strongly Agree”).

Some participants were less likely to believe that using the mobile technology was a more efficient way to enter data into the system (ratings near the middle of the 7-point scale). The speed of this process depends on workers’ abilities and preferences. In addition, the system is not one continuous process, it includes intermediary steps of retrieving digitized notes, editing, and cutting and pasting.

Participants described in detail the process to get progress notes into CONNECTIONS. After 24 hours, the workers would access the dictated notes on the secure Santrax Web site. They would then edit or add to the notes, then cut and paste them into CONNECTIONS. Some workers avoided the system and continued to type their notes into CONNECTIONS directly because they were fast typists. Others did not mind the process and were able to talk fast, and cut and paste easily.

Participants reported they encountered some technical difficulties with the technologies in the early stages of the pilot. For example, Santrax initially was set up with a three-second pause rule, i.e., that the system disconnects if there is a pause in the dictation of greater than three-seconds. This was a surprise to most caseworkers and also caused some initial frustration. Some participants reported that they could not tell that the call was disconnected, and so continued dictation without recording. Partway through the pilot, the Local District worked with the service provider to extend the pause period to five-seconds. This allowed caseworkers additional time to collect their thoughts. This change was looked at favorably by participants. Some calls were also disconnected because of the low volume of the speaker’s voice and was interpreted as a pause. The need for a higher voice volume limited the number of places that caseworkers could use the service, due to privacy concerns.

These difficulties caused frustration for some participants. For instance, some dictated narratives multiple times because of the dictation service or the cell service dropping their calls. Many suggested it was important to have some type of back-up method for gathering your thoughts (i.e., on a piece of paper) in case there was a dropped call. Some participants reported using landlines to call the service provider to alleviate the problems.

Others were frustrated with the perceived process inefficiencies such as the template for capturing demographic data. It was considered bothersome by some and they either got used to it or developed a work around. Overall, the Local District addressed many of these concerns by working with the dictation service to extend the pause time, develop a prompt that enabled the caseworker to end a call, and requested that the voice component be available within minutes of dictation.

Participants expressed the importance of technical support. Throughout the testing period, the implementation team provided contact information for the service’s technical support, consultation on problems during weekly meetings, and were available by phone or email to resolve user needs. Still, some caseworkers mentioned they did not know who to turn to in order to get the issues resolved. Some went to supervisors, while others decided not to use the system.

The participants also expressed a desire to have the system allow them to dictate notes on multiple families without hanging up and calling back. The current system allowed dictation for one case per call. Caseworkers disliked this constraint because they often have multiple visits in one day and may not dictate their notes until the end of the day. Calling back into the system four or five times is seen as tedious and time consuming. The Local District has asked the service to devise a solution.

How well a technology fits with various locations and modes of transportation is key to its effectiveness. The mode of transportation affects whether people can successfully use mobile devices. To increase the mobility of the dictation service, using a cell phone is optimal. But the service can also be used at the office or at participants’ personal residences through a landline telephone. Factors such as data privacy or confidentiality are important with the system because participants are dictating sensitive case notes. Dictation will not work in all public spaces. Therefore we asked participants where and how they used each of the technologies. Their answers, based on survey and workshop data, appear below.

The system was used primarily for progress notes. Some were able to dictate supporting materials, such as arrest reports, court reports, statements from clients, and medical reports into the system for easy cutting and pasting into CONNECTIONS at a later time. The cell phone, while used in conjunction with Santrax, was also used for other necessary CPS work activities. The cell phone was described as a good tool to use in the field for staying in touch with supervisors, co-workers, and clients, as well as providing a sense of security when visiting homes, and for making collateral contacts while out in the field. Several expressed that they felt connected. Managing contacts was an important benefit of having a cell phone. One cell phone user was described by peers as a “whiz kid.” When asked what the whiz kid did with the cell phone, she remarked that it was used to program clients’ phone numbers into the cell phone for easy access, schedule alarms for appointments, and to listen to messages about appointments (voice mail). Not all participants were able to use the cell phone in this way, admitting that they did not know how.

The most frequent location for using the system and cell phone was in the field in a county-issued vehicle. Cell phones were used in any location where a signal was available, including the client’s home, court, or in their cars while moving from one visit to another. Hands-free accessories which included headsets were provided for privacy while dictating notes. While the Local District did not suggest that caseworkers use the hands-free accessories or headsets while driving, some caseworkers found this to be an effective way to dictate notes from one visit to the next. Some stated that they did not feel comfortable using the cell phone with hands-free accessories while driving, expressing that it was bulky and they had difficulty driving while dictating notes. Therefore, they would pull over to the side of the road, in a parking lot, or down the street from the client’s home. Others reported that they did not have a cell phone or that the locations where they traveled had poor cell phone connection, so they used the system at home or in the office.

The participants unanimously selected the courthouse as the number one location where they hoped to use the mobile technology. They also reported that dictating in the courthouse was difficult because there was no space where they could call into the system in private.

The system is available at any time, anywhere, but the time available to the worker to use the system is limited. The reported time periods they logged on to the system varied from 5 to 25 minutes. There are times when the travel times between visits are lengthy and others much shorter. The non-routine flow of each day impacts the times when they can dictate notes.

Overall, the advantages to using the telephonic dictation system and cell phones were:

• completing progress notes right after a visit
• having the flexibility to use it anywhere at anytime
• keeping in constant contact with supervisors or co-workers creating the opportunity to work outside the office
• dictating in different languages (i.e., Spanish for translation to English)

Overall, the disadvantages to using the telephonic dictation system and cell phones were:

• loss of work, or interruption of work due to failed connectivity or transcription failure slow learning curve that caused some backlog and job stress
• lack of skill and training in dictation for some people
• useful for narrative parts of reports, but not for data entry not connected directly with CONNECTIONS
• editing and copying/pasting is time consuming

Adjusting to a New Technology

The introduction of new technologies or ways of working often is accompanied by an adjustment period or learning curve. Participants expressed that initially, it was time consuming to learn to dictate, master the specialize commands associated with the system, and learn how to retrieve and work with the digitized notes. At first it was a disruption in the way they did work. Several participants concluded it was not natural for them to dictate. By the end of the testing period, some had mastered dictation and others still rejected this way of working. One participant said, “At the beginning it takes time to memorize the commands, but now it is easy.” While another person stated they “there is a learning curve and job change that goes along with it.”

Documentation and Reporting – Progress Notes

Progress notes are important casework documentation, so we examined how the technology use interacted with casework duties, such as improving recall of details, entering notes during down time, or changing work routines. The ratings shown in Table 16 provide a summary of participants’ views about timeframes for preparing and entering progress notes.

The ratings are mixed. All of the characteristics’ average ratings were slightly below the midpoint of the scale. Participants were evenly split over whether they were able to prepare progress notes during down time (43 percent disagreed; and 43 percent agreed) and when asked if they usually entered progress notes all in one sitting (37 percent disagreed; and 37 percent agreed).

Table 8 - Average participant ratings for progress notes timeframes: Westchester

Progress Notes Characteristics	Mean Prior to Pilot (n)	Mean During Pilot (n)
was able to prepare my progress notes during down time	3.81 (27)	3.71 (14)
usually entered progress notes all in one sitting	3.96 (27)	3.75 (16)
usually entered progress notes during regular working hours	3.85 (27)	3.75 (16)

Notes: Progress Notes Characteristics were measured using a 7-Point Scale (1 = “Strongly Disagree” and 7 = “Strongly Agree”).

The dictation capability may have improved the completeness of notes. Many stated that they were able to recall more details when dictating notes right after a visit. One participant said, “My field time was more productive, it’s good to be able to record, while issues are very fresh in my mind.” This advantage did not apply unless caseworkers had the chance to enter notes immediately after a visit. Others stated that the service did not help them because they were already efficient at doing progress notes or that they just did not feel comfortable with or liked using the technology.

Participants reported that the quality of progress notes were not impacted by using the technology, other than being able to recall more details if used after a visit. They said that what constitutes quality in notes varies among supervisors. Some like long, very detailed notes, while others prefer concise notes. Therefore, it was difficult to assess whether the quality of notes were improved by using the technology. In addition, the type of case and type of note determines how much detail is needed; progress notes can vary from as few as 30 to over 2000 words.

Time Savings

Participants’ ratings of the efficiency of these technologies are found in Table 9 below. The results are mixed. For example, the average participant ratings across all three questions is just above the middle of the 7-point scale. More than half (56 percent) reported that they perceived time savings with the device while 44 percent disagreed. During the workshops, some participants described the system as a terrific addition to their tool kits, while others felt that it did not work for them at all. One participant said, “[The] ability to have someone else do the typing saved me very valuable time to do other things that I had to do before and had limited time to do it.” While still another participant expressed, “I have always been up-to-date [with progress notes], I did not need additional tools to complete my job.”

Table 9 -Average Participant Ratings of Devices for Efficiency: Westchester

Efficiency Impacts	Santrax (n)	Cell Phone (n)
the device saved me time	4.06 (16)	4.00 (12)
was a more efficient way to work	3.67 (15)	4.29 (14)
the device allowed me to accomplish other tasks	3.88 (16)	3.83 (12)

Notes: Time Savings, and Accomplishment of other tasks were rated on a 7-point scale (1 = “Strongly Disagree” and 7 = “Strongly Agree”). More efficient way to work was rated on a 7-point scale (1 = “Much Less Efficient” and 7 = “Much More Efficient”).

The mixed pattern is present for cell phones also. While more participants believed that the cell phone saved them time (50 percent), another 42 percent did not. In addition to time savings, many participants commented on the increases in communication that occurred. One participant said, “I like having clients being able to reach me when I am in the field, it saves a lot of time and makes planning much easier.”

Ratings of using the system or cell phones as a more efficient way to work were mixed. While the average participant ratings for telephonic dictation were near the midpoint of the scale (3.67 on 7-point scale) and the average participant ratings for cell phones was higher (4.29 on a 7-point scale), about 47 percent reported that telephonic dictation was an efficient way to work, while 57 percent reported that the cell phone was a more efficient way to work.

There was a split response in participants’ rating that the device allowed them to accomplish other tasks. Of those that stated that it allowed them to do other work, some remarked that they were able to make phone calls or spend more time with client families, or close cases, but this impact is moderate.

Stressors

Overall, normal work routines provide caseworkers with job stress (means above the mid point on a 7- point scale). The introduction of technology added some additional job stress. Many participants reported that when the system goes down or does not work properly, it causes them a lot of job stress. The problems caseworkers faced with progress notes not being digitized in 24-hours or missing were very frustrating because it put caseworkers behind schedule, they had to repeat work, and they were not sure who they had to call to fix the problem. Since management and the public law requires certain tasks to be performed by certain deadlines, technical problems that were out of the caseworkers’ control caused additional job stress. The Local District attended to many of these technical issues. The data collection period did not allow CTG to determine the long term effects of job stress.

Table 10 - Average participant ratings of work-related stress during the pilot: Westchester

General Impacts on Work	Mean Value (n) During the Pilot
normally I was under a lot of work-related stress	4.69 (16)
open cases caused me a lot of stress	4.38 (16)

Notes: Work-related stress, and Stress due to open cases were rated on a 7-point scale (1 = “Strongly Disagree” and 7 = “Strongly Agree”).

Changing Working Habits and Location

Many participants suggested that the use of mobile technologies changed the location in which they could work (i.e., out in the field) and the time their work got done. About two-fifths never work from home, but approximately one out of five participants work from home a few times a week. More than ten percent work from home daily.

The pattern changed for participants working after normal business hours. Almost half of participants worked after normal business hours at least a few times a week. One caseworker said, “We are paid to do a job in 7.5 hours. If you are behind, then it is your fault, if it spills over into your personal time, that is your problem.”

Caseworkers did express concern that the current district policies and administration practices were not currently set up to deal with caseworkers working from home. But they were optimistic that using telephonic dictation could help reduce the amount of time they work after normal business hours.

We asked participants to rate the extent to which they were satisfied with the mobile technologies for doing their work. Overall, satisfaction with both the dictation system and the cell phone were slightly below the midpoint of the seven point scale (Table 11). Some participants were very satisfied, while others were not at all satisfied. Participants reported that they would recommend the device to do child protective (means above 5 on a 7-point scale) and the same is true for their overall opinion of the device (means above 4 and 5 on a 7-point scale). One case worker said, “the technology is helpful if it works well.” This quote expresses what we have heard throughout the evaluation, and that is the need for reliable cellular service, coupled with reliable dictation and 24-hour return of notes to win the full support and acceptance of the technologies from caseworkers.

Table 11 - Average participant ratings in overall satisfaction and recommendation of devices: Westchester

Overall Evaluation	Santrax (n)	Cell Phone (n)
overall satisfaction with device	3.93 (14)	3.77 (13)
would recommend device to be used to do child protective work	5.21 (14)	5.46 (13)
overall opinion of device for your work	4.30 (10)	5.45 (11)

Notes: Overall Satisfaction was rated on a 7-point Scale (1 = “Not at all Satisfied” and 7 = “Very Satisfied”). Recommendation of Mobile Technology was rated on a 7-point Scale (1 = “Not at all Recommend” and 7 = “Strongly Recommend”). Overall opinion was rated on a 7-point Scale (1 = “Low” and 7 = “High”).

In their overall opinion, approximately eight out of ten participants would continue to use the technologies in the field. In addition, all of the participants who answered the question would recommend the devices to their coworkers for use in the field.

Deployment

A district wide or multiple district deployment of the telephonic dictation system and cell phones needs to consider several factors such as 1) the reliability of cellular infrastructure across regions and the state, 2) third-party service quality, 3) improved process and product, 4) the need for substantial training, and 5) change management.

The current statewide cellular infrastructure is well established and several quality service providers are available. Even with relatively well established infrastructures, there are still areas without reliable cellular connections. Since the mobility benefits of using telephonic dictation are optimized on cellular phone service, assessing the reliability and networks of different carriers and their service guarantees is very important. In addition, back up plans and procedures should be evaluated and communicated to employees in case of any major cellular outage.

A system dependent on a single service provider is at risk. If that vendor fails, alternatives are needed. Planning should include assessment of the scalability of the service to hundreds, if not thousands of caseworkers and thousands of cases. Similarly, reliable user support will be important to overall success. Quality and service guarantees would need to be established for working with the third-party vendors to ensure they can deliver reliable products.

Many participants said that the system and way of working is not a turn-key process. Therefore, adequate training in dictation, the use of the system, and basic computer skills (i.e., cutting and pasting) should be provided and coordinated to ensure that caseworkers have the skills necessary to realize process improvements. One caseworker said it would be desirable to have templates that would allow caseworkers to automatically cut and paste information into CONNECTIONS, but there are no plans to provide this enhancement to the system.

The implementation team mentioned that caseworkers’ willingness to accept the technology was influenced by other coworkers. When one had difficulty, others did not try it or just stopped using it. It is possible to establish work groups that help each other learn the technology.

Security

The Local District experienced some security concerns with using a third party service provider, namely: 1) the extent to which information would be kept confidential by vendor employees transcribing notes, 2) the policies related to disposal of the information, voice recording, and records in general, 3) the security of the transmission of the notes through the Internet and, 4) the security of the server and Web interface used to access the digitized notes. Another issue mentioned, but not of major concern, was the level of privacy available when dictating progress notes in public spaces.

Monroe County tested a voice recognition software, Dragon Naturally Speaking (DNS) and digital pens. The pilot project began in mid-September with the distribution of DNS and is scheduled to end in December 2006. The test included 47 Child Protective Services (CPS) case workers and supervisors in the pilot. Of these, 23 responded to the post-pilot survey, 26 participated in the baseline survey, and 20 participated in the information gathering sessions and the Final Assessment Workshop, organized by CTG.

The two technologies differ in three main aspects: the required skill sets, the physical and operational characteristics of the technology, and mobility. The voice recognition software is a technology that requires the users to dictate their notes. Digital pen users recorded their notes by writing on special- purpose paper. The voice recognition software is an application that is installed on a computer, while digital pens are physically separate devices. The voice recognition software can be a mobile technology if used with some other device, such as a digital recorder or is installed on a portable computer. The digital pens are naturally mobile, but require a PC (desktop or laptop) to extract and interpret the writing stored in the pen. Both the voice recognition software and the digital pens collect analog data and convert it into text in digital format for the user to store and manipulate.

Caseworkers using the voice recognition software dictate their notes into a MS Word document and can edit the content at the same time. After completing this process, users cut and paste the dictated notes into CONNECTIONS. The voice recognition software was capable of allowing the caseworkers to dictate notes directly into CONNECTIONS, but this capability was not intended for direct entry and therefore was not fully functional and did not receive high ratings for efficiency. Caseworkers who used the digital pens wrote their notes on the special-purpose paper, storing a copy of the notes on the internal memory of the digital pen. Upon returning to the office, caseworkers downloaded the stored data onto the desktop PC, which converted the image of each page from the pen to a separate MS Word file. Caseworkers could then copy the notes into a single document, edit the recorded notes, and then paste the notes into CONNECTIONS.

This section reports on how people rated characteristics and interactions with the technologies used during the pilot. Technical characteristics are physical characteristics or technical features. Interaction characteristics reflect how users rated the technology when using and interacting with it.

Participants felt that the data entry methods used by the voice recognition software were slightly problematic and unnatural, and that the technology was not as portable as desired. Overall, participants were satisfied with the digital pen technology. Participants believed that the weight, portability, battery life, and durability of the digital pen were very good. However, ratings on the size and data entry method were slightly below average. While size, weight, battery life, and durability were not applicable to the voice recognition software, the ratings illustrate that participants consider the portability and data entry to be poor.

Table 12 - Average participant ratings of mobile technologies technical characteristics: Monroe

Technical Characteristics	Dragon Naturally Speaking (n)	Digital Pen (n)
size	NA	3.38 (13)
weight	NA	5.15 (13)
portability	2.20 (20)	5.46 (13)
battery life	NA	5.27 (11)
data entry	3.09 (22)	3.25 (12)
durability	NA	5.36 (11)

Notes: Size was rated on a 6-point scale (1 = “Too small” and 6 = “Too large”). Weight was rated on a 6-point scale (1 = “Too Heavy” and 6 = “Just right”). Portability, Battery Life and Data Entry were rated on a 6-point scale (1 = “Poor” and 6 = “Excellent”). Durability was rated on a 6-point scale (1 = “Not at all durable” and 6 = “Durable”).

During the information gathering sessions, participants identified advantages and disadvantages to using the mobile technologies. By using DNS, participants were able to enter information directly into CONNECTIONS without the need to copy and paste. Since the software was not intended for direct entry, technical issues prevented this process from working efficiently. Participants considered the digital pens to be helpful in ending the duplication of basic notes, since they can digitize the notes they are already taking. However, participants identified some disadvantages using the digital pen: they are highly sensitive to handwriting styles and were not able to interpret shorthand notes, which led to caseworkers needing to adjust the way they write. Finally, caseworkers felt that the small size of the digital pen was a liability, making it easy to lose.

The voice recognition software was used for progress notes, e-mail, and case summaries. Caseworkers noted that dictating directly into CONNECTIONS was possible, but quite slow. In contrast, digital pens were used for general note taking in meetings and field contacts. The pens were unobtrusive and could be used with clients. CPS workers could also practice with the pens at home. Pens were used for progress notes, safety assessments, and investigation notes. In general, CPS workers complained about the cumbersome task of cutting and pasting from MS Word into the central database.

Table 13 - Percentage of workers using the mobile technology and mean time in weeks: Monroe

Mobile Technology		Use (n)	Mean Time Spent (n)
The voice recognition software		100% (23)	3.05 (20)
	Version 8	100% (17)
	Version 9	0% (9)
digital pen		48% (23)	1.40 (10)

All caseworkers who participated in the survey used DNS version 8. On average, they used DNS three weeks, with a range from one to six weeks. About half of the participants used digital pens for an average of about one and a half weeks, with a range from one week to three weeks.

Use and Interaction Characteristics

Regarding the interaction characteristics, DNS was not very easy to use and people did not feel very comfortable with it. Participants encountered some problems that were out of their control. They disagreed that it was quicker to enter/input/dictate their progress notes using the voice recognition software. As for the digital pen, the analysis shows that participants found it relatively easy to use and they felt comfortable with it. However, they also encountered important technical problems and thought it was not a more efficient way to enter/input/dictate their progress notes.

Table 14 - Average participant ratings of mobile technology use characteristics: Monroe

Interaction Characteristics	Dragon Naturally Speaking (n)	Digital Pen (n)
ease of use	3.62 (21)	4.00 (12)
comfort with technology	3.36 (22)	4.00 (12)
quicker data entry mode	2.74 (19)	2.60 (10)
encountered technical problems	3.62 (21)	3.91 (11)

Notes: Ease of Use was rated on a 6-point scale (1 = “Very Difficult” and 6 = “Very Easy”). Quicker Data Entry Mode, Comfort with Technology, and Encountered Technical Problems were rated on a 6-point scale (1 = “Strongly Disagree” and 6 = “Strongly Agree”).

Work Location

The voice recognition software is not a mobile technology without a digital recorder or a laptop. Therefore it was used mainly at the office. The digital pen is an unobtrusive device and can be easily used for many types of activities and in many different locations. Caseworkers felt comfortable using it on home visits. During the information gathering sessions, participants also identified one key advantage to using digital pens for CPS work in general: the digital pens are good for writing when waiting in court, which is when a caseworker would normally be documenting their cases.

Types of Activities

Slightly more than three quarters of participants used the voice recognition software for their progress notes and about half used it for Microsoft Office applications. About one-third of them used it for investigation conclusion, and only a few participants used it for the other uses shown in Table 15 (below). About a third of the participants used a digital pen for their progress notes and Microsoft Office applications. A few participants used it for investigating conclusion, e-mail, and safety assessment.

Table 15 - Percentage of Caseworkers using the Mobile Technology: Monroe

Task	Percentage of Caseworkers using the Mobile Technologies (n)
	Dragon Naturally Speaking	digital pen
progress notes	83 % (23)	35 % (23)
adding/changing case worker or planner	4 % (23)	0 % (23)
demographics	4 % (23)	0 % (23)
safety assessment	9 % (23)	4 % (23)
risk assessment profile	9 % (23)	0 % (23)
Investigation conclusion	30 % (23)	9 % (23)
Microsoft office applications	44 % (23)	30 % (23)
e-mail	17 % (23)	9 % (23)
schedule organizer	4 % (23)	0 % (23)

Some caseworkers reported using DNS to complete progress notes faster, being able to dictate faster than type. For faster typists, the DNS system did not have this effect. Increased efficiency, or the lack of it, may be related to the complexity of the DNS system, which in the beginning, requires training. In fact, the time and effort needed to become adept may be high, which produces frustration for some of the caseworkers.

Digital pens also had some impacts on overall work, but caseworkers had high expectations that were not always met. Some expected the pen output to look like a finished product, but it did not. Others wrote more and with more complete sentences, so that the notes would be closer to a finished product and more easily placed in CONNECTIONS. Some users wrote brief notes and incomplete sentences and then modified the text prior to entering it into CONNECTIONS. They also noted the potential to use the pens during opportune moments during field work to develop finished products for uploading.

Impact on progress notes

The participants rated several characteristics of their progress notes before and during the pilot (see Table 16). According to these ratings, the length of the progress notes improved during the pilot. Caseworkers also reported being able to enter their progress notes during regular office hours. In contrast, most of the other progress note characteristics worsened during the pilot. This may be a result of the problems of learning a new technology and the complexity of incorporating it in their daily work.

Table 16 - Characteristics of Progress Notes before and During the Pilot: Monroe

Progress Notes Characteristics	Mean Value (n)
	Prior to the Pilot	During the Pilot
about right length	5.15 (27)	5.24 (21)
completeness	5.30 (27)	4.81 (21)
adequate details	5.70 (27)	5.00 (21)
up-to-date	4.44 (27)	4.43 (21)
able to prepare of progress notes out of the office	3.81 (27)	1.89 (19)
able to enter progress notes into CONNECTIONS in the field	3.84 (25)	1.15 (20)
able to enter progress notes all at one sitting	3.96 (27)	3.86 (21)
able to entered progress notes during regular work hours	3.85 (27)	4.14 (21)

Notes: Progress Notes Characteristics were measured using a 7-point scale (1 = “Strongly Disagree” and 7 = “Strongly Agree”).

Participants reported that neither the voice recognition software nor the digital pens saved them much time. Participants generally disagreed that the voice recognition software or the digital pens aided in accomplishing other tasks. In addition, they concluded that using the technology is only moderately efficient for doing their work.

Table 17 - Average Participant Ratings of Devices for Efficiency: Monroe County

Efficiency Impacts	Dragon Naturally Speaking (n)	Digital Pen (n)
the device saved me time	2.52 (21)	2.25 (12)
was a more efficient way to work	3.53 (19)	3.36 (11)
the device allowed me to accomplish other tasks	2.32 (19)	1.82 (11)

Notes: Time Savings, and Accomplishment of other tasks were rated on a 6-point scale (1 = “Strongly Disagree” and 6 = “Strongly Agree”). More efficient way to work was rated on a 6-point scale (1 = “Much Less Efficient” and 6 = “Much More Efficient”).

Participants mentioned one major disadvantage of using a digital pen: they produce separate files for each page of notes written on the special-purpose paper. They are thus required to cut and paste from individual files into a single MS Word document, and then from the MS Word document into CONNECTIONS. The inability to have multiple windows open at the same time in the central data application makes this problem even worse. Note entering takes extra time as a result of this cumbersome process.

Impacts on job stress

The pilot test experience appears to have had a small impact on stress levels. Prior to the pilot, participants rated their stress levels only slightly higher than during the pilot period. The survey responses also showed that stress due to open cases was rated slightly lower during the pilot period.

Overtime issues

Prior to the pilot only a small percentage of participants worked from home on a daily basis. Only one-third of the respondents indicated they did so a few times a week, while slightly over one-fourth rarely worked from home, and nearly two-fifths stated they never worked from home. There was an overall decrease in the number of participants working from home during the pilot. No participants reported working from home on a daily basis, and only one in ten worked a few times a week. Nearly one-third of respondents rarely worked from home, and than half never worked from home. The number reporting never working from home rose from 37 percent prior to the pilot, to 62 percent during the pilot, though this may be in part a result of fewer respondents to the later survey.

Table 18 - Percentage of caseworkers that work from home or after hours to complete progress notes – Monroe County

General Impacts on Work	Prior to the Pilot					During the Pilot
	N	Daily	A few Times/ Week	Rarely	Never	N	Daily	A few Times/ Week	Rarely	Never
worked from home to complete progress notes	27	4%	33%	26 %	37%	21	0 %	10%	29 %	62%
stayed in the office after normal work hours to complete progress notes	27	26%	41%	15 %	19%	21	0 %	57%	33 %	10%

In general, the technology use seems to have produced a shift: reduced extra work time spent at home and increased after hours work in the office. Prior to the pilot slightly more than one-fourth of the participants worked after hours at the office on a daily basis, less than half a few times a week, about one in ten rarely worked after hours, and about one in twenty never worked after hours. During the pilot, the daily after-hours work dropped to zero, but the occasional rate rose to 57 percent. Only one- tenth of the participants never worked after working hours. This may be a consequence of the desktop installation of DNS and the unavailability of the digital recorders.

Dragon Naturally Speaking

The most common advantage mentioned during the information gathering sessions was related to how well dictation fit with workers’ preferred way of recording information. Those caseworkers who could dictate faster than type preferred the voice recognition system. The workers also reported that dictating helps to relieve users’ wrists and hands, thus limiting the risk of repetitive stress injuries.

Caseworkers also identified disadvantages of DNS. They did not like the inability to multitask while using DNS (i.e., making phone calls while typing). The requirements for training were considered too time consuming for caseworkers’ current work-schedule. They also described DNS as often difficult to use in cubical office environments due to ambient noise and dictation often being audible across many workplaces.

Digital Pens

Participants identified several potential advantages with digital pens. The technology could be used to complete local forms. In fact, the software has a tagging capability that when used with special forms it can put the data directly into a Word document. However, these special forms would need to be ordered from the manufacturer at an additional cost.

Some participants mentioned that the digital pens are useful for taking notes or numbers, but not long narratives such as progress notes. For instance, they were useful for an administrator and an IT person to use for short meeting notes. Therefore, using the technology and the pads could be beneficial if the data captured is routine, rather than narratives. Finally, supply of special-purpose pads is also perceived as a potential problem due to its relatively high cost.

Overall Satisfaction

Table 19 shows that participants were only mildly enthusiastic about these technologies overall. Their inclination to recommend these systems for further use was not strongly positive. However when asked for a simple “yes” or “no” about future use or recommending the technologies, the users were mostly positive, though more for the digital pens (72% yes), than for the DNS (65% yes).

Table 19 - Average participant ratings in overall satisfaction and recommendation of devices: Monroe

Overall Evaluation	Dragon Naturally Speaking (n)	Digital Pen (n)
overall satisfaction with technology	3.19 (21)	3.08 (12)
would recommend device to be used to do child protective work	3.57 (21)	3.75 (12)
overall opinion of technology for your work	3.42 (19)	3.55 (11)

Notes: Overall Satisfaction was rated on a 7-point scale (1 = “Not at all Satisfied” and 7 = “Very Satisfied”). Recommendation of Mobile Technology was rated on a 7-point scale (1 = “Not at all Recommend” and 7 = “Strongly Recommend”). Overall Assessment was rated on a 7-point scale (1 = “Low” and 7 = “High”, with a “No opinion” option).

The deployment issues for DNS are typical of new workplace technologies. There was not adequate training or change management provisions included in the pilot test. The hardware did not appear to consistently meet the DNS system’s minimum requirements.⁴ DNS is a relatively complex tool and dictating is not natural for many workers nor was it a common work skill prior to the pilot project. In addition, OCFS does not currently support DNS, resulting in possible lack of technical support. Although DNS does have the advantage of being a stand alone product, rather than a system, thus requiring no ongoing expense other than periodic upgrades.

Deployment of the digital pens involves similar problems, aside from dictation skills. Further development of input capabilities would depend on support for future forms development. The full potential of digital pens will depend on investments in forms applications for easier input to CONNECTIONS.

Using a digital recorder in conjunction with the voice recognition software can pose some security concerns. The contents on a digital recorder cannot be encrypted, hence the loss or theft of a device containing notes could result in loss of confidential information. However, this problem is similar to the potential loss of paper notes which are no more secure, but in common use. In addition, dictating in public would seldom be possible, so caseworkers would need private places to use the recorders for dictation.

The digital pens present similar security concerns. The main potential issue was that there are no easy ways to secure the pens, and therefore, loss of the device would result in loss of confidential information.