A Review of:
Keyes, K., & Dworak, E. (2017). Staffing chat reference with undergraduate student assistants at an academic library: A standards-based assessment. Journal of Academic Librarianship, 43(6), 469–478. https://doi.org/10.1016/j.acalib.2017.09.001
Abstract
Objective – To determine whether undergraduate students can provide quality chat reference service.
Design – Content analysis of undergraduate student, professional librarian, and paraprofessional staff responses in chat reference transcripts.
Setting – Academic library.
Subjects – 451 chat reference transcripts.
Methods – Chat reference transcripts from May 2014–September 2016 were collected. Five categories of answerer were coded: librarian in the reference department (LibR), librarian from another department (LibNR), staff without a Master of Library Science (staff), staff with a Master of Library Science (+staff), and student employee (student). A random sample of 15% of each category of answerer was selected for analysis. The answerer categories were collapsed to librarians, staff, and students for the results section.
Four criteria were used to code chat reference transcripts: difficulty of query, answerer behaviour, problems with transcript answer, and comments from coders. Coding for difficulty was based on the READ scale (Reference Effort Assessment Data). Answerer behaviour was based on The RUSA Guidelines (Reference and User Services Association). Behaviours assessed included: clarity, courtesy, grammar, greeting, instruction, referral, searching, sign off, sources, and whether patrons were asked if their question was answered. All coding was done independently between the two researchers, with very good interrater reliability. Data for variables with disagreement were removed from the analysis. The chi-square test was used to analyze the association between variables. Analysis also included patrons’ ratings and comments about their chat experience. Content and tone were assessed for each patron comment.
Main Results – Answerer behaviours showed a significant difference between groups for 3 of the 10 behaviours assessed: courtesy (p=0.031), grammar (p=0.001), and sources (0.041). The difference between groups for courtesy was: staff (88%), librarians (76%), and students (73%). Grammar was correct in most transcripts, but there was a significant difference between the answerer groups: librarians (98%), staff (90%), and students (73%). There was a significant difference between groups that offered sources: librarians (63.8%), staff (62.5%), and students (43.8%).
There was no significant difference between the answerer groups for the other seven behaviours. Overall, 31% of transcripts showed that answerers asked if a patron’s query was answered or if they needed further help. The analysis showed that 79% of transcripts were coded as clear or free of jargon. Greetings were found in 65% of transcripts. Instruction was indicated in 59% of transcripts. Referrals were offered in 27% of all transcripts. Of the transcripts where searching was deemed necessary, 82% showed evidence of searching. A sign off was present in 56% of all transcripts. Transcripts with noted problems were deemed so because of lack of effort, being incomplete or incorrect, having no reference interview, or the answerer should have asked for help. There was no significant difference between answerer groups with respect to problem questions.
Of the 24% of patrons who rated their chat experience, 90% rated it as good or great, and no significant difference was found between answerer groups. Question difficulty was coded 50% at level 0-2 (easier), 39% at level 3 (medium difficulty), and 11% at level 4-5 (more difficult).
Conclusion – Undergraduate students are capable of providing chat reference that is similar in quality to that of librarians and staff. However, increased training is needed for students in the areas of referrals, providing sources, and signing off. Students do better than librarians and staff with greetings and are more courteous than librarians. There is room for improvement for staff and librarians offering chat services. Tiered chat reference service using undergraduates is a viable option.
Local Users, Consortial Providers: Seeking Points of Dissatisfaction with a Collaborative Virtual Reference Service