Objective – To determine whether a newly developed interactive, Web-based tutorial on OVID MEDLINE was acceptable to students, and to identify whether the tutorial improved students’ information skills.
Design – Objective and subjective assessment within a small cohort study.
Setting – An evidence based practice module within a Master's in Research (MRes) program at the University of Salford, UK.
Subjects – A total of 13 usable evaluations were received from graduate students who took an evidence based practice module as part of their MRes coursework.
Methods – Information skills (IS) were taught in weeks two and three of a 12-week module on evidence based practice. Each of the two IS sessions lasted approximately three hours. At the beginning of the first session, baseline skills were assessed by asking the students to perform a literature search on either the effectiveness of nursing interventions for smoking cessation, or the effectiveness of rehabilitation after stroke. The OVID MEDLINE tutorial was introduced at the first session, and guided hands-on practice was offered. Homework was given, and between-session use of the tutorial was encouraged. At the end of the second session, students were asked to complete another search in order to assess short-term impact of the tutorial. Both sets of search results were scored using a checklist rubric that looked for Boolean operators, use of MeSH terms, use of limits, number and relevance of references, and other assessment criteria. The rubric was a modified version of a tool published by Rosenberg et al.
The tutorial remained available throughout the 12-week module, at which time a systematic literature review was assigned in order to measure longer-term impact. As an additional subjective measurement, a questionnaire regarding the information skills sessions and tutorial was given at the end of the second IS session (week 3).
Main Results – Thirteen objective assessments (literature search results) were returned and usable. According to the scored pre-training search, two students could use multiple search techniques correctly and in a systematic manner. The post-training search results indicated that six students could systematically search, which is triple the original number. At the end of the 12-week EBP module, that number had increased to seven students. This demonstrated a significant difference between pre-training and post-training scores (P = 0.040), as well as a significant difference between post-training and post-module scores (P = 0.008).
Eight of the subjective questionnaires, which measured perceptions on a five-point scale, were returned. All responses indicated that “the sessions were useful, well structured and interesting” (83). Seven of the eight were entirely positive, either agreeing or strongly agreeing with each of the eleven questions about things such as search skill improvement, information skills knowledge, and confidence in searching. The small sample size made it difficult to generalise these results. Ad hoc comments varied and sometimes contradicted each other, such as one request for simpler tutorial instructions in contrast with the comment that the “tutorial ‘couldn’t be simpler’” (84).
Conclusions – Students rated the IS sessions positively, including the Web-based MEDLINE tutorial. Search skills improved, as was demonstrated by comparing pre-training search results with post-training and end-of-module searches. Continuing feedback indicates that the tutorial is used within other departments and programs as a standalone tutorial.
SciRide Finder: a citation-based paradigm in biomedical literature search