The Quality of Canadian and U.S. Government Health Documents Remains Unchallenged Until Better Research Can Be Undertaken

A review of: Lambert, Frank. “Assessing the Authoritativeness of Canadian and American Health Documents: A Comparative Analysis Using Informetric Methodologies.” Government Information Quarterly 22.2 (2005): 277-96. Objective - To assess by means of citation analysis whether the public trust afforded health documents published by the Canadian and U.S. governments is appropriate, and to ascertain whether differences in the respective health care systems influence how publications are produced. Design – Comparative study. Setting – The Canadian Depository Service Program (DSP) and the U.S. Department of Health and Human Services (DHHS) web sites. Subjects – One hundred sixty-six electronic documents sourced from the DSP website, and 284 electronic documents sourced from the DHHS website. Methods – Subjects were randomly selected from repositories offering the most comprehensive collections. Documents with evidence of references to other works used in preparation were separated from those without such characteristics. Data variables were collected from documents with evidence of references. Statistical analysis of the data was undertaken. Main results – Of the respective samples, 89 (53%) from the DSP and 109 (38.4%) from the DHHS contained references. Personal authors were identified in 46 (51.7%) and 63 (58%) of the respective subsets. Handbooks and guidebooks accounted for the largest portion of the DSP subset (29; 32.6%) and government periodicals were the largest constituent of the DHHS subset (41; 37.6%). Scholarly journals were the most common reference type for both the DSP (44%) and the DHHS (58.5%) subsets. The number of references per document was widely dispersed for both subsets; the DSP mean was approximately 64 (SD=114.68) and the DHHS was 73.71 (SD=168.85). Kruskal-Wallis subset analysis of median number of references by document type found differences generalizable to the entire DSP and DHHS populations. Health Canada Reports, handbooks, and guidebooks contained significantly more references than periodical articles or fact sheets. Certain DHSS documents, classified as “other,” contained more references than periodical articles. Canadian documents were more likely to contain references than U.S. documents. Comparison of documents to determine whether one country employs more rigorous citation practices did not produce statistically significant results. U.S. Federal Government documents are more likely to be referenced in other U.S. government health documents, compared to Canadian publications. The presence of references in documents from either country significantly affected likelihood of being cited by web authors. Conclusion – Significant differences in reference use frequencies between DSP and DHHS documents challenges Foskett’s stance that documents of value contain references (Foskett). Use of peer-reviewed scholarly journals for both DSP and DHHS publications was reassuring, suggesting a fairly rigorous publication standard. Reliance of DHHS publications upon federal government documents remains unclear. Referencing of DSP documents, irrespective of reference usage suggests a level of trust towards Canadian government health publications. Web authors appear more comfortable citing referenced DHHS documents. Further study could involve the examination of reference frequency by journal compared against journal impact factors.