Core Outcome Set for Reporting Outcomes of Interventions for Velopharyngeal Dysfunction: Final Results of the COS-VPD Initiative

Objective To date, the recording of outcomes of interventions for velopharyngeal dysfunction (VPD) has not been standardized. This makes a comparison of results between studies challenging. The aim of this study was to develop a core outcome set (COS) for reporting outcomes in studies examining the management of VPD. Design A two-round Delphi consensus process was used to develop the COS. Patients, Participants The expert Delphi panel comprised patients and caregivers of patients with VPD, surgeons and speech and language therapists specializing in cleft palate, and researchers with expertise in VPD. Interventions A long list of outcomes was derived from the published literature. In each round of a Delphi survey, participants were asked to score outcomes using the Grading of Recommendations, Assessment, Development, and Evaluations scale of 1 to 9, with 1 to 3 labeled “not important,” 4 to 6 labeled “important but not critical,” and 7 to 9 labeled “critical.” Main outcome measure Consensus criteria were specified a priori. Outcomes with a rating of 75% or more of the panel rating 7 to 9 and 25% or fewer rating 1 to 3 were included in the COS. Results A total of 31 core outcomes were identified from the Delphi process. This list was condensed to combine topic areas to produce a final COS of 10 outcomes, including both processes of care and patient-reported outcomes that should be considered for reporting in future studies of VPD. Conclusions Implementation of the COS-VPD will facilitate consistency of outcomes data collection and comparison of results across studies.

A standardized protocol for efficient and reliable quality control of brain registration in functional MRI studies

Automatic alignment of brain anatomy in a standard space is a key step when processing magnetic resonance imaging for group analyses. Such brain registration is prone to failure, and the results are therefore typically reviewed visually to ensure quality. There is however no standard, validated protocol available to perform this visual quality control. We propose here a standardized QC protocol for brain registration, with minimal training overhead and no required knowledge of brain anatomy. We validated the reliability of three-level QC ratings (OK, Maybe, Fail) across different raters. Nine experts each rated N=100 validation images, and reached moderate to good agreement (Kappa from 0.4 to 0.68, average of 0.54±0.08), with the highest agreement for “Fail” images (Dice from 0.67 to 0.93, average of 0.8±0.06). We then recruited volunteers through the Zooniverse crowdsourcing platform, and extracted a consensus panel rating for both the Zooniverse raters (N=41) and the expert raters. The agreement between expert and Zooniverse panels was high (kappa=0.76), demonstrating the feasibility of crowdsourcing QC of brain registration. Our brain registration QC protocol will help standardize QC practices across laboratories, improve the consistency of reporting of QC in publications, and will open the way for QC assessment of large datasets which could be used to train automated QC systems.

In search of the optimal outcome measure for patients with advanced cancer and gastrointestinal obstruction: A qualitative research study.

100 Background: Gastrointestinal obstruction (GIO) is the most common indication for palliative surgical consultation in patients with advanced cancer. The purpose of this study was to delineate the symptom burden and experience of patients with GIO. Methods: Twenty patients with advanced cancer and GIO described symptoms at the time of surgical consultation. We used content analysis of interview transcripts and symptoms were ranked by frequency and compared to a general symptom assessment survey (MD Anderson Symptom Inventory). Results: Malignancy type included colorectal (N = 9), gastric (N = 4), urothelial/renal (N = 3), and other (N = 4), while site of obstruction was small bowel in 11 (55%), gastric outlet in 3 (15%), and large bowel in 6 (30%). Thirteen patients (65%) had received chemotherapy within 6 weeks. Imaging evidence of a primary/recurrent tumor, carcinomatosis, or ascites was documented in 13 (65%), 11 (55%), and 16 (80%) patients, respectively. Thirty symptoms were identified on qualitative interviewing. MD Anderson Symptom Inventory symptoms of pain, nausea, and vomiting were frequently noted. Frequently mentioned GIO-specific symptoms included bloating, cramping, not having a bowel movement, and inability to eat. Conclusions: Qualitative research methodology can identify symptoms of importance to patients which can be used to assess improvement after treatment of GIO. Expert panel rating will be used to develop the final symptom item list prior to psychometric testing of the survey.

The Eurocleft Study: Intercenter Study of Treatment Outcome in Patients with Complete Cleft Lip and Palate. Part 3: Dental Arch Relationships

Objective To compare dental arch relationships up to age 17 in individuals with complete unilateral cleft lip and palate (UCLP) treated at five European centers. Design Longitudinal cohort study, where results were previously reported at 9 years and follow-up measurements were obtained for 12 and 17 years. Setting Multidisciplinary cleft services in Northern Europe. Subjects 127 consecutively treated individuals with repaired UCLP. Main outcome measure Panel rating of dental arch relationship. Results The results revealed that at 17 years of age three of the centers had better ratings in dental arch relationship (means scores: 1.7, 1.9, and 2.2, respectively) than the other two centers (3.3, 3.4) at statistically significant levels (p < .01 to p < .001). Conclusion The results confirm that systematic differences in dental arch relationships may occur between different cleft centers, but do not allow specific causal factors to be identified.

USING DATA TO ENHANCE THE EXPERT PANEL PROCESS

Objective: To enhance the validity of a well-known expert panel process, we used data from patient surveys to identify and correct rating errors.Methods: We used the two-round RAND/UCLA panel method to rate indications of harmful (presence of problems), hazardous (at risk for problems), and nonhazardous (no known risks) drinking in older adults. Results from the panel provided guidelines for classifying older individuals as harmful, hazardous, or nonhazardous drinkers, using a survey. The classifications yielded unexpectedly high numbers of harmful and hazardous drinkers. We hypothesized possible misclassifications of drinking risks and used the survey data to identify indications that may have led to invalid ratings. We modified problematic indications and asked three clinician panelists to evaluate the clinical usefulness of the modifications in a third panel round. We revised the indications based on panelist response and reexamined drinking classifications.Results: Using the original indications, 48% of drinkers in the sample were classified as harmful, 31% as hazardous, and 21% as nonhazardous. A review of the indications revealed framing bias in the original rating task and vague definitions of certain symptoms and conditions. The modified indications resulted in classifications of 22% harmful, 47% hazardous, and 31% nonhazardous drinkers.Conclusions: Analysis of survey data led to identification and correction of specific errors occurring during the panel-rating process. The validity of the RAND/UCLA method can be enhanced using data-driven modifications.