Comparing the short and original versions of the Activities-specific Balance Confidence (ABC) scale in older adults

Balance confidence assessment in older adults has implications for falls and quality of life. It remains unclear whether the original Activities-specific Balance Confidence (ABC-16) scale or the shortened 6-item (ABC-6) scale is recommended. To further inform the decision-making process of balance confidence tool selection, a secondary analysis of an existing dataset consisting of 77 community-dwelling older adults was performed. ABC-16 and ABC-6 association and agreement, internal consistencies, and relationships with self-rated health (SRH) were assessed. Participants were primarily female (80.5%) between the ages of 60 and 87 years. Results indicated a strong association between the scales [r = .97, p<.001); ICC(2,1) = .80] but limited agreement (95% Limits of Agreement range = 22.1; mean difference of 7.2 points in the direction of the ABC-16). Cronbach’s alphas were .95 (ABC-16) and .89 (ABC-6), suggesting high internal consistency for both scales but possible item redundancy with the ABC-16. Regression model 1 (ABC-6 = primary predictor) explained more of the variance (R2=.36) in SRH compared to model 2 (ABC-16 = primary predictor; R2=.29). Hotelling’s t-test [t(74)=2.4, p=.008] indicated that the correlation coefficient (Multiple R) from the ABC-6 model was significantly higher than the correlation coefficient from the ABC-16 model. In conclusion, despite a high correlation, the two scales did not agree strongly and should not be considered interchangeable. Given that the ABC-16 takes longer to administer, does not relate to SRH as strongly, and could have redundant items, the ABC-6 should be considered for balance confidence assessment in older adults.

Reverse-correlation reveals internal error-corrections during information-seeking

In the area of metacognition research, different methods have been used to study participants’ subjective sense of confidence in their choices. Among the most often used methods are explicit reports of subjective confidence, post-decision wagering and measuring additional info-seeking behavior. While all three methods are thought to measure confidence, they differ greatly in terms of practical execution and theoretical foundation. The method of reverse correlation has previously been used to determine which aspects of the stimulus influence decisions and confidence judgments. Here we compare the three methods of confidence assessment using reverse correlation analysis. Explicit reports and post-decision wagering revealed a positive association of stimulus information with choices and reduced decision weights for low-confidence trials. When confidence was assessed using the info-seeking method, low-confidence trials showed an inverted association with primary stimulus information. Using modelling of the behavioral data, we show how the reverse correlation results of all three methods can be explained by a simple model of confidence when internal error-corrections are allowed during seeking of additional information.

953. Online Medical Education Improves Knowledge of Monoclonal Antibody Treatment for COVID-19 Among Physicians

Background Treatments aimed at patients with mild to moderate COVID-19 offer an opportunity to improve rates of hospitalizations and progression to severe disease. The aim of this study was to assess the educational impact of a series of continuing medical education (CME) activities on the knowledge, competence, and confidence of primary care (PCP), infectious disease (ID), and ER/critical care physicians regarding the management of COVID-19 with monoclonal antibody (mAb) therapy. Methods The educational series consisted of 9 online, CME activities in multiple formats. At the individual activity level, educational effect was assessed with a repeated pairs pre-/post-assessment study including a 3-item, multiple choice, knowledge/competence questionnaire and one confidence assessment question, with each participant serving as his/her own control. To assess changes in knowledge, competence, and confidence data from all clinicians who completed both pre- and post-questions were aggregated across activities and stratified by learning themes. McNemar’s test (P< .05) assessed educational effect. Data were collected from 12/20 to 5/21. Results To date, the 9 activities have reached over 24,000 physicians. Selected improvements in knowledge and competence measured as relative % change in correct responses pre/post education across the learning themes are reported here. (i) 45% improvement in PCPs and a 31% improvement in ID specialists’ knowledge/competence in identifying patients who would benefit from mAbs (P < .01). (ii) 83% improvement in PCPs and a 42% improvement in ID specialists’ confidence in identifying patients who would benefit from mAbs (P < .001). (iii) 15% improvement in ID specialists’ knowledge/competence on the clinical data on mAbs for COVID-19 (P < .001). (iv) 32% improvement in PCPs knowledge/competence in understanding the mechanism of action (MOA) of mAbs for COVID-19 (P < .001) Conclusion This series of online, CME-certified educational activities delivered in multiple formats resulted in significant improvements in knowledge and competence regarding the management of patients with mild to moderate COVID-19. This analysis also uncovered remaining educational gaps; 55% of content related to identifying patients who would benefit from mAbs was not retained post-education. Disclosures All Authors: No reported disclosures

959. Online CME Successful at Improving Knowledge, Competence and Confidence on Incorporating mAbs for COVID-19 Among a Global Audience

Background The incorporation of effective treatments is critical to improving patient care for COVID-19. We assessed the educational impact of a series of continuing medical education (CME) activities on knowledge, competence, and confidence changes in US and OUS physicians related to the use of monoclonal antibodies (mAbs) for COVID-19. Methods 10 online, CME-certified activities were delivered in multiple formats. For individual activities, educational effect was assessed with a repeated pairs pre-/post-assessment study including a 1 to 7-item, multiple choice, knowledge/competence questionnaire and one confidence assessment question. To assess changes in knowledge, competence, and confidence, data were aggregated across activities and stratified by learning theme. McNemar’s test or paired samples t-test (P< .05) assessed educational effect. The activities launched between November 2020 and May 2021; data were collected through May 2021. Results To date, the 10 activities have reached over 50,000 clinicians, including 24,627 physicians. Selected improvement/reinforcement in knowledge/competence measured as relative % change in correct responses pre/post education across the learning themes are reported. (i) 89% improvement/reinforcement among US ID specialists in knowledge/competence incorporating mAbs into patient care and 83% improvement among outside the US (OUS) ID specialists (P < .001). (ii) 70% improvement/reinforcement among US PCPs in knowledge/competence incorporating mAbs into patient care and 55% improvement among OUS PCPs (P < .001). (iii) 52% improvement/reinforcement in knowledge/competence among US PCPs regarding clinical data for mAbs and 44% among OUS PCPs (P < .001). (iv) 42% of US ID specialists and 29% of OUS ID specialists had a measurable improvement in confidence in identifying patients who would benefit from mAbs (P < .001). Conclusion This series of online, CME-certified educational activities resulted in significant improvements in knowledge, competence, and confidence regarding the appropriate use of mAbs for SARS-CoV-2 in clinical practice. These results demonstrate the effectiveness of global curriculum-based education for clinicians designed to address specific gaps in care. Disclosures All Authors: No reported disclosures

Implementing Confidence Assessment in Low-Stakes, Formative Mathematics Assessments

Confidence assessment (CA) involves students stating alongside each of their answers a confidence rating (e.g. 0 low to 10 high) to express how certain they are that their answer is correct. Each student’s score is calculated as the sum of the confidence ratings on the items that they answered correctly, minus the sum of the confidence ratings on the items that they answered incorrectly; this scoring system is designed to incentivize students to give truthful confidence ratings. Previous research found that secondary-school mathematics students readily understood the negative-marking feature of a CA instrument used during one lesson, and that they were generally positive about the CA approach. This paper reports on a quasi-experimental trial of CA in four secondary-school mathematics lessons (N = 475 students) across time periods ranging from 3 weeks up to one academic year, compared to business-as-usual controls. A meta-analysis of the effect sizes across the four schools gave an aggregated Cohen’s d of –0.02 [95% CI –0.22, 0.19] and an overall Bayes Factor B01 of 8.48. This indicated substantial evidence for the null hypothesis that there was no difference between the attainment gains of the intervention group and the control group, relative to the alternative hypothesis that the gains were different. I conclude that incorporating confidence assessment into low-stakes classroom mathematics formative assessments does not appear to be detrimental to students’ attainment, and I suggest reasons why a clear positive outcome was not obtained.

Assessment of the nutritional knowledge of undergraduate pharmacy students: The need to advance nutrition education in the training of pharmacy students

Objective: This study aimed to assess pharmacy students’ knowledge of aspects of nutrition relevant to pharmacy practice and as a means of imparting information through feedback to support learning and professional development. Method: 230 pharmacy students completed a 30-item nutritional informal assessment questionnaire, along with confidence assessment questions. Students were asked about the sources of information they used to answer the knowledge assessment questions, their learning experience with regards to nutrition, and their inclination to learn more about nutrition. Results: Generally, Year four pharmacy students had significantly higher scores than Year two students. However, their overall performance was still lower than that expected of graduate pharmacists. About 75.0% of the study participants agreed that they would like to learn more about nutrition. Conclusion: This study indicates the need to integrate better nutritional education into the undergraduate pharmacy curriculum. The results also suggest a need for Malaysian schools of pharmacy to offer focused learning on nutrition aligned with national health priorities and the competency level in relation to nutrition of pharmacy graduates in the country.

Decision Confidence Assessment in Multi-Class Classification

This paper presents a novel approach to the assessment of decision confidence when multi-class recognition is concerned. When many classification problems are considered, while eliminating human interaction with the system might be one goal, it is not the only possible option—lessening the workload of human experts can also bring huge improvement to the production process. The presented approach focuses on providing a tool that will significantly decrease the amount of work that the human expert needs to conduct while evaluating different samples. Instead of hard classification, which assigns a single label to each class, the described solution focuses on evaluating each case in terms of decision confidence—checking how sure the classifier is in the case of the currently processed example, and deciding if the final classification should be performed, or if the sample should instead be manually evaluated by a human expert. The method can be easily adjusted to any number of classes. It can also focus either on the classification accuracy or coverage of the used dataset, depending on user preferences. Different confidence functions are evaluated in that aspect. The results obtained during experiments meet the initial criteria, providing an acceptable quality for the final solution.

Online CME closes gaps in CSCC management revealed in a virtual patient simulation.

e23013 Background: Cutaneous squamous cell carcinoma (CSCC) is the second most common skin cancer type. Although highly curable in its early stages, advanced or metastatic CSCC has been associated with mortality rates as high as 70%. The objective of this study was to assess the educational impact of a series of continuing medical education (CME) activities on the knowledge, competence, and confidence of oncologists on the assessment and management of CSCC. Methods: The educational series consisted of 4 online, CME-certified activities in multiple delivery formats, including an initial virtual patient simulation, followed by a clinical practice assessment (CPA) and 2 video discussions with synchronized slides. For the video discussions, educational effect was assessed with a repeated pairs pre-/post-assessment study including a 3-item, multiple choice, knowledge/competence questionnaire and one confidence assessment question, with each participant serving as his/her own control. To assess changes in knowledge, competence, and confidence data from all clinicians who completed both pre- and post-questions were aggregated across activities and stratified by learning theme. McNemar’s test or paired samples t-test ( P <.05) assessed educational effect. The first activity launched June 2019 and the last launched August 2020; data were collected until December 2020. Results: Following education oncologists had significant improvements in knowledge and competence compared with responses from the CPA. Improvement in knowledge and competence measured as relative % change in correct responses pre/post education across the learning themes identified in the virtual patient simulation are reported: Identifying patients who are not candidates for surgical intervention: 9% increase (56% CPA/ 77% pre/ 84% post; n=123-314; P <.05). Tailoring treatment: 30% increase (41% CPA/ 56% pre/ 73% post; n=119-314; P <.05). Tailoring treatment–cemiplimab: 22% increase (32% CPA/ 55% pre/ 67% post; n=123-314; P <.05). Tailoring treatment–cetuximab: 16% increase (15% CPA/ 37% pre/ 43% post; n=123-314; P <.05). New and emerging immunotherapies for CSCC: 47% increase (28% CPA/ 30% pre/ 44% post; n=119-314; P <.05). Conclusions: These results demonstrate the effectiveness of curriculum-based education for oncologists to address specific gaps in care as identified in a virtual patient simulation. This series of CME-certified online educational activities resulted in significant improvements in knowledge and competence in the assessment and treatment of patients with CSCC.

CME curriculum improves clinician knowledge, competence, and confidence in managing patients with NF1-related tumors.

e23009 Background: Neurofibromatosis type 1 (NF1) is an uncommon complex autosomal dominant disorder caused by germline mutations in the NF1 tumor suppressor gene. Therapeutic options have historically been limited with surgical debulking being the preferred treatment modality. However, recent clinical data have shown improved outcomes with the use of MEK inhibitors. The objective of this study was to assess the educational impact of a series of continuing medical education (CME) activities on the knowledge, competence, and confidence of physician learners with respect to the treatment of NF1-related tumors. Methods: The educational series consisted of four online, CME-certified activities. For each activity, educational effect was assessed with a repeated pairs pre-/post-assessment study including a 3-item, multiple choice, knowledge/competence questionnaire and one confidence assessment question, with each participant serving as his/her own control. To assess changes in knowledge, competence, and confidence data from all clinicians who completed both pre- and post-questions were aggregated across activities, stratified by learning theme, and evaluated by target specialty. A repeated pairs pre-/post-assessment study design was used and McNemar’s test or paired samples t-test (P < .05) assessed educational effect. The first activity in the series launched Dec. 2019 and the last launched June 2020; data were collected until Dec. 2020. Results: Overall significant improvements were seen after education for oncologists (N = 258, P < .001), neurologists (N = 474, P < .001), surgeons (N = 427, P < .001), neurosurgeons (N = 93, P < .05), and pediatricians (N = 56, P < .001), regardless of practice setting. A sub-analysis of community specialists also showed significant improvements after education: oncologists (N = 117, P < .01), neurologists (N = 188, P < .001), and surgeons (N = 206, P < .001). Outcomes assessed by learning themes and specialty showed relative percent increases, with the majority being statistically significant. Conclusions: This analysis demonstrates that oncologists’ and other specialists’ knowledge, competence, and confidence regarding the treatment of NF1-related tumors improved after education, as measured through a comprehensive analysis of CME outcomes data from a variety of activities designed to achieve these ends. These results have the ability to translate to improvements in clinical care. Despite the improvements, additional educational activities are needed to address residual gaps and further increase clinicians’ ability in this clinical setting.

Efficacy of a fully remote primary lecture series at an academic hematology and oncology fellowship program.

11031 Background: 615 fellows began training in American combined hematology/oncology fellowship programs in July 2020. These new fellows face a steep learning curve. The coronavirus pandemic has significantly affected how we learn, with programs having to convert most collective learning to a completely virtual format. Research on the efficacy of introductory lecture series in academic hematology/oncology programs is limited especially regarding virtual formats. We introduced a virtual introductory lecture series with the goal of increasing the clinical confidence and knowledge base of first-year fellows. Methods: A once weekly remotely-delivered two-hour primer series was designed with lectures given by both third-year fellows and faculty from July-August 2020. Fellows were asked to complete pre & post-test evaluations of each lecture. Evaluations included a combination of knowledge-based questions & self-reported confidence assessment. Results: 14 fellows were assigned pre- and post-tests in the study. 1 fellow was excluded due to lack of participation. A total of 123 paired pre and post-tests were compared. Data analysis was performed with SPSS v 24.0 using the paired samples t-test. Pre and post-tests were graded on a scale of 0-100. The pre to post mean difference compares the mean test result of the post tests to that of the corresponding pretests. Questions were divided into 2 groups. The 1st group tested the fellow’s medical knowledge regarding the pathology while the 2nd group tested the comfort in the management, diagnosis and treatment. In the statistical analysis, these questions were defined as “Knowledge” and “Comfort” accordingly, the sum as “Complete”. A statistically significant improvement in post-test knowledge for fellows of all years was noted with a pre to post test mean difference of 12.52, P <.0001. The difference was more pronounced among 1st year fellows with a pre to post test mean difference of 16.84, P <.0001. A similar improvement was seen for the comfort in management questions. The post-test comfort pre to post test mean difference was 10.48, P <.0001 for fellows of all years and 6.70, P <.0001 for first year fellows. Conclusions: A remotely-delivered introductory lecture series for fellows in a hematology/oncology training program increases both clinical knowledge and clinical confidence in fellows of all years of training.[Table: see text]