Evaluation framework to guide implementation of AI systems into healthcare settings

ObjectivesTo date, many artificial intelligence (AI) systems have been developed in healthcare, but adoption has been limited. This may be due to inappropriate or incomplete evaluation and a lack of internationally recognised AI standards on evaluation. To have confidence in the generalisability of AI systems in healthcare and to enable their integration into workflows, there is a need for a practical yet comprehensive instrument to assess the translational aspects of the available AI systems. Currently available evaluation frameworks for AI in healthcare focus on the reporting and regulatory aspects but have little guidance regarding assessment of the translational aspects of the AI systems like the functional, utility and ethical components.MethodsTo address this gap and create a framework that assesses real-world systems, an international team has developed a translationally focused evaluation framework termed ‘Translational Evaluation of Healthcare AI (TEHAI)’. A critical review of literature assessed existing evaluation and reporting frameworks and gaps. Next, using health technology evaluation and translational principles, reporting components were identified for consideration. These were independently reviewed for consensus inclusion in a final framework by an international panel of eight expert.ResultsTEHAI includes three main components: capability, utility and adoption. The emphasis on translational and ethical features of the model development and deployment distinguishes TEHAI from other evaluation instruments. In specific, the evaluation components can be applied at any stage of the development and deployment of the AI system.DiscussionOne major limitation of existing reporting or evaluation frameworks is their narrow focus. TEHAI, because of its strong foundation in translation research models and an emphasis on safety, translational value and generalisability, not only has a theoretical basis but also practical application to assessing real-world systems.ConclusionThe translational research theoretic approach used to develop TEHAI should see it having application not just for evaluation of clinical AI in research settings, but more broadly to guide evaluation of working clinical systems.

Cooperación de profesionales de las bibliotecas de Ciencias de la Salud como respuesta a la pandemia de la Covid-19

The Covid-19 pandemic has brought about notable changes in our society, including new forms of relationships and work. It has especially affected the professions related to the health system, which includes health science librarians, who are responsible for providing up-to-date and high-quality information to health workers on the front line of this fight. This article presents the work of a group of professionals working in Spanish health science libraries (#AyudaBiblioteca), organized to collaborate and provide a better response to the information needs resulting from Covid-19. It has 90 participants, mainly from hospital libraries, as well as from research organizations, scientific societies, professional associations, ministries, universities, health technology evaluation agencies, and independent professionals. WhatsApp has been used as a means of communication, and Google Drive as a document repository. Resources are displayed on a public website, and search strategies are deposited in Diigo. Twitter is used with the hashtags #AyudaBiblioteca and #COVID19 as an external means of communication to receive and disseminate information. This collaboration has allowed us to join forces in the search for and coordinate collection of reliable, evidence-based, proven, and high-quality information, demonstrating the benefits of teleworking and online collaboration. The benefits of open science and how it improves research are also reflected upon. This initiative shows the importance of health science librarians in supporting health, academic, and research staff. Resumen La pandemia de la Covid-19 ha supuesto notables cambios en nuestra sociedad, incorporando nuevas formas de relación y de trabajo. Ha afectado especialmente a las profesiones del sistema sanitario, del que forman parte los bibliotecarios de Ciencias de la Salud, encargados de proporcionar información actualizada y de calidad a los sanitarios al frente de esta lucha. En este artículo se presenta el trabajo del colectivo de profesionales de bibliotecas de Ciencias de la Salud españolas #AyudaBiblioteca, organizados para colaborar y dar una mejor respuesta a las necesidades informativas sobre la Covid-19. Cuenta con 90 participantes mayoritariamente de bibliotecas de hospitales, y también de organismos de investigación, sociedades científicas, colegios profesionales, consejerías, universidades, agencias de evaluación de tecnologías sanitarias y profesionales independientes. Como medio de comunicación se ha utilizado WhatsApp y como depósito de documentación Google Drive. Los recursos se visualizan en una web pública, y las estrategias de búsqueda se vuelcan en Diigo. Se utiliza Twitter con los hashtags #AyudaBiblioteca y #COVID19 como medio de comunicación externa para recibir y difundir información. Esta colaboración ha permitido aunar fuerzas en la búsqueda y recopilación coordinada de información fiable, basada en evidencia, contrastada y de calidad, y demuestra los beneficios del teletrabajo y de la colaboración online. También refleja las ventajas de la ciencia abierta y cómo mejora la investigación. Esta iniciativa muestra la importancia de los bibliotecarios de Ciencias de la Salud como apoyo al personal sanitario, académico y de investigación.

Mobile Health Technology Evaluation

Globally, the advancement of mobile technology and the growing number of mobile phone users has promoted the boom in mobile health services. The influence of mobile technology has, in fact, made healthcare delivery more accessible, affordable and effective today. Consumers are thus increasingly using mobile devices as health service delivery aids across various countries. However, questions remain as to how consumer traits like personal innovativeness and self-efficacy, financial costs related to the service delivery and demographics like age and gender may affect the usage and adoption of mobile health services, especially for emerging economies like Bangladesh. Conceptual model of the study identifies self-efficacy, facilitating conditions, effort expectancy and performance expectancy to be significant constructs that influences users' overall perceptions of mobile health services, along with moderating effects of both age and gender upon the selected factors. Finally, the study highlights managerial implications, future research directions and limitations.