Usability evaluation of a virtual reality smartphone app for a living museum

This paper elaborates the empirical evidence of a usability evaluation of a VR and non-VR virtual tour application for a living museum. The System Usability Scale (SUS) was used in between participants experiments (Group 1: non-VR version and Group 2: VR version) with 40 participants. The results show that the mean scores of all components for the VR version are higher compared to the non-VR version, overall SUS score (72.10 vs 68.10), usability score (75.50 vs 71.70), and learnability (58.40 vs 57.00). Further analysis using a two-tailed independent t test showed no difference between the non-VR and VR versions. Additionally, no significant difference was observed between the groups in the context of gender, nationality, and prior experience (other VR tour applications) for overall SUS score, usability score, and learnability score. Α two-tailed independent t test indicated no significant difference in the usability score between participants with VR experience and no VR experience. However, a significant difference was found between participants with VR experience and no VR experience for both SUS score (t(38) = 2.17, p = 0.037) and learnability score (t(38) = 2.40, p = 0.021). The independent t test results indicated a significant difference between participant with and without previous visits to SCV for the usability score (t(38) = −2.31, p = 0.027), while there was no significant differences observed in other components. It can be concluded that both versions passed based on the SUS score. However, the sub-scale usability and learnability scores indicated some usability issue.

The Cognitive Walkthough for Implementation Strategies (CWIS): A Pragmatic Method for Assessing Implementation Strategy Usability

Background. Implementation strategies have flourished over the last decade in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially-mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which implementation strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to surface aspects of strategies that may inhibit their usability and, ultimately, their effectiveness. This article presents a novel cognitive walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support mental health clinicians in the education sector to adopt measurement-based care.Method. The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially-mediated implementation strategies in health. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed a quantitative assessment of strategy usability.Results. Average subtask success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale: 0-100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative coding and classified by severity and problem type to explain the ratings. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.Conclusions. Ratings indicated that usability of the consultation protocol was at the low end of the “acceptable” range. Collectively, the 21 usability issues explained the ISUS quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.

Develop and evaluate an augmented reality posture training tool to promote work safety

Conventional work posture training tools included pamphlets, one-time training orientation, and/or videos. These tools did not always yield satisfactory training outcomes, and the incident rate of work-related musculoskeletal disorders did not substantially lower. In this research, modern augmented reality (AR) technology was leveraged to deliver interactive, holistic, whole-body visual information to convey safe work postures. The developmental procedure followed DMAIC by first defining specifications of training content, which led to the development of the training tool, including 3D reconstruction of a virtual instructor and building of user interface based on user-centered framework. This AR training tool was measured and analyzed through usability evaluation, and quantitative and qualitative data were obtained for cross-validation and usability issue source identification. Findings revealed the utility of 3D reconstruction of a virtual instructor and practicality of adopting conventional usability evaluation method for AR user interface usability evaluation. Feedback from the usability evaluation via questionnaire, think aloud, and post-task open-ended responses are employed to iteratively design the next version of the AR posture training tool.

Assessing the Usability of the Automated Self-Administered Dietary Assessment Tool (ASA24) among Low-Income Adults

The Automated Self-Administered Dietary Assessment Tool (ASA24) is a web-based tool that guides participants through completion of a 24-h dietary recall and automatically codes the data. Despite the advantages of automation, eliminating interviewer contact may diminish data quality. Usability testing can assess the extent to which individuals can use the ASA24 to report dietary intake with efficiency, effectiveness, and satisfaction. This mixed-methods study evaluated the usability of the ASA24 to quantify user performance and to examine qualitatively usability issues in a sample of low-income adults (85% female, 48.2 years on average) participating in a nutrition coupon program. Thirty-nine participants completed a 24-h dietary recall using the ASA24. Audio and screen recordings, and survey responses were analyzed to calculate task times, success rates, and usability issue frequency. Qualitative data were analyzed thematically to characterize usability issues. Only one participant was able to complete a dietary recall unassisted. We identified 286 usability issues within 22 general usability categories, including difficulties using the search function, misunderstanding questions, and uncertainty regarding how to proceed to the next step; 71.4% of participants knowingly misentered dietary information at least once. Usability issues may diminish participation rates and compromise the quality of ASA24 dietary intake data. Researchers should provide on-demand technical support and designers should improve the intelligence and flexibility of the ASA24’s search functionality.

Too much or too little? Investigating the usability of high and low data displays of the same electronic medical record

The high data density on electronic medical record screens is touted as a major usability issue. However, it may not be a problem if the data is relevant and well-organized. Our objective was to test this assumption using a comprehensive set of measures that assess the three pillars of usability: efficiency (both physical and cognitive), effectiveness, and satisfaction. Physicians were asked to go through a series of tasks using two versions of the same electronic medical record: one where all the display items were separated into tabs (the original display), and one where important display items were grouped logically in one tab (the redesigned display). Results supported the hypothesis that combining relevant data in organized fashion into a smaller location would improve usability. The findings highlight the role of good display organization to mitigate the effects of high data density, as well as the importance of assessing cognitive load as part of usability studies.

An Automatic User Activity Analysis Method for Discovering Latent Requirements: Usability Issue Detection on Mobile Applications

Starting with the Internet of Things (IoT), new forms of system operation concepts have emerged to provide creative services through collaborations among autonomic devices. Following these paradigmatic changes, the ability of each participating system to automatically diagnose the degree of quality it is providing is inevitable. This paper proposed a method to automatically detect symptoms that hinder certain quality attributes. The method consisted of three steps: (1) extracting information from real usage logs and automatically generating an activity model from the captured information; (2) merging multiple user activity models into a single, representative model; and (3) detecting differences between the representative user activity model, and an expected activity model. The proposed method was implemented in a domain-independent framework, workable on the Android platform. Unlike other related works, we used quantitative evaluation results to show the benefits of applying the proposed method to five Android-based, open-source mobile applications. The evaluation results showed that the average precision rate for the automatic detection of symptoms was 70%, and the success rate for user implementation of usage scenarios demonstrated an improvement of around 21%, when the automatically detected symptoms were resolved.

Legal Ontologies in ICT and Law

Re-usability is frequently declared as sine qua non feature of modern ontology engineering. Although thoroughly examined in general theory of knowledge management models the re-usability issue is still barely a declaration in the domain of legal ontologies. The similar situation also applies to statute-specific ontologies. Those knowledge modeling entities are well described especially as an opposition to the general application legal ontologies. Yet it is trivial to say that most of the developed legal ontologies so far are those generic ones. And this sole fact should not surprise as the very specialized knowledge models – usually harder to develop – are at the same time narrowed with their utility. Of course in terms of re-usability this simply means that this feature may be largely disabled in this kind of knowledge models. In this chapter we face both challenges, i.e. as an excuse for presentation of the most interesting in our opinion trends and works in the field we will demonstrate the practical approach to modeling copyright law case by re-using statute-specific ontologies.

Legal Ontologies in ICT and Law

As discussed in the fouth chapter, re-usability is frequently declared as sine qua non feature of modern ontology engineering. Although thoroughly examined in general theory of knowledge management models the re-usability issue is still barely a declaration in the domain of legal ontologies. The similar situation also applies to statute-specific ontologies. Those knowledge modeling entities are well described especially as an opposition to the general application legal ontologies. Yet it is trivial to say that most of the developed legal ontologies so far are those generic ones. And this sole fact should not surprise as the very specialized knowledge models – usually harder to develop – are at the same time narrowed with their utility. Of course in terms of re-usability this simply means that this feature may be largely disabled in this kind of knowledge models. In this chapter the authors face both challenges, i.e. as an excuse for presentation of the most interesting in their opinion trends and works in the field the authors demonstrate the practical approach to modeling copyright law case by re-using statute-specific ontologies.

Graphical symbols for consumer products in an international context

Graphical symbols are widely used on consumer and professional products. This paper discusses some of the practical issues involved in the design and application of graphical symbols, taking into account activities in the field of international standardisation and industrial practice. Special emphasis is given to the importance of understanding the role of graphical symbols in the communication process. The need to view the comprehension of graphical symbols as a usability issue is also stressed. Any meaningful statement about the comprehensibility of a graphical symbol must take users, tasks and context of use into account. The paper concludes with summary of guiding principles for designers based on the issues discussed.