Eye Tracking: A Process-Oriented Method for Inferring Trust in Automation as a Function of Priming and System Reliability

2019 ◽  
Vol 49 (6) ◽  
pp. 560-568 ◽  
Author(s):  
Yidu Lu ◽  
Nadine Sarter
Keyword(s):  
Eye Tracking ◽  
System Reliability ◽  
Trust In Automation ◽  
Process Oriented

On the Dual Nature of Transparency and Reliability: Rethinking Factors that Shape Trust in Automation

2017 ◽  
Vol 61 (1) ◽  
pp. 308-312 ◽  
Author(s):  
Elizabeth Kaltenbach1 ◽  
Igor Dolgov
Keyword(s):  
System Reliability ◽  
Automated Systems ◽  
Dual Nature ◽  
High Transparency ◽  
Trust In Automation ◽  
Prior Literature ◽  
Positive Valence ◽  
Trust Scale

Prior literature has found that increasing system reliability and transparency can positively impact operators’ trust of automated systems; however, these factors are typically confounded. In the present study, we separated these factors by manipulating different stages of automation. Participants engaged in a simulated coffee manufacturing task using an interface with differing levels of reliability (65% or 95%) and transparency (one line or multiple lines of system display). The Human Computer Trust Scale (HCTS) and the Trust in Automated Systems Scale (TAS) were used to measure trust. When examining scores on TAS items with a positive-valence, we novelly observed that transparency interacted with reliability, such that high transparency and low reliability negatively impacted trust in the system. Alternatively, trust was not negatively affected by poor reliability when transparency was low, due to trivial cost of corrective behaviors that compensated for poor reliability and lack of system history understanding by the operators.

Automation reliability and trust: A Bayesian inference approach

2018 ◽  
Vol 62 (1) ◽  
pp. 202-206 ◽  
Author(s):  
Chenlan Wang ◽  
Chongjie Zhang ◽  
X. Jessie Yang
Keyword(s):  
Bayesian Inference ◽  
System Reliability ◽  
Probabilistic Inference ◽  
Inference Process ◽  
Bayesian Estimates ◽  
Trust In Automation ◽  
Human Operators

Research shows that over repeated interactions with automation, human operators are able to learn how reliable the automation is and update their trust in automation. The goal of the present study is to investigate if this learning and inference process approximately follow the principle of Bayesian probabilistic inference. First, we applied Bayesian inference to estimate human operators’ perceived system reliability and found high correlations between the Bayesian estimates and the perceived reliability for the majority of the participants. We then correlated the Bayesian estimates with human operators’ reported trust and found moderate correlations for a large portion of the participants. Our results suggest that human operators’ learning and inference process for automation reliability can be approximated by Bayesian inference.

Description-Experience Gap: The Role of Feedback and Description in Human Trust in Automation

2017 ◽  
Vol 61 (1) ◽  
pp. 313-313 ◽  
Author(s):  
Scott Mishler ◽  
Jing Chen ◽  
Edin Sabic ◽  
Bin Hu ◽  
Ninghui Li ◽  
...  
Keyword(s):  
System Reliability ◽  
High Reliability ◽  
Warning System ◽  
Amazon Mechanical Turk ◽  
Interesting Phenomenon ◽  
Reliability Level ◽  
The People ◽  
Trust In Automation ◽  
Two Factors ◽  
Small Probabilities

Human trust in automation is widely studied because the level of trust influences the effectiveness of the system (Muir, 1994). It is vital to examine the role that the people play and how they interact with the system (Hoff & Bashir, 2015). In the decision-making literature, an interesting phenomenon is the description-experience gap, with a typical finding that experience-based choices underweight small probabilities, whereas description-based choices overweight small probabilities (Hertwig, Barron, Weber, & Erev, 2004; Hertwig & Erev, 2009; Jessup, Bishara, & Busemeyer, 2008). We applied this description-experience gap concept to the study of human-automation interaction and had Amazon Mechanical Turk workers evaluate emails as legitimate or phishing. An anti-phishing warning system provided recommendations to the user with a reliability level of 60%, 70%, 80%, or 90%. Additionally, the way in which reliability information was conveyed was manipulated with two factors: (1) whether the reliability level of the system was stated explicitly (i.e., description); (2) whether feedback was provided after the user made each decision (i.e., experience). Our results showed that as the reliability of the warning system increased, so did decision accuracy, agreement rate, self-reported trust, and perceived system reliability, consistent with prior research (Lee & See, 2004; Rice, 2009; Sanchez, Fisk, & Rogers, 2004). The increase in performance and trust with the increase in reliability indicates that participants were paying attention to and using the automation to make decisions. Feedback was also highly influential in performance and establishing trust, but description only affected self-reported trust. The effect of feedback strengthened at the higher levels of reliability, showing that individuals benefited the most from feedback when the automated warning system was more reliable. Additionally, unlike prior studies that manipulated description and experience/feedback separately (Hertwig, 2012), we varied description and feedback conditions systematically and discovered an interaction between the two factors. Our results show that feedback is more helpful in situations that do not provide an explicit description of the system reliability, compared to those who do. An implication of the current results for system design is that feedback should be provided whenever possible. This recommendation is based on the finding that providing feedback benefited both users’ performance and trust in the system, and on the hope that the systems in use are mostly of high reliability (e.g., > .80). A note for researchers in the field of human trust in automation is that, if only subjective measures of trust are used in a study, providing description of the system reliability will likely cause an inflation in the trust measures.

Conveying Automation Reliability and Automation Error Type An Empirical Study in the Cyber Domain

2018 ◽  
Vol 62 (1) ◽  
pp. 172-173
Author(s):  
Jing Chen ◽  
Scott Mishler ◽  
Bin Hu
Keyword(s):  
Decision Making ◽  
False Alarm ◽  
System Reliability ◽  
Detection System ◽  
False Alarms ◽  
Automation System ◽  
Error Type ◽  
Trust In Automation ◽  
Human Decision ◽  
Phishing Detection

Background Emails have become an integral part of our daily life and work. Phishing emails are often disguised as trustworthy ones and attempt to obtain sensitive information for malicious reasons (Egelman, Cranor, Hong, 2008;). Anti-phishing tools have been designed to help users detect phishing emails or websites (Egelman, et al., 2008; Yang, Xiong, Chen, Proctor, & Li, 2017). However, like any other types of automation aids, these tools are not perfect. An anti-phishing system can make errors, such as labeling a legitimate email as phishing (i.e., a false alarm) or assuming a phishing email as legitimate (i.e., a miss). Human trust in automation has been widely studied as it affects how the human operator interacts with the automation system, which consequently influences the overall system performance (Dzindolet, Peterson, Pomranky, Pierce, & Beck, 2003; Lee & Moray, 1992; Muir, 1994; Sheridan & Parasuraman, 2006). With interacting with an automation system, the human operator should calibrate his or her trust level to trust a system that is capable but distrust a system that is incapable (i.e., trust calibration; Lee & Moray, 1994; Lee & See, 2004; McGuirl & Sarter, 2006). Among the various system capabilities, automation reliability is one of the most important factors that affect trust, and it is widely accepted that higher reliability levels lead to higher trust levels (Desai et al., 2013; Hoff & Bashir, 2015). How well these capabilities are conveyed to the operator is essential (Lee & See, 2004). There are two general ways of conveying the system capabilities: through an explicit description of the capabilities (i.e., description), or through experiencing the system (i.e., experience). These two ways of conveying information have been studied widely in human decision-making literature (Wulff, Mergenthaler-Canseco, & Hertwig, 2018). Yet, there has not been systematic investigation on these different methods of conveying information in the applied area of human-automation interaction (but see Chen, Mishler, Hu, Li, & Proctor, in press; Mishler et al., 2017). Furthermore, trust and reliance on automation is not only affected by the reliability of the automation, but also by the error types, false alarms and misses (Chancey, Bliss, Yamani, & Handley, 2017; Dixon & Wickens, 2006). False alarms and misses affect human performance in qualitatively different ways, with more serious damage being caused by false-alarmprone automation than by miss-prone automation (Dixon, Wickens, & Chang, 2004). In addition, false-alarm-prone automation reduces compliance (i.e., the operator’s reaction when the automation presents a warning); and miss-prone automation reduces reliance (i.e., the operator’s inaction when the automation remains silent; Chancey et al., 2017). Current Study The goal of the current study was to examine how the methods of conveying system reliability and automation error type affect human decision making and trust in automation. The automation system was a phishing-detection system, which provided recommendations to users as to whether an email was legitimate or phishing. The automation reliability was defined as the percentage of correct recommendations (60% vs. 90%). For each reliability level, there were a false-alarm condition, with all the automation errors being false alarms, and a miss condition, with all the errors being misses. The system reliability was conveyed through description (with an exact percentage described to the user) or experience (with immediate feedback to help the user learn; Barron, & Erev, 2003). A total of 510 participants were recruited and completed the experiment online through Amazon Mechanical Turk. The experimental task consisted of classifying 20 emails as phishing and legitimate, with a phishing-detection system providing recommendations. At the end of the experiment, participants rated their trust in this automated aid system. The measures included a performance measure (the decision accuracy made by the participants), as well as two trust measures (participants’ agreement rate with the phishing-detection system, and their self-reported trust in the system). Our results showed that higher system reliability and feedback increased accuracy significantly, but description or error type alone did not affect accuracy. In terms of the trust measures, false alarms led to lower agreement rates than did misses. With a less reliable system, though, the misses caused a problem of inappropriately higher agreement rates; this problem was reduced when feedback was provided for the unreliable system, indicating a trust-calibration role of feedback. Self-reported trust showed similar result patterns to agreement rates. Performance was improved with higher system reliability, feedback, and explicit description. Design implications of the results included that (1) both feedback and description of the system reliability should be presented in the interface of an automation aid whenever possible, provided that the aid is reliable, and (2) for systems that are unreliable, false alarms are more desirable than misses, if one has to choose between the two.

An Eye-Tracking Study on Audiovisual Speech Perception Strategies Adopted by Normal-Hearing and Deaf Adults Under Different Language Familiarities

2020 ◽  
Vol 63 (7) ◽  
pp. 2245-2254 ◽  
Author(s):  
Jianrong Wang ◽  
Yumeng Zhu ◽  
Yu Chen ◽  
Abdilbar Mamat ◽  
Mei Yu ◽  
...  
Keyword(s):  
Speech Perception ◽  
Eye Tracking ◽  
Normal Hearing ◽  
Audiovisual Speech ◽  
Audiovisual Speech Perception ◽  
Allocation Pattern ◽  
Deaf Adults ◽  
Bilingual Speaker ◽  
Primary Hypothesis ◽  
Standard Chinese

Purpose The primary purpose of this study was to explore the audiovisual speech perception strategies.80.23.47 adopted by normal-hearing and deaf people in processing familiar and unfamiliar languages. Our primary hypothesis was that they would adopt different perception strategies due to different sensory experiences at an early age, limitations of the physical device, and the developmental gap of language, and others. Method Thirty normal-hearing adults and 33 prelingually deaf adults participated in the study. They were asked to perform judgment and listening tasks while watching videos of a Uygur–Mandarin bilingual speaker in a familiar language (Standard Chinese) or an unfamiliar language (Modern Uygur) while their eye movements were recorded by eye-tracking technology. Results Task had a slight influence on the distribution of selective attention, whereas subject and language had significant influences. To be specific, the normal-hearing and the d10eaf participants mainly gazed at the speaker's eyes and mouth, respectively, in the experiment; moreover, while the normal-hearing participants had to stare longer at the speaker's mouth when they confronted with the unfamiliar language Modern Uygur, the deaf participant did not change their attention allocation pattern when perceiving the two languages. Conclusions Normal-hearing and deaf adults adopt different audiovisual speech perception strategies: Normal-hearing adults mainly look at the eyes, and deaf adults mainly look at the mouth. Additionally, language and task can also modulate the speech perception strategy.

Activation and Persistence of Implicit Causality Information in Spoken Language Comprehension

2010 ◽  
Vol 57 (1) ◽  
pp. 5-16 ◽  
Author(s):  
Pirita Pyykkönen ◽  
Juhani Järvikivi
Keyword(s):  
Eye Tracking ◽  
Language Comprehension ◽  
Spoken Language ◽  
Visual World ◽  
Implicit Causality ◽  
Causality Information ◽  
Spoken Language Comprehension

A visual world eye-tracking study investigated the activation and persistence of implicit causality information in spoken language comprehension. We showed that people infer the implicit causality of verbs as soon as they encounter such verbs in discourse, as is predicted by proponents of the immediate focusing account ( Greene & McKoon, 1995 ; Koornneef & Van Berkum, 2006 ; Van Berkum, Koornneef, Otten, & Nieuwland, 2007 ). Interestingly, we observed activation of implicit causality information even before people encountered the causal conjunction. However, while implicit causality information was persistent as the discourse unfolded, it did not have a privileged role as a focusing cue immediately at the ambiguous pronoun when people were resolving its antecedent. Instead, our study indicated that implicit causality does not affect all referents to the same extent, rather it interacts with other cues in the discourse, especially when one of the referents is already prominently in focus.

An eye tracking system for analysis of pilots' scan paths.

1997 ◽  
Author(s):  
Paul A. Wetzel ◽  
Gretchen Krueger-Anderson ◽  
Christine Poprik ◽  
Peter Bascom
Keyword(s):  
Eye Tracking ◽  
Tracking System ◽  
Scan Paths
Sign in / Sign up

Export Citation Format

Share Document