Modern surface transportation vehicles often include different levels of automation. Higher automation levels have the potential to impact surface transportation in unforeseen ways. For example, connected vehicles with higher levels of automation are at a higher risk for hacking attempts, because automated driving assistance systems often rely on onboard sensors and internet connectivity (Amoozadeh et al., 2015). As the automation level of vehicle control rises, it is necessary to examine the effect different levels of automation have on the driver-vehicle interactions. While research into the effect of automation level on driver-vehicle interactions is growing, research into how automation level affects driver’s responses to vehicle hacking attempts is very limited. In addition, auditory warnings have been shown to effectively attract a driver’s attention while performing a driving task, which is often visually demanding (Baldwin, 2011; Petermeijer, Doubek, & de Winter, 2017). An auditory warning can be either speech-based containing sematic information (e.g., “car in blind spot”) or non-sematic (e.g., a tone, or an earcon), which can influence driver behaviors differently (Sabic, Mishler, Chen, & Hu, 2017). The purpose of the current study was to examine the effect of level of automation and warning type on driver responses to novel critical events, using vehicle hacking attempts as a concrete example, in a driving simulator. The current study compared how level of automation (manual vs. automated) and warning type (non-semantic vs. semantic) affected drivers’ responses to a vehicle hacking attempt using time to collision (TTC) values, maximum steering wheel angle, number of successful responses, and other measures of response. A full factorial between-subjects design with the two factors made four conditions (Manual Semantic, Manual Non-Semantic, Automated Semantic, and Automated Non-Semantic). Seventy-two participants recruited using SONA ( odupsychology.sona-systems.com ) completed two simulated drives to school in a driving simulator. The first drive ended with the participant safely arriving at school. A two-second warning was presented to the participants three quarters of the way through the second drive and was immediately followed by a simulated vehicle hacking attempt. The warning either stated “Danger, hacking attempt incoming” in the semantic conditions or was a 500 Hz sine tone in the non-semantic conditions. The hacking attempt lasted five seconds before simulating a crash into a vehicle and ending the simulation if no intervention by the driver occurred. Our results revealed no significant effect of level of automation or warning type on TTC or successful response rate. However, there was a significant effect of level of automation on maximum steering wheel angle. This is a measure of response quality (Shen & Neyens, 2017), such that manual drivers had safer responses to the hacking attempt with smaller maximum steering wheel angles. In addition, an effect of warning type that approached significance was also found for maximum steering wheel angle such that participants who received a semantic warning had more severe and dangerous responses to the hacking attempt. The TTC and successful response results from the current experiment do not match those in the previous literature. The null results were potentially due to the warning implementation time and the complexity of the vehicle hacking attempt. In contrast, the maximum steering wheel angle results indicated that level of automation and warning type affected the safety and severity of the participants’ responses to the vehicle hacking attempt. This suggests that both factors may influence responses to hacking attempts in some capacity. Further research will be required to determine if level of automation and warning type affect participants ability to safely respond to vehicle hacking attempts. Acknowledgments. We are grateful to Scott Mishler for his assistance with STISIM programming and Faye Wakefield, Hannah Smith, and Pettie Perkins for their assistance in data collection.
Usability Evaluation—Advances in Experimental Design in the Context of Automated Driving Human–Machine Interfaces