With a Little Help from My Friend: Emotional Expressiveness in a Female Digital Human and User Gender Interact to Affect Psychological and Physiological Outcomes (Preprint)

BACKGROUND Loneliness is a growing public health problem that has been exacerbated in vulnerable groups during the COVID-19 pandemic. Social support interventions have been shown to improve loneliness and may be delivered through technology. Digital humans are a new type of computer agent that show promise as supportive peers in healthcare. For digital humans to be effective and engaging support persons, it is important that they can develop closeness with people. Closeness can be improved by emotional expressiveness, particularly in female relationships. However, it is unknown whether emotional expressiveness improves closeness in relationships with digital humans and affects physiological responses. OBJECTIVE This study investigated whether emotional expression by a digital human can affect psychological and physiological outcomes, and whether the effects are moderated by user gender. METHODS 198 healthy adults (101 females, 95 males, 2 gender-diverse individuals) were block-randomized by gender to complete a 15-minute self-disclosure conversation with a female digital human, in one of six conditions. In these conditions, the digital human varied in modality richness and emotional expression in the face and voice (emotional/ neutral/ no face; emotional/ neutral voice). Perceived loneliness, closeness, social support, caring perceptions, and stress were measured after the interaction. Physiological measures including heart rate, skin temperature, and electrodermal activity were collected during the interaction using an Empatica E4 watch. Three-way factorial ANOVA with post hoc tests were conducted to analyse the effect of face type, voice type, and user gender on outcomes. RESULTS Overall, emotional expression in the voice was associated with greater caring perceptions and physiological arousal during the interaction, and unexpectedly, lower feelings of support. Gender was found to moderate the effect of emotional expressiveness on loneliness, social, and certain physiological outcomes. For females, an emotional voice digital human was associated with improved perceptions of closeness, social support, and caring perceptions, whereas for males, a neutral voice digital human was associated with improvements in closeness, social support, and caring perceptions. For females, a neutral face was associated with lower loneliness and subjective stress compared to no face. Whereas interacting with no face (i.e., a voice only black screen) resulted in lower loneliness and subjective stress for males compared to a neutral or emotional face digital human. No significant results were found for heart rate or skin temperature. However, average electrodermal activity was significantly higher for males while interacting with the emotional voice digital human. CONCLUSIONS Findings suggest that emotional expressiveness in a female digital human has different effects on loneliness, social, and physiological outcomes for males and females. Results may inform the design of digital human support persons, and have theoretical implications. Further research is needed to evaluate how more pronounced emotional facial expressions in a digital human might impact results. CLINICALTRIAL Australia New Zealand Clinical Trials Registry (ANZCTR) registration application Id: 381816