Abstract:

Emotion and motion are complementary sides of our social experiences though seldom studied in tandem. This study investigates variations in emotional responses during movement coordination between a human and a Virtual Partner (VP). VP is an avatar whose virtual hand is displayed on a computer screen in front of a human participant. Its finger movement is driven by the Haken-Kelso-Bunz equations, an empirically validated model that captures intrapersonal, sensorimotor and interpersonal coordination. Thus the human being and the VP are able to coordinate with each other in a biologically and socially realistic manner. It is known that under certain coupling conditions, human beings tend to perceive VP as an intentional agent. 21 subjects were instructed to coordinate finger movement with VP in either inphase or antiphase patterns. By adjusting model parameters, we were able to manipulate the ‘intention’ of VP as cooperative or competitive with the human’s task goal. Here we investigate how humanness attribution, coordination patterns, and VP’s intention relate to measures of emotional response. Skin potential responses (SPR) were recorded to quantify the intensity of emotional response. In each of 80 trials, subjects coordinated for 8 sec. to be either inphase or antiphase with the VP, during which VP’s intention was (pseudo-) randomly assigned for every 4-sec. period to be either inphase (cooperative) or antiphase (competitive) with the human. At the end of each trial, participants rated the VP’s intention earlier/later in the coordination and whether the partner was actually another human being or a machine. SPR measurements were first validated and then applied to emotional responses during coordination and rating. We found greater emotional response when participants reported that their partner was human. This was observed both during coordination (ANOVA, p=0.020), and during rating (p=0.012). Furthermore, during rating, greater emotional response was found for cooperative than competitive behavior (p=0.012), modulated by VP’s change of intention and actual coordination pattern. We show, therefore, that emotional responses are strongly influenced by features of the virtual partner’s behavior associated with humanness, cooperation and change of intention. Our findings have implications for mental health (e.g. autism and schizophrenia) and the design of socially cooperative machines.

Keywords: Emotion, Human Computer Interaction, Social Neuroscience, Turing Test, Skin Conductance Response, Coordination Dynamics