Boiling the frog: ethical and behavioral impacts of technological exposure and availability

Emotion detection is a rapidly advancing method of quantifying the human experience. Past literature shows emotional data is highly sensitive and private. It also shows habituation effects (previous exposure to a concept) can result in more lenient ethical evaluation of actions. To build effective virtual agents, emotional data must be used in a way that is ethical and inoffensive to humans. Agents which are designed to interact with humans in virtually any capacity should strive to better understand them to be more competitive, more understanding, and generally more effective. We must understand the impact that agents using such data will have on people. To explore these points, we have conducted a 168-participant 2x2 experimental design with an additional user-choice factor to examine effects of pre-exposure to emotion detection on the participant's evaluation of its ethicality. We hypothesize that habituation affects participants' ethical evaluation of the technology presented to them. We found these effects and behavioral impacts when participants played an economic game. We show that the agent's presence on human perception and behavior is significant; proper agent design requires attention to these factors.