The Objective
Establish a framework to examine the changes in nonverbal communication (NVC) in virtual environments (VEs), and use this framework to gain insight into how NVC has evolved over time.
Framework for Virtual Worlds
Virtual worlds fall into three categories, defined by technological advances having transformative effects on how users are able to use nonverbal communication in a virtual environment. The three main categories of this framework are the MUD, the PC-based virtual world, and the head-mounted display based virtual reality environment. Each “genre” changed the ways that users employed NVC in a way that we found to be significant, and is expanded upon in this paper.
Framework for Studying Nonverbal Communication
To understand now NVC was employed in these VEs, we established a framework that breaks down NVC into its constituent parts. These parts are:
- Kinesics
- Facial Expressions
- Haptics
- Oculesics
- Proxemics
- Vocalics
- Chronemics
- Physical appearance and Environment
Virtual Worlds
The Multi-User Dungeon
MUDs were the first true multi-user virtual environments. They had a completely text-based interface, which allowed for complete control over avatar descriptions, but because all interactions needed to be typed, lacked the ability to express truly unconscious NVC actions.
A screenshot of a standard MUD interface.
PC-Based Virtual Worlds
PC-based VEs introduced graphical interfaces, creating clear lines of gaze and proximity between avatars that were fuzzy in MUDs. Later PC VEs would introduce voice communication, which alongside text communication gave users access to very rich ways to express vocalics.
Several Second Life avatars sitting together.
HMD-Based Virtual Reality
Head and body tracking introduced with VR technology allowed previously unprecedented ways for users to inhabit their avatars, creating gesture, gaze, and proxemic behaviors that were nearly identical to these behaviors observed in face to face communication.
Multiple avatars in VRChat enter a building.
Findings
Certain forms of gesture and facial expressions are communicated well through an avatar in a virtual world, but only in VR are we able to consistently observe unintentional or subconscious movements. Haptics and gaze struggle throughout history as inter-avatar movements are not generally well-implemented, but user experiences with these have improved significantly with the introduction of body and head tracking. Unique implementations of NVC beyond developer intent were observed with higher frequency in earlier platforms, but drop off as broader ranges of NVC are supported, and the actions to use this NVC through an avatar become more natural.
Recommendations for Future Research
While gaze and kinesics are very well studied, it is unclear or less-studed how users are able to use the ability to alter and create virtual worlds within these VEs to communicate information on any level. Additionally, it is unclear from the current body of research if voice communication differs in a VE than telephony or video calling.