Archival interface design representing the study of emotion and physical reaction influenced by sound
Scope & My Role
Team Project
Motion Capture
Unity Prototyping
Team Member
Joy Lee
Bryana Lee
Timeline
2023, 10 Weeks
Project Type
Body Tracking
Human-Tech Prototype
Affection explores the subtle ways our environment influences both emotion and movement. We wanted to explore how sound—an often overlooked yet powerful external factor—can shape the way people physically and mentally respond to their surroundings.
Our experiment was guided by the hypothesis that changes in sound levels would influence both the physical movements and mental states of participants, leading to observable differences in their walking styles before and after exposure to unexpected sound. Through a series of motion-capture experiments, we observed how changes in sound levels influence walking patterns, posture, and mood, revealing the invisible connection between auditory stimuli, emotion, and the body.
Skip to Final Outcome
For our testing, we asked each participant to walk from point A to point B while blindfolded. To observe shifts in behavior before and after exposure to sound, we conducted three trials for each test: the first without any sound, the second with a loud object dropped during the walk, and the third again without sound.
Trial #1 - Before Sound
Trial #2 - Exposure to Sound
Trial #3 - After Sound
Before Sound
After Sound
Participants are focused on their task, but cautious
Participants are scared something will happen
Blind fold allows them to become more focused with their senses
Participants are more cautious and careful with their steps when walking
Confidence level is lower because of blind fold
Confidence is lost due to fear from the sound
The participants’ responses to sound in our experiment revealed a wide range of emotional and physical reactions, highlighting the powerful influence of sound on human behavior.
Some participants were slightly hesitant, showing slight fear while blindfolded, whereas others were confident.
When the sound was introduced, emotional responses varied: some participants remained relatively neutral, while others were visibly startled, even screaming.
Afterward, participants often appeared to expect something else to happen, with increased awareness and sensitivity visibly affecting their confidence in walking the same path.
Overall, these observations show that sound triggers not only immediate reactions but also impacts participants' movement patterns and confidence levels over time.
We imported the motion capture data into a default avatar in Rokoko Studio to visually compare each participant’s before-and-after states. After observing how the data translated onto the avatar, we began designing the interface and screen flows to demonstrate the comparison.
The experiment demonstrated that different sound levels (decibels) significantly impact how people react and move. The sound influenced not only participants’ immediate responses but also their movement afterward, leading them to walk more cautiously.
Our team was fascinated by exploring the concept of affection, especially how it can refer to both positive and negative responses. Typically, people think of affection as a positive emotion, but in our study demonstrated that some experiences can trigger discomfort or even traumatic reactions, ultimately influencing future behavior. This experiment shows that affection—our physical and emotional responses to unexpected sound—can take many forms and shape our behavior and perception of our surroundings, whether through calm or distress.