Deliverable: tangible emotion-recognition vase
Role: interaction designer, user researcher, hardware engineering
Methods: user research, interaction design, design ideation, hardware and software engineering
Artifacts: Tangible User Interface Emotion Vase, github, video
Social media platforms currently fall short when trying to fulfill the psychological needs humans seek, such as love, belonging, self-actualization, and privacy. We aim to explore the tangibility of these intangible human relationships through a tangible user interface: Momento.
Momento is designed to be a pair of IoT charms or tokens, and shared with someone you care about. When the two tokens are together, they record your activities, movements, sounds, and emotions, which can be displayed in a tangible display, and a message can be sent to your pair that you are missing them. The system is designed to encourage people to engage in more in person activities, and the ambient nature of the product will remember the moments for the relationship and rewards the two partners for creating more memories.
In our design exploration, we created a prototype that still explores this tangibility of relationships through a vase form. The emotion flower vase detects a user’s emotion using the Affectiva SDK , moves when the user talks to it, and can text the partner the user’s current emotion by pressing on a flower petal.
Before designing our concept of the memento, our team started by researching how people in relationships interact with each other. We asked participants about all types of relationships: partners, parents, siblings, best friends. From our research, we found that people enjoyed giving gifts to their loved ones, and enjoyed participatory gamification like in Fitbit. We set out to design an ambient device that could be given as a gift, and encouraged the pair to spend time together to create more beautiful and complex interactions based on their activities. These activities could be measured by accelerometer for movement, geolocation, photo sensor, force sensor, and volume sensor. A storyboard from our findings can be seen below.
From our storyboard, we began designing possible tokens for our momento. We wanted the tokens to be wearable charms, modeling the size dimensions from watch face sizes. We also wanted to explore possible shapes that could represent different relationships: a heart for love, a circle for friendship, and an silly oblong shape for parent-child relationships, etc. We modelled these designs in clay and tested if they were of proper dimensions for wearability, with the intention of using a small display on each of the tokens for visualizations abstracted by the inputs.
Pivoting: Inspiration from Organics
Some technical problems with implementing the bluetooth and the feasibility of a small screen on each of these tokens lead us to pivot to investigating some interesting concepts from our original research: people like giving presents to their loved ones, including flowers. We were inspired by the nitinol “memory” wire used by Robotany in Breeze, where the organic tree is able to sense the presence of visitors and move based on their proximity.
We wanted to capture that same essence of organic life and beautiful subtle movements to create the tangible elements of our design around social relationships. Our next iteration featured origami flowers and nitinol wire in a circuit, triggered by force sensors. Users can have their flowers “bloom” when they are thinking of their partner.
Final Design: Emotion Vase
Still related to our original momento idea, we were inspired by the idea of giving flowers to a loved one: flowers that never wither, are able to sense your emotions and respond to your presence, while still being able to communicate with the person who gave you this gift how you feel.
Our flowers wiggle when the user talks to it (volume sensor and servo motor tied to each flower), and is able to send a text of the user’s emotions to their partner by pushing a butterfly (hidden force sensor) in the petal. We use the Affectiva SDK to capture the user’s emotion, and our vase responds by changing color to match the emotion displayed.
From observing participants, it is clear that people liked interacting with the flower element. As stated by one participant: “I had fun with this project because I don’t normally expect plants to interact with me when I talk to them… if someone was on the other end [via text message], the movements would remind me we’re connected.” There was something that was familiar and novel about designing a flower vase to represent tangibility of relationships: flowers are often given as gifts, can easily be placed anywhere in a home without appearing out of place or obtrusive. However, these flowers will persist long after traditional flowers and will react to your presence, which is a new and novel experience users appreciated and enjoyed.
Participants found emotion recognition to be a valuable part of capturing a relationship in theory, but in practice many users were interested in the emotion recognition as a game rather than experiencing the intended interaction of ambient design. This effect could be eliminated if the camera was built into the Memento itself, and the Memento was implemented in a longitudinal study within the participant’s daily lives, connected to someone who they cared about. From a study like this, we could see if Memento captures a tangible way for people to communicate, and possible interactions that we may not have intended (e.g., “I will make a surprised face to send to you since I like those emojis better than happy emojis.”).
Memento explores the physical affordance of tangible design and how it could potentially enhance relationships. The initial observations of users interacting with Memento have shown the desire of users to communicate with their loved one in tangible ways, including a shared interest in the ability to “send emotions” though an IoT device and a gentle reminder that their loved one is always near through the subtle movement of the flowers. For this iteration, we focused on the vase since it is immediately responsive to an individual’s interaction. The overarching goal of the project is to have users carry charms as they go through their day, interacting with the people they care about, and the vase is their representation of all their relationships when they get home. Memento is designed to be ambient in nature, an unobtrusive reminder of one’s relationships even over distance, and to encourage people to engage with each other in real life activities and creating memories together. By capturing and sharing emotion, activities, location of individuals, we are able to capture relationships at a new depth that current avenues of social media cannot. Ongoing research is being conducted as to what interactions would be the most valuable for the charms to capture and how best to represent them within the flower vase.