Augmented (AR) and virtual reality (VR) have found their way to everyday life. The idea that the future of humans in the AR and VR tech space includes multiple opportunities to enhance (augment) performance and provide new ways for experiencing and interacting with the world appeals to many. Will this be the next stage of “body and sensing smart”?
Current health and wellbeing markets are quite a jungle of wearable, self-monitoring gadgets. Research databases have an exhausting number of publications on how the human autonomic and central nervous system physiology are affected by a wide range of internal human factors such as vigilance, alertness, stress, emotions and external environmental elements like temperature, noise and visual stimuli (to name a few). Already in 1960 Science published an article on how visual stimuli affect eye pupil size which is controlled by the autonomic nervous system. The pupil has since been claimed to be a direct window to the human mind and emotions.
I find myself repeatedly pondering a question I was asked to address in one of my brain research lectures in the early 2000: Would you like to actually see another person’s emotional state? I was asked to imagine having a kaleidoscope that is able to translate into colors changes in human physiology caused by different emotions. With this “emotion-scope” I could actually see people emitting these emotions.
Emotion kaleidoscope – changing eye color and hovering clouds
Now, nearly twenty years later, as AR and VR have arrived, my imagination has created into my mind’s eye a scene in which I encounter humans whose eyes change color depending on emotions they are experiencing at a certain time point. As an alternative, I go around wearing emotion–radar–glasses that make it possible for me to see a color halo around a person that is caused by a change in electrodermal activity of the skin initiated by emotions. I also “see” a person with a “black” cloud above the head denoting bad temper and another with a rosy one hovering above the head of a joyful person. These two people with very different clouds are part of a team discussing a work topic. A bunch of clouds of varying size and color are floating above the humans. How should I, the person who put forth the need to start doing things differently, handle this palette of emotions. Is it really helpful information to have?
From sprouting device wires to wireless health tech wearables
The question of an emotion radar was first put to me in the early days of the Brainwork Research Lab at The Finnish Institute of Occupational Health where I was working as a research professor. Nearly twenty years ago, a study setup combining physiologic and cognitive measurements required that the person participating was strapped to a chair. Numerous wires were sprouting out from sensors placed on different body areas and connected to data gathering boxes. Not a very comfortable study setting even for an eager volunteer to endure.
The era of wireless, more user-friendly technologies was around the corner and it didn’t take all that many years before wearables claiming to measure a wide range of factors affecting human physiology, performance and behavior appeared in the health tech market.
The Brain-Mind-Body connection
The emotional, feeling brain soon became a media favorite. It has been well documented in a multitude of studies that the amygdala a brain area anatomically resembling an almond and located deep in the middle parts of the brain is an important key player in detecting and storing experienced emotions. Thinking of a past event can reactivate emotions one experienced in the past. People that were present at the same event remember different things from it and also their memory emotional imprints from it differ. A team outing can emotionally be experienced as great fun or a pain in the neck and a wide range of other emotions.
Emotions trigger bodily responses and affect autonomic nervous system physiology. This “body-data” is forwarded to the amygdala. So physiologic devices that measure e.g., heart rate, breathing and sweat have been linked to emotions. They were fast claimed to provide objective data on a person’s feelings.
Mirroring emotions
Frontal lobes contain neural structures (e.g., mirror cells) that enable us mentally to relate to other people. We can understand that the state of mind or life situation of another person participating in a mutual conversation or activity may be different from our own. The cognitive skill to finetune one’s way of speaking by being able to imagine different emotional responses to words spoken is a social skill that in this SOME age is too often under-used.
It is true that it is sometimes difficult to relate to and get a “reading” of another person. The mouth may be saying one thing, but the truth of the matter stays hidden in one’s mind. A group of people can be individually in very different mind sets and emotional states. Thus, I understand that there may be eagerness to develop augmented and virtual reality-based solutions that would make human emotions visible. Still, I strongly caution against this for many reasons.
Emotion tsunami
Human emotions are triggered by a multitude of factors. A person’s mood or emotional reaction may not be a response to something occurring presently. Or the person may be thinking of something totally different that has caused the emotional state. Also, if I would have a device with which to see the emotional responses of a group of people to my behavior, I would be exposed to a variety of different emotions. I would be overwhelmed by an emotion tsunami. How to handle this situation? What type of emotions do I want to elicit and to what purpose? How do I respond to contradictory feelings? What about the intensity of the emotion and temperament differences? Some people are hot heads and others cool cucumbers come what may.
I have several times attended leadership training where people are urged to openly speak about emotions at workplaces. The take home message: Encourage people to freely express their feelings at team meetings. Ask about emotions, if needed. I confess, I have not followed this advice. I consider it to reflect insensitivity to human feelings. For many people it is difficult to speak up verbally even on concrete matters. The threshold for sharing one’s emotions is even higher.
Respect emotion privacy
Emotions are in many ways private. People should definitely have a right to this privacy. Using an emotion radar to get feedback on how people react to me is also unreliable. Providing people with emotion radars does not necessarily result in increase of empathy and understanding of others. We may also be letting a genie out of the bottle if this system is used to emotionally manipulate people. It can be a form of brain washing.
If a person grieving the loss of a loved one isn’t ready to share the sorrow, empathy is giving the person space. It should be enough to know that a person is going through a difficult time and isn’t willing to participate in group laugh yoga or other types of emotion bonding or training.
The effects of emotions on human neurophysiology also vary: Some get “fired up” easily by emotions that to some are only lukewarm physiologically. Thus, individual variety is very large. Also, the first emotional reaction to an unexpected development at work can after further thought change. So, what is the relevance of “seeing” another’s first reaction? It can in fact be harmful if it is taken as a final reaction. Often first reactions are coloured by the surprise element.
Hone your natural emotional intelligence
We humans read the emotional states of others by paying attention to facial expressions, gestures, body language, use of voice. This inherent skill can get trampled over when people go about their lives by rushing around and not taking the time to really have a look at what another person is naturally emitting. AR/VR based emotion capturing tools are not solutions with which to mend things when the skill of seeing and hearing non-verbal information is in poor shape due to under-use.
Emotional intelligence consists of several abilities: The ability to ponder and understand one’s own emotions and their origins. The ability to handle our emotions and if needed to change our ways of responding emotionally. The ability to understand that social contexts in which to show emotions vary. These abilities make it possible to relate to other peoples’ feelings.
The valuable filter of emotions
Empathy is the ability to relate to another person and understand that emotional responses to life events differ between people. A filter of emotions is a valuable thing in human interaction. We humans are entitled to emotional privacy and deciding what we want to share. Some futuristic, eager inventors, dream of systems that would enable direct emotion transfer between people without giving thought to what this might bring forth: Is it desirable that the emotion of hate can be easily transferred from one person to another? Developing augmented human solutions should not be done by wearing rosy glasses. The downsides of an appealing idea should be remembered.
Tangled emotions clouding reason
Would you personally want to share all of your emotions with others? Would you like to have a working culture where everyone is demanded to share their emotional responses to different issues arising in daily work and in addition, “truth of talk” -confirmation is done using an emotion radar? How would you feel if you would be exposed to a wide range of emotional information from a group of people? Could you tackle the overflow of emotions? What is a desired emotional response and what not? Who determines this? Would understanding between people significantly improve if everybody sees – during interaction – the changing eye colors, clouds hovering above the head or halos around the human body of everyone? I claim that the result would be a messy and tangled web of emotions and it would effectively cloud reason. Finding a good balance between listening to the heart and mind would be near impossible.
A blog in Finnish on this topic has also been published in my Tiedekokki blog in the magazine Tiede, so tune in!