A groundbreaking technology that can recognize human emotions in real time has been developed by a research team. This innovative technology is poised to revolutionize various industries, including next-generation wearable systems that provide services based on emotions.
Understanding and accurately extracting emotional information has long been a challenge due to the abstract and ambiguous nature of human affects such as emotions, moods, and feelings. To address this, the research team has developed a multi-modal human emotion recognition system that combines verbal and non-verbal expression data to efficiently utilize comprehensive emotional information.
At the core of this system is the personalized skin-integrated facial interface (PSiFI) system, which is self-powered, facile, stretchable, and transparent. It features a first-of-its-kind bidirectional triboelectric strain and vibration sensor that enables the simultaneous sensing and integration of verbal and non-verbal expression data. The system is fully integrated with a data processing circuit for wireless data transfer, enabling real-time emotion recognition.
Utilizing machine learning algorithms, the developed technology demonstrates accurate and real-time human emotion recognition tasks, even when individuals are wearing masks. The system has also been successfully applied in a digital concierge application within a virtual reality (VR) environment.
The technology is based on the phenomenon of “friction charging,” where objects separate into positive and negative charges upon friction. Notably, the system is self-generating, requiring no external power source or complex measuring devices for data recognition.
The senior author commented, “Based on these technologies, we have developed a skin-integrated face interface (PSiFI) system that can be customized for individuals.” The team utilized a semi-curing technique to manufacture a transparent conductor for the friction charging electrodes. Additionally, a personalized mask was created using a multi-angle shooting technique, combining flexibility, elasticity, and transparency.
The research team successfully integrated the detection of facial muscle deformation and vocal cord vibrations, enabling real-time emotion recognition. The system’s capabilities were demonstrated in a virtual reality “digital concierge” application, where customized services based on users’ emotions were provided.
The first author of the study, stated, “With this developed system, it is possible to implement real-time emotion recognition with just a few learning steps and without complex measurement equipment. This opens up possibilities for portable emotion recognition devices and next-generation emotion-based digital platform services in the future.”
The research team conducted real-time emotion recognition experiments, collecting multimodal data such as facial muscle deformation and voice. The system exhibited high emotional recognition accuracy with minimal training. Its wireless and customizable nature ensures wearability and convenience.
Furthermore, the team applied the system to VR environments, utilizing it as a “digital concierge” for various settings, including smart homes, private movie theaters, and smart offices. The system’s ability to identify individual emotions in different situations enables the provision of personalized recommendations for music, movies, and books.
The senior author emphasized, “For effective interaction between humans and machines, human-machine interface (HMI) devices must be capable of collecting diverse data types and handling complex integrated information. This study exemplifies the potential of using emotions, which are complex forms of human information, in next-generation wearable systems.”
https://www.nature.com/articles/s41467-023-44673-2
Latest News
Protein that helps COVID-19…
By newseditor
Posted 26 Jul
Spinal Muscular Atrophy (SM…
By newseditor
Posted 26 Jul
Link between bowel movement…
By newseditor
Posted 26 Jul
Inhibition of IL-11 signall…
By newseditor
Posted 25 Jul
Brain changes linked to obe…
By newseditor
Posted 25 Jul
Other Top Stories
Alternate splicing sequence determinants identified!
Read more
MicroRNAs regulating cholesterol and triglyceride homeostasis ident…
Read more
RNA Controls Poly-glutamine Protein Phase Transitions
Read more
Nuclear membrane repairs the 'dark matter' of DNA
Read more
Transmembrane protein sorting driven by membrane curvature
Read more
Protocols
A systems biology approach…
By newseditor
Posted 24 Jul
quantms: a cloud-based pipe…
By newseditor
Posted 22 Jul
Emerging tools and best pra…
By newseditor
Posted 19 Jul
Directly selecting cell-typ…
By newseditor
Posted 17 Jul
PUFFFIN: an ultra-bright, c…
By newseditor
Posted 16 Jul
Publications
Hepatocyte-intrinsic SMN de…
By newseditor
Posted 26 Jul
Aberrant bowel movement fre…
By newseditor
Posted 26 Jul
A pseudoautosomal glycosyla…
By newseditor
Posted 26 Jul
Microglia protect against a…
By newseditor
Posted 26 Jul
Rigor and reproducibility i…
By newseditor
Posted 26 Jul
Presentations
Myelin plasticity in the ve…
By newseditor
Posted 10 Jun
Hydrogels in Drug Delivery
By newseditor
Posted 12 Apr
Lipids
By newseditor
Posted 31 Dec
Cell biology of carbohydrat…
By newseditor
Posted 29 Nov
RNA interference (RNAi)
By newseditor
Posted 23 Oct
Posters
A chemical biology/modular…
By newseditor
Posted 22 Aug
Single-molecule covalent ma…
By newseditor
Posted 04 Jul
ASCO-2020-HEALTH SERVICES R…
By newseditor
Posted 23 Mar
ASCO-2020-HEAD AND NECK CANCER
By newseditor
Posted 23 Mar
ASCO-2020-GENITOURINARY CAN…
By newseditor
Posted 23 Mar