김기락, 이상아, 김나현, 정문열. (2023). 메타버스 대화의 몰입감 증진을 위한 대화 감정 기반 실시간 배경음악 시스템 구현. 컴퓨터그래픽스학회논문지, 29(4), 1-6.

Kim Kirak, Lee Sangah, Kim Nahyeon, Jung Moonryul. Real-time Background Music System for Immersive Dialogue in Metaverse based on Dialogue Emotion. J Korea Comput Graph Soc 2023;29(4):1-6.

Abstract

To enhance immersive experiences for metaverse environements, background music is often used. However, the background music is mostly pre-matched and repeated which might occur a distractive experience to users as it does not align well with rapidly changing user-interactive contents.

Thus, we implemented a system to provide a more immersive metaverse conversation experience by 1) developing a regression neural network that extracts emotions from an utterance using KEMDy20, the Korean multimodal emotion dataset 2) selecting music corresponding to the extracted emotions from an utterance by the DEAM dataset where music is tagged with arousal-valence levels 3) combining it with a virtual space where users can have a real-time conversation with avatars.

[paper (korean)]