Please use this identifier to cite or link to this item:
Title: Inverse and transitivity of cross-modal correspondence in mulsemedia
Authors: Mesfin, G
Hussain, N
Covaci, A
Ghinea, G
Keywords: Mulsemedia;video;audio;haptic;crossmodal correspondence quality of experience
Issue Date: 23-Jul-2018
Publisher: IEEE
Citation: 2018 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2018, 2018
Abstract: The human brain has evolved to learn, adapt and operate optimally in multisensory settings. Exploring the multisensory-phenomena in a human computer interaction (HCI) context could help design interfaces and displays that tap into users’ mental models. In this paper we explore the influence of crossmodal correspondences on the Quality of Experience (QoE) in a mulsemedia setup. A mulsemedia enhanced test-bed was developed to perform delivery of video enhanced with audio and haptic effects that were designed considering crossmodal congruencies principles. Accordingly, we performed an experiment on the correspondence from the visual features of selected videos into high/low pitch audio when they are experienced in conjunction with auto-generated haptic effects. Our results showed good insights into inverse and transitive cross-modal correspondences. In addition, the audio signal generated out of the angular shape (visual feature) and its corresponding haptic effect is found to have enhanced the viewers’ quality of experience.
Appears in Collections:Dept of Computer Science Research Papers

Files in This Item:
File Description SizeFormat 
FullText.docx510.83 kBMicrosoft Word XMLView/Open

Items in BURA are protected by copyright, with all rights reserved, unless otherwise indicated.