Marina Josué
ABSTRACT
Nowadays multimedia applications are employed in different fields as entertainment, education, government services, health and e-commerce. Moreover, some applications incorporate sensory effects along with traditional multimedia content, stimulating other human senses beyond sight and hearing to convey information. Those applications are called mulsemedia applications, where the maintenance of synchronization among media objects and sensory effects is a key point for user quality of experience. In order to minimize delays or failures on content reproduction, this paper proposes a new operation on media objects and sensory effects that can be offered to authors of mulsemedia applications. This operation enables the preparation of media object presentation or the preparation of sensory effect rendering, considering device limitations on which the application will be executed. As proof of concept, our proposal is implemented on multimedia applications using the NCL language, through specification of a new NCL event type, named preparation event. Furthermore, a new NCL media property is presented, allowing the control of continuous media reproduction. Finally, three use cases are defined to demonstrate the use of the preparation event in multimedia and mulsemedia applications.
- Zhong Su, Qiang Yang, and Hong-Jiang Zhang. A prediction system for multimedia pre-fetching in internet. In Proceedings of the eighth ACM international conference on Multimedia, pages 3--11. ACM, 2000. Google Scholar
Digital Library
- B. D. Davison. Predicting web actions from html content. In Proceedings of the thirteenth ACM conference on Hypertext and hypermedia, pages 159--168. ACM, 2002. Google ScholarDigital Library
- R. F. Rodrigues and L. F. G. Soares. A framework for prefetching mechanisms in hypermedia presentations. In Multimedia Software Engineering, 2002. Proceedings. Fourth International Symposium on, pages 278--285. IEEE, 2002. Google ScholarDigital Library
- A. M. Figueroa. Pré-busca de conteúdo em apresentações multimídia, Março 2014.Google Scholar
- E. B. Saleme and C. A. S. Santos. Playsem: a platform for rendering mulsemedia compatible with mpeg-v. In Proceedings of the 21st Brazilian Symposium on Multimedia and the Web - WebMedia'15, pages 145--148. ACM, 2015. Google ScholarDigital Library
- G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. Mulsemedia: State of the art, perspectives, and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 11:17, 2014. Google ScholarDigital Library
- J. Kim, C.G. Lee, Y. Kim, and J. Ryu. Construction of a haptic-enabled broadcasting system based on the mpeg-v standard. Signal Processing: Image Communication, 28(2):151--161, 2013. Google ScholarDigital Library
- M. Waltl, C. Timmerer, and H. Hellwagner. Improving the quality of multimedia experience through sensory effects. In Quality of Multimedia Experience (QoMEX), 2010 Second International Workshop on, pages 124--129. IEEE, 2010.Google ScholarCross Ref
- M. F. de Sousa, R. Kulesza, and C. A. G. Ferraz. A model-driven approach for mulsemedia application domain. In Proceedings of the 22nd Brazilian Symposium on Multimedia and the Web - WebMedia'16, pages 111--118. ACM, 2016. Google ScholarDigital Library
- M. F. de Sousa, C. A. G. Ferraz, R. Kulesza, I. Ayres, and M. Lima. Mulsemaker: An mdd tool for mulsemedia web application development. In Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web, pages 317--324. ACM, 2017. Google ScholarDigital Library
- M. Josue, R. Abreu, F. Barreto, D. P. Mattos, G. Amorim, J. A. F. Dos Santos, and D. C. Muchaluat-Saade. Modeling sensory effects as first-class entities in multimedia applications. In ACM Multimedia Systems Conference, 2018, Amsterdam. MMSys 2018, pages 1--12. ACM, 2018. Google ScholarDigital Library
- Z. Yuan, T. Bi, G.M. Muntean, and G. Ghinea. Perceived synchronization of mulsemedia services. IEEE Transactions on Multimedia, 17(7):957--966, 2015.Google ScholarDigital Library
- ITU-Recommendation. Nested context language (ncl) and ginga-ncl for iptv services. April 2009.Google Scholar
- W3C. Html audio/video dom currenttime property, 2018. https://www.w3schools.com/tags/av_prop_currenttime.asp.Google Scholar
- L. F. G. Soares, M. F. Moreno, C. S. S. Neto, and M. F. Moreno. Ginga-ncl: declarative middleware for multimedia iptv services. IEEE Communications Magazine, 48(6):74--81, 2010. Google ScholarDigital Library
- Britta Meixner and Christoph Einsiedler. Download and cache management for html5 hypervideo players. In Proceedings of the 27th ACM Conference on Hypertext and Social Media, pages 125--136. ACM, 2016. Google ScholarDigital Library
- W3C. Html5, 2018. https://www.w3schools.com/.Google Scholar
- W3C. Media source extensions. w3c recommendation 17 november 2016, 2016. https://www.w3.org/TR/media-source/.Google Scholar
- Mozilla Firefox. https://www.mozilla.org/pt-br/firefox, 2018.Google Scholar
- Google Chrome. https://www.google.com.br/chrome/, 2018.Google Scholar
- Internet Explorer. https://www.microsoft.com/pt-br/download/internet-explorer.aspx, 2018.Google Scholar
- D. Fisher et al. Link prefetching faq, {online}, 2014. Disponível em: https://developer.mozilla.org/en-US/docs/Web/HTTP/Link_prefetching_FAQ.Google Scholar
- K. Yoon. End-to-end framework for 4-d broadcasting based on mpeg-v standard. Signal Processing: Image Communication, 28(2):127--135, 2013. Google ScholarDigital Library
- B. Choi, E.S. Lee, and K. Yoon. Streaming media with sensory effect. In Information Science and Applications (ICISA), 2011 International Conference on, pages 1--6. IEEE, 2011.Google ScholarCross Ref
- Information technology -- Media context and control -- Part 3: Sensory information, 2011.Google Scholar
- Real Time Transport Protocol and User Datagram Protocol. Mpeg-2 ts.Google Scholar
- Roberto Ierusalimschy. Programming in lua. R. Ierusalimschy, 2006.Google Scholar
- ISO/IEC. Information technology - media context and control - part 6: Common types and tools. Technical report, International Organization for Standardization 23005-6., 2013.Google Scholar
Index Terms
- Preparation of Media Object Presentation and Sensory Effect Rendering in Mulsemedia Applications
Recommendations
Modeling sensory effects as first-class entities in multimedia applications
MMSys '18: Proceedings of the 9th ACM Multimedia Systems ConferenceMultimedia applications are usually composed by audiovisual content. Traditional multimedia conceptual models, and consequently declarative multimedia authoring languages, do not support the definition of multiple sensory effects. Multiple sensorial ...
Mulsemedia preparation: a new event type for preparing media object presentation and sensory effect rendering
MMSys '19: Proceedings of the 10th ACM Multimedia Systems ConferenceIn multimedia applications, a short delay in content delivery may cause synchronization failures among media objects, decreasing the presentation quality and user quality of experience. Moreover, some applications incorporate sensory effects to ...
Automatic preparation of media objects in multimedia applications
WebMedia '19: Proceedings of the 25th Brazillian Symposium on Multimedia and the WebIn multimedia applications, spatiotemporal relationships among media objects should be controlled during the execution phase in order to preserve the quality of presentation. When the content that composes the application is delivered over a ...
Comments