Skip to main content

Advertisement

Springer Nature Link
Log in

FANTASIA: a framework for advanced natural tools and applications in social, interactive approaches

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

With the recent availability of industry-grade, high-performing engines for video games production, researchers in different fields have been exploiting the advanced technologies offered by these artefacts to improve the quality of the interactive experiences they design. While these engines provide excellent and easy-to-use tools to design interfaces and complex rule-based systems to control the experience, there are some aspects of Human-Computer Interaction (HCI) research they do not support in the same way because of their original mission and related design patterns pointing at a different primary target audience. In particular, the more research in HCI evolves towards natural, socially engaging approaches, the more there is the need to rapidly design and deploy software architectures to support these new paradigms. Topics such as knowledge representation, probabilistic reasoning and voice synthesis demand space as possible instruments within this new ideal design environment. In this work, we propose a framework, named FANTASIA, designed to integrate a set of chosen modules (a graph database, a dialogue manager, a game engine and a voice synthesis engine) and support rapid design and implementation of interactive applications for HCI studies. We will present a number of different case studies to exemplify how the proposed tools can be deployed to develop very different kinds of interactive applications and we will discuss ongoing and future work to further extend the framework we propose.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. https://unity3d.com/

  2. www.unrealengine.com

  3. https://neo4j.com/developer/cypher-query-language

  4. www.mivoq.it

  5. www.sc.sdus.edu

  6. Data extracted from the 20/04/2017 Wikipedia.it dump.

  7. https://sites.google.com/view/sugar-evalita/

References

  1. André C, Ghio A, Cavé C, Teston B (2007) PERCEVAL: a computer-driven system for experimentation on auditory and visual perception. arXiv:0705.4415

  2. Byun TM, Tiede M (2017) Perception-production relations in later development of american english rhotics. PloS One 12(2):e0172,022

    Article  Google Scholar 

  3. Caselli MC, Casadio P (1995) Il primo vocabolario del bambino. Franco Angeli, Milano

    Google Scholar 

  4. Cera V, Origlia A, Cutugno F, Campi M (2018) Semantically annotated 3d material supporting the design of natural user interfaces for architectural heritage. In: Proceedings of the AVI-CH workshop (to appear)

  5. Cosi P, Paci G, Sommavilla G, Tesser F (2016) Mivoq-ptts-a revolutionary new way of thinking tts. In: Proceedings of interspeech, pp 3888–3889

  6. Di Maro M, Valentino M, Riccio A, Origlia A (2017) Graph databases for designing high-performance speech recognition grammars. In: IWCS 2017—12th international conference on computational semantics—short papers

  7. Dietze F, Karoff J, Valdez AC, Ziefle M, Greven C, Schroeder U (2016) An open-source object-graph-mapping framework for neo4j and scala: Renesca. In: International conference on availability, reliability, and security. Springer, pp 204–218

  8. Drakopoulos G, Kanavos A, Makris C, Megalooikonomou V (2015) On converting community detection algorithms for fuzzy graphs in neo4j. In: Proceedings of the 5th International Workshop on Combinations of Intelligent Methods and Applications, CIMA

  9. González J, Escobar J, Sánchez H, De la Hoz J, Beltrán J (2017) 2d and 3d virtual interactive laboratories of physics on unity platform. In: Journal of physics: conference series, vol 935. IOP Publishing, p 012069

  10. Hornecker E, Stifter M (2006) Learning from interactive museum installations about interaction design for public settings. In: Proceedings of the 18th Australia conference on computer-human interaction: design: activities, artefacts and environments. ACM, pp 135–142

  11. Irwansyah F, Yusuf Y, Farida I, Ramdhani M (2018) Augmented reality (ar) technology on the android operating system in chemistry learning. In: IOP Conference series: materials science and engineering, vol 288. IOP Publishing, p 012068

  12. Jiménez P, Diez JV, Ordieres-Mere J (2016) Hoshin kanri visualization with neo4j. empowering leaders to operationalize lean structural networks. Procedia CIRP 55:284–289

    Article  Google Scholar 

  13. Kersten TP, Tschirschwitz F, Deggim S (2017) Development of a virtual museum including a 4d presentation of building history in virtual reality. Int Archives Photogrammetry Remote Sens Spatial Inform Sci 42:361

    Article  Google Scholar 

  14. Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (2005) A conversational agent as museum guide–design and evaluation of a real-world application. In: International workshop on intelligent virtual agents. Springer, pp 329–343

  15. Kuhl PK (2004) Early language acquisition: cracking the speech code. Nature Rev Neuroscience 5(11):831–843

    Article  Google Scholar 

  16. Lison P, Kennington C (2016) Opendial: a toolkit for developing spoken dialogue systems with probabilistic rules. ACL 2016:67

    Google Scholar 

  17. Martinie C, Navarre D, Palanque P, Barboni E, Canny A (2018) Toucan: an ide supporting the development of effective interactive java applications. In: Proceedings of the ACM SIGCHI symposium on engineering interactive computing systems. ACM, p 4

  18. McKeown G, Valstar MF, Cowie R, Pantic M (2010) The SEMAINE corpus of emotionally coloured character interactions. In: Proceedings of ICME, pp 1079–1084

  19. Niewiadomski R, Bevacqua E, Mancini M, Pelachaud C (2009) Greta: an interactive expressive eca system. In: Proceedings of the 8th international conference on autonomous agents and multiagent systems-Volume 2, 1399–1400. International Foundation for Autonomous Agents and Multiagent Systems

  20. Origlia A, Cosi P, Rodà A, Zmarich C (2017) A dialogue-based software architecture for gamified discrimination tests. In: Proceedings of the first workshop on games-human interaction @ CHItaly

  21. Origlia A, Paci G, Cutugno F (2017) MWN-E: a graph database to merge morpho-syntactic and phonological data for italian. In: Proceedings of Subsidia

  22. Origlia A, Rodà A, Zmarich C, Cosi P, Nigris S, Colavolpe B, Brai I (2018) Gamified discrimination tests for speech therapy applications. In: Proceedings of the annual conference of the Italian association of speech science (AISV) (to appear)

  23. Origlia A, Rossi A, Chiacchio ML, Cutugno F (2016) Cultural heritage presentations with a humanoid robot using implicit feedback. In: Proceedings of the AVI-CH workshop

  24. Origlia A, Savy R, Poggi I, Cutugno F, Alfano I, D’Errico F, Vincze L, Cataldo V (2018) An audiovisual corpus of guided tours in cultural sites: data collection protocols in the chrome project. In: Proceedings of the AVI-CH workshop (to appear)

  25. Petersen T (1990) Developing a new thesaurus for art and architecture. Library Trends 38(4):644–658

    Google Scholar 

  26. Pianta E, Bentivogli L, Girardi C (2002) Developing an aligned multilingual database. In: Proceedings of the 1st international conference on global wordnet

  27. Polka L, Jusczyk PW, Rvachew S (1995) Methods for studying speech perception in infants and children, Speech perception and linguistic experience: Issues in cross-language research, 49–89

  28. Qiu W, Yuille A (2016) Unrealcv: connecting computer vision to unreal engine. In: European conference on computer vision. Springer, pp 909–916

  29. Schmid S (1999) Fonetica e fonologia dell’italiano Paravia scriptorium

  30. Shah S, Dey D, Lovett C, Kapoor A (2018) Airsim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and service robotics. Springer, pp 621–635

  31. Shrinivasan YB, Zhang Y (2017) CELIO: an application development framework for interactive spaces. arXiv:1710.01772

  32. Squire K, Jenkins H, Holland W, Miller H, Alice O, Philip Tan K, Todd K (2003) Design principles of next-generation digital gaming for education, Educ Technol, 17–23

  33. Tallal P (1976) Rapid auditory processing in normal and disordered language development. J Speech Language Hear Res 19(3):561–571

    Article  Google Scholar 

  34. Thiebaux M, Marsella S, Marshall AN, Kallmann M (2008) Smartbody: behavior realization for embodied conversational agents. In: Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems-Volume 1, 151–158. International Foundation for Autonomous Agents and Multiagent Systems

  35. De la Torre F, Hodgins J, Bargteil A, Martin X, Macey J, Collado A, Beltran P (2008) Guide to the carnegie mellon university multimodal activity (cmu-mmac) database, Robotics Institute, 135

  36. Traum D, Aggarwal P, Artstein R, Foutz S, Gerten J, Katsamanis A, Leuski A, Noren D, Swartout W (2012) Ada and grace: direct interaction with museum visitors. In: Intelligent virtual agents. Springer, pp 245–251

  37. Tsao FM, Liu HM, Kuhl PK (2004) Speech perception in infancy predicts language development in the second year of life: a longitudinal study. Child Development 75(4):1067–1084

    Article  Google Scholar 

  38. Valentino M, Origlia A, Cutugno F (2017) Multimodal speech and gestures fusion for small groups. In: Proceedings of the workshop on ”designing, implementing and evaluating mid-air gestures and speech-based interaction” @ CHItaly 2017 [online]

  39. Webber J (2012) A programmatic introduction to neo4j. In: Proceedings of the 3rd annual conference on systems, programming, and applications: software for humanity. ACM, pp 217–218

  40. Zmarich C, Bonifacio S (2005) Phonetic inventories in Italian children aged 18-27 months: a longitudinal study. In: INTERSPEECH, pp 757–760

Download references

Acknowledgements

Antonio Origlia’s work is funded by the Italian PRIN project Cultural Heritage Resources Orienting Multimodal Experience (CHROME) #B52F15000450001.

Author information

Authors and Affiliations

  1. URBAN/ECO Research Center, University of Naples “Federico II”, 80138, Napoli, NA, Italy

    Antonio Origlia

  2. Department of Electrical Engineering and Information Technology, University of Naples “Federico II”, 80138, Napoli, NA, Italy

    Francesco Cutugno

  3. Department of Information Engineering, University of Padua, 35122, Padova, PD, Italy

    Antonio Rodà

  4. Institute of Cognitive Sciences and Technology (CNR-ISTC), 35137, Padova, Italy

    Piero Cosi & Claudio Zmarich

Authors
  1. Antonio Origlia
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Francesco Cutugno
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Antonio Rodà
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Piero Cosi
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Claudio Zmarich
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Antonio Origlia.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Origlia, A., Cutugno, F., Rodà, A. et al. FANTASIA: a framework for advanced natural tools and applications in social, interactive approaches. Multimed Tools Appl 78, 13613–13648 (2019). https://doi.org/10.1007/s11042-019-7362-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7362-5

Keywords