Constantinides, N., Constantinides, A., Koukopoulos, D., Fidas, C., Belk, M., "CulturAI: Exploring Mixed Reality Art Exhibitions with Large Language Models for Personalized Immersive Experiences", ACM UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization, Pages 102 - 105, 2024 https://doi.org/10.1145/3631700.3664874 [Best Demo Award]
Abstract
Mixed Reality (MR) technologies have transformed the way in which we interact and engage with digital content, offering immersive experiences that blend the physical and virtual worlds. Over the past years, there has been increasing interest in employing Artificial Intelligence (AI) technologies to improve user experience and trustworthiness in cultural contexts. However, the integration of Large Language Models (LLMs) into MR applications within the Cultural Heritage (CH) domain is relatively underexplored. In this work, we present an investigation into the integration of LLMs within MR environments, focusing on the context of virtual art exhibitions. We implemented a HoloLens MR application, which enables users to explore artworks while interacting with an LLM through voice. To evaluate the user experience and perceived trustworthiness of individuals engaging with an LLM-based virtual art guide, we adopted a between-subject study design, in which participants were randomly assigned to either the LLM-based version or a control group using conventional interaction methods. The LLM-based version allows users to pose inquiries about the artwork displayed, ranging from details about the creator to information about the artwork’s origin and historical significance. This paper presents the technical aspects of integrating LLMs within MR applications and evaluates the user experience and perceived trustworthiness of this approach in enhancing the exploration of virtual art exhibitions. Results of an initial evaluation provide evidence about the positive aspect of integrating LLMs in MR applications. Findings of this work contribute to the advancement of MR technologies for the development of future interactive personalized art experiences.
S. Sylaiou, P. Dafiotis, D. Koukopoulos, C. Koukoulis, R. Vital, A. Antoniou, and C. Fidas, "From Physical to Virtual Art Exhibitions and Beyond: Survey and some Issues for Consideration for the Metaverse", Journal of Cultural Heritage, Elsevier, 2023, accepted
Abstract
This manuscript addresses the increasing importance of Extended Reality (XR) in art exhibitions. It elaborates on the technologies employed, on design issues, pertinent aims, as well as evaluation metrics. The main research questions concern the types of technologies currently used, the primary design considerations, and the ways user experience can be enhanced in accordance with the artists’ and curators' vision, as well as visitors' needs. Moreover, the paper also examines relevant evaluation criteria. The manuscript reviews the types of technologies used for virtual art exhibitions and examines their design characteristics and pertinent issues on interfaces, resources, and affordances. An analysis of the use of virtual humans as a model for increased engagement follows a discussion of a comparison between virtual versus physical exhibitions. Moreover, the parameters that affect user experience are discussed thoroughly, given their pivotal importance in such exhibitions. Finally, the paper concludes with a discussion of the above-mentioned aspects of XR exhibitions with an emphasis on optimizing design for art-related content and offers informed suggestions for future directions. The overall scope of this paper is to examine ways to exploit XR as a creative medium rather than merely replicate physical exhibitions. In this context, the papers’ discussion includes the Metaverse.
R. Vital, S. Sylaiou, D. Koukopoulos, K. Koukoulis, P. Dafiotis, and C. Fidas, "Comparison of extended reality platforms and tools for viewing and exhibiting art", Digital Applications in Archaeology and Cultural Heritage, Vol. 31, 2023 https://doi.org/10.1016/j.daach.2023.e00298
Abstract
The rapid evolution of mobile smart devices and the development of inexpensive immersive devices (VR googles, google box etc.) in the past two decades have brought opportunities in the field of curating, storing, and viewing art. These new ways of experiencing art with the help of the latest technologies extend from physical to the virtual space, and any combination of the two. This paper presents a comparative study of selected existing Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) Platforms and Tools for viewing and exhibiting Art. This study explores the key features of each platform, whether the platform is open source or a commercial product, the enabling of interactivity between the users within each platform, and finally the strong and weak points of each. To conclude, this study explores how such platforms and tools can transform the field of curating and exhibiting art.
S. Sylaiou, P. Dafiotis, D. Koukopoulos, C. Koukoulis, R. Vital, A. Antoniou, and C. Fidas, "From Physical to Virtual Art Exhibitions and Beyond: Survey and some Issues for Consideration for the Metaverse", Journal of Cultural Heritage, Elsevier, 2023, accepted
Abstract
This manuscript addresses the increasing importance of Extended Reality (XR) in art exhibitions. It elaborates on the technologies employed, on design issues, pertinent aims, as well as evaluation metrics. The main research questions concern the types of technologies currently used, the primary design considerations, and the ways user experience can be enhanced in accordance with the artists’ and curators' vision, as well as visitors' needs. Moreover, the paper also examines relevant evaluation criteria. The manuscript reviews the types of technologies used for virtual art exhibitions and examines their design characteristics and pertinent issues on interfaces, resources, and affordances. An analysis of the use of virtual humans as a model for increased engagement follows a discussion of a comparison between virtual versus physical exhibitions. Moreover, the parameters that affect user experience are discussed thoroughly, given their pivotal importance in such exhibitions. Finally, the paper concludes with a discussion of the above-mentioned aspects of XR exhibitions with an emphasis on optimizing design for art-related content and offers informed suggestions for future directions. The overall scope of this paper is to examine ways to exploit XR as a creative medium rather than merely replicate physical exhibitions. In this context, the papers’ discussion includes the Metaverse.
Dimitrios Koukopoulos, Panagiotis Dafiotis, Stella Sylaiou, Konstantinos Koukoulis, and Christos Fidas, "Evaluating the design of an art student framework supporting XR exhibitions", 26th Pan-Hellenic Conference on Informatics with International Participation (PCI 2022)
Abstract
A successful system design may lead to a successfully implemented system. This paper demonstrates the followed evaluation procedure for the creation of a system dedicated to art schools. Specifically, we focus on meeting the needs of art teachers, art students, and visitors who are the stakeholders of such a system. Art teachers need to be able to initiate art exhibitions for their students, art students to create virtual exhibitions, and art exhibition visitors to have immersive and interactive experiences. This evaluation is a part of an EU-funded research project, namely CREAMS which aims to design, develop end evaluate a framework and open-source tools based on virtual, augmented, and mixed reality for the creation of virtual exhibitions for art students. Here, we present the followed user-centered design methodology that we followed to reach the first system design. The methodology begins with the literature review and continues with a filtering of the review with experts, the creation of the corresponding mockups, the evaluation of the mockups by the end users, the specification of quantitative and qualitative metrics on employing XR technologies in Higher Education Institutions and concludes with the identification of a set of specifications that drives the development of a specific conceptual system design.
Dimitrios Koukopoulos, Panagiotis Dafiotis, Stella Sylaiou, Konstantinos Koukoulis, Marios Belk and Christos Fidas, "Towards an art exhibition framework for scaffolding art students creativity", 9th International Euro-Mediterranean Conference, EuroMed 2022.
Abstract
Nowadays there is a necessity for the development of tools offering digital services in various learning environments due to the pandemic conditions. Art education is a learning environment where the provision of such tools is fragmentary. Especially in the field of art exhibitions, there is not a framework that can support all stakeholders, i.e. students offering them tools at all the stages of the exhibition procedure from the digitization of their artworks to the authorization and visualization of art exhibitions, as well as art school teachers providing them with suitable exhibition initiation and assessment services and visitors giving them applications (web-based or mobile apps) to have multimodal experiences ranging from virtual reality indoor art exhibitions, to augmented reality outdoor or in-home exhibitions, and mixed reality interactive exhibitions. Another issue that arises in such environments is the need for personalization due to their human-centric nature. In this work, we attempt to give a first answer to those needs by presenting an art exhibition framework for scaffolding art students’ creativity that is the outcome of the research conducted within an EU-funded research project, namely CREAMS. The first focus group evaluation results suggest that stakeholders are positive concerning the supported framework tools.
D. Koukopoulos, P. Dafiotis, S. Sylaiou, K. Koukoulis, A, Antoniou, R. Vital, and C. Fidas, "Personalization services in art education environments: first survey results", 13th IEEE International Conference on Information, Intelligence, Systems and Applications, 2022.
Abstract
Personalization is the main vehicle in many learning environments for the provision of enhanced services to the end-users. This paper deals with the problem of providing personalization services in an art education environment. Specifically, we are interested in assisting art teachers to initiate an art exhibition for their students, art students to create virtual exhibitions and art exhibition visitors to have immersive and interactive experiences. The results discussed here are one of the first outputs of an EU-funded research project, namely CREAMS which aims to design, develop end evaluate a framework and open-source tools based on virtual, augmented, and mixed reality for the creation of virtual exhibitions for art students. Specifically, in this paper, we present a literature review concerning eXtended Reality (XR) technologies and personalization for art education purposes. Based on the findings of the literature review, along with expert opinions, we present a system design incorporating specific personalization services. In order to emphasize the strength of the proposed personalization services in the framework, we present a set of motivating scenarios.
D. Koukopoulos, P. Dafiotis, S. Sylaiou, K. Koukoulis, and C. Fidas, "XR technologies for self-regulated student exhibitions in art education: survey and first design considerations," 2nd International Conference on Interactive Media, Smart Systems and Emerging Technologies, 2022.
Abstract
This paper provides a survey of the first results of an EU-funded research project, namely CREAMS that aims to design, develop and evaluate a framework and open-source tools based on virtual, augmented, and mixed reality for the creation of virtual exhibitions for art students. The virtual exhibitions will exploit the potential of digital technology to provide rich, multilayered, multimodal content such as digital narratives, that will contextualize the artworks, offer additional material, and engage audiences in more meaningful ways. Specifically, in this paper, we present a literature review concerning system designs, implementations, and best practices of XR technologies usage in art education for exhibition purposes. Our findings are then filtered based on experts’ feedback for triangulation purposes. Based on the finding of this preliminary user requirement analysis, we propose a first conceptual design of the envisioned system, along with a number of motivating scenarios emphasizing the main system functionalities. This is the first step of a user-centered design methodology we aim to apply throughout the building of the CREAMS framework.
Latest News
