The impact of hyperrealistic virtual humans on cultural heritage dissemination

Theoretical basis

This paper’s theoretical foundation combines affordance theory and situated to study users’ cognition of virtual human design in specific cultural contexts.

The term “affordance” was first proposed by the American ecological psychologist Gibson⁷. He emphasized the organism’s direct perception theory⁸ and further proposed the “direct perception-action” theoretical framework⁹. The concept of affordance has had a broad impact in fields such as human‒computer interaction, computer vision, media communication, information systems, and design. Scholars have conducted various studies and practical works on affordance classification^10,11, neural generation mechanisms^12,13, interaction design^14,15, etc., and have achieved many effective results. For example, Markus et al.¹⁶ proposed the important concept of technological affordance related to information technology, and Xie Weihong et al.¹⁷ constructed the technological affordance theoretical framework according to the logic of “situation-mechanism-result.” Zou Jing et al.¹⁸ suggested that affordance cues are jointly generated by the dimensions of human characteristics, product or service systems, and human behavior.

The American cognitive psychologist Norman introduced affordance theory into the fields of design and human‒computer interaction, defining it as the possibility of action that is readily perceivable by an agent¹⁹. Norman argued that perceived affordance provides effective cues for good interaction between users and products during the user experience process²⁰. This provides new ideas for design practice by facilitating the exploration of human perception and cognition of design objects and environments. Therefore, affordance theory is widely applied in design practice fields such as interaction design, product design, interface design, and landscape design²¹.

In the field of digital virtual design, affordance plays a crucial role, providing dynamic technical responses for the creativity and design of digital products. Affordance emphasizes the impact of technology, media, and the environment on the user experience and behavior. The digital design cues driven by affordances can effectively enhance the user experience by helping users better understand and respond to digital objects in the new media environment. In digital environments, affordance involves both cross-platform multimedia presentations and direct influences on users’ perceptions of design elements, including product shapes, interaction methods, and scenes^14,22. Exploring affordance cues to understand the relationships between users and design and guiding users to achieve expected behaviors through appropriate design methods can improve design effectiveness and user experience quality.

Situated cognition theory was first proposed by scholars such as Brown, who suggested that effective cognitive activity is not simply information processing but an action that must be realized and developed in real situations and behaviors²³. Researchers have explored this topic from multiple dimensions, including learning theory, ecological psychology, and cognitive psychology^24,25,26. In practice, situated cognition theory, which is widely applied in fields such as human‒computer interaction, information design, and user experience²⁷, provides a comprehensive framework to explain how cognitive activities are carried out in specific contexts and how contexts influence individuals’ perceptions, understandings, and memories. This theory contributes to a comprehensive understanding of the complexity and diversity of cognitive processes.

Situated cognition is closely related to user experience. By studying users’ needs, motivations, and emotions through perceptual experiences in specific contexts, situated cognition exerts a comprehensive influence on the user’s experience and integrates all factors in the interaction process for product or service design. In the experience economy, “user-centered” communication is the logical starting point for cultural innovation communication, emphasizing the user’s subjective experience and self-perception during interactions with products²⁸. User experience has become a key indicator for measuring product quality^29,30. Hence, scholars have conducted various effective studies on users’ experiences and perceptions from different angles. For example, Brajnik et al.³¹ suggest that user experience includes all perceived aesthetic experiences, the product’s meaningful experiences to users, and the emotional experiences generated by users. Alben³² finds that user experience encompasses user feelings, product understanding, goal completion, and application scenarios of the product. Konstantakis et al.³³ construct a three-level index for evaluating user experience. Hassenzahl and Tractinsky³⁴ propose that the formation of a user experience is the result of interactions among users, contexts, and systems. Yu Guoming³⁵, from the perspective of cognitive neuroscience communication, divides the perception of user experience into three aspects, namely, usefulness, ease of use, and satisfaction, while distinguishing among sensory experience, interactive experience, and emotional experience. Luo Shijian et al.³⁶ argue that the realization of functions requires users to identify various symbolic relationships in specific social and cultural contexts. Xin Xiangyang³⁷ suggests that user behavior itself is influenced by context and that user experience design is a contextualized practical activity.

Research framework

Through digital dissemination methods, cultural heritage can be innovatively transformed and developed, continuously attracting social attention through new communication channels³⁸. On the basis of affordance theory, this study sorts out the design factors of cultural heritage hyperrealistic virtual humans from five affordance cues: emotion, vision, interaction, voice, and situation. In accordance with situated cognition theory, a user cognition experiment was conducted on the Douyin platform with short video content. The aim is to investigate users’ cognition and attitudes toward hyperrealistic virtual human design through situational experiences in detail and propose effective optimization design strategies. This not only promotes the deep integration of digital virtual technology and cultural heritage but also provides clear strategic suggestions for the protection, innovation, and development of cultural heritage. The theoretical framework of the research is presented in Fig. 1.

Theoretical framework of the study.

Research design

This study adopts an experimental method, first allowing participants to watch and experience specified cultural heritage hyperrealistic virtual human short videos through social media and then collecting and analyzing the data through a questionnaire survey.

To avoid interference from social media platforms in terms of media ecology and user range on research results, this study selects the same social media platform—Douyin (the Chinese version of TikTok)—for sample screening. Douyin, a social media platform in China where short videos are the main content form, currently has approximately 700 million active users³⁹. Douyin is not only a content viewing platform but also a social interaction space where users can interact, comment, like, and share content^40,41. First, keyword searches were conducted on the Douyin platform. Independent accounts (i.e., noninstitutional accounts) with tags such as “cultural heritage hyperrealistic virtual humans” and “cultural heritage hyperrealistic digital humans” were initially screened to establish a preliminary experimental sample database. Next, on the basis of the preliminary screening, the follower count, posting activity, and interaction frequency data of the selected accounts were compared. Accounts with duplicate cultural heritage hyperrealistic virtual human identities were rescreened, and the best account was retained. Notably, accounts not updated after September 1, 2023, were deleted to ensure the validity of the samples. After multiple rounds of screening, the top 5 virtual humans were selected as research samples on the basis of the comprehensive ranking of “follower count & like count” (see Table 1).

After the experimental samples were selected, the specific experimental materials were further determined. The short videos of the five sample accounts were sorted in descending order on the basis of their popularity. The top 8 short videos from each account were then selected as experimental materials, resulting in a total of 40 experimental materials.

This experiment takes cultural heritage hyperrealistic virtual humans as the research object, aiming to analyze users’ perceptions and recognition of virtual human design elements in detail. The independent variables are the design factors of cultural heritage hyperrealistic virtual humans, and the dependent variable is user satisfaction. Therefore, based on affordance theory, this study investigates virtual human character design, styling design, dynamic design, sound design, and scene design from five dimensions, namely, personality, vision, interaction, sound, and environment. The five design factors and specific indicators of hyperrealistic virtual humans are displayed in Fig. 2.

Design factors of hyperrealistic virtual humans in cultural heritage.

In the formal experiment, to further ascertain the comprehensiveness and validity of the design factors and specific indicators, in-depth interviews were conducted for verification. Virtual human design requires a certain level of expertise, and accordingly, the interviewees must have experience with virtual digital humans. Therefore, 9 interviewees, 5 males and 4 females aged 20–45 years, were selected from different population categories (researchers, practitioners, and students) using informant interviews⁴². The educational backgrounds of the 9 interviewees involved cultural heritage protection, computer and intelligence, digital media, and design art studies. Among them, 4 had doctoral degrees, 3 had master’s degrees, and 2 were undergraduate students. The interview method was primarily face-to-face, with some online interviews. The interviews were conducted in December 2023, and each interview lasting 20–30 min. The interview content records are available in Appendix A.

After the interviews were completed, the textual content of the interviews was organized, summarized, and analyzed using an affinity diagram⁴³. It was found that all the information mentioned by the interviewees was already included in the five design factors and their specific indicators, with no new factors identified. Therefore, the results of the user interviews further validated that the design factors and specific indicators of hyperrealistic virtual humans for cultural heritage are comprehensive and reliable.

To ensure the reliability and stability of the experiment, the participants were required to have experience using Douyin. Initially, 32 people were recruited, all of whom voluntarily participated and were aware of the experimental requirements. They all had normal corrected vision with no color blindness or color weakness. Before the formal experiment, 2 participants withdrew for personal reasons, leaving 30 participants. The gender ratio of the participants was 1:1; the age distribution from highest to lowest was 26–30 years (30%), 18–25 years (26.67%), 31–40 years (20%), 41–50 years (13.33%), and 51–60 years (10%). The age distribution matches the main user demographic of Douyin.

The first stage of the experiment was user experience. To control for order effects, the viewing sequence of the five virtual human accounts (each with eight videos) was randomized for each participant. The participants were free to perform media interaction behaviors while watching videos on Douyin, with each participant watching for approximately 30 min. The second stage of the experiment involved a questionnaire survey. According to the 5 experimental samples, each participant needed to complete 5 corresponding questionnaires. The questionnaire design used a 5-point Likert scale^44,45, with question options including “Excellent, Good, Neutral, Poor, Bad.” The questionnaire design is available in Appendix B. Participants completed the questionnaires through the WJX platform ( a platform providing functions equivalent to Amazon Mechanical Turk). The two stages of the user cognition experiment are shown in Fig. 3.

Two stages of the user cognition experiment.

The reliability and validity of the 150 questionnaires from 30 participants were subsequently analyzed using the SPSSAU (Statistical Product and Service Solutions AU, The reliability analysis of the sample data revealed that the Cronbach’s α coefficient of the questionnaire was 0.9096 ( > 0.8), indicating high reliability and stability, which is suitable for further analysis; the validity analysis revealed that the KMO value was 0.7106, and Bartlett’s test was significant (p < 0.05). The validity reflected by the questionnaire data is good, indicating that respondents could accurately express their opinions and feelings when answering questions, thereby indicating that the questionnaire data are true and reliable and suitable for factor analysis.