Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI

Abstract

Research in AI-enabled decision support often focuses on technological factors influencing reliance on AI. However, the end-users of AI systems are individuals with diverse personalities which potentially lead to differences in collaborative human-computer interaction, resulting in harmful under- and over-reliance. The influence of psychological traits on appropriate reliance must be understood to enable development of more effective AI support addressing a diverse user base. This experimental mixed-methods study (N = 250) examined the impact of psychological traits on trust in and reliance on AI advice in classification tasks. Propensity to trust, affinity for technology interaction, and control beliefs in interacting with technology were identified as predictors for trust, which affect reliance. Thus, consideration must be given to the expected propensity to trust and the level of technological expertise among user groups when designing systems that aim to promote suitable degrees of trust and appropriate reliance.

Keywords:

• AI reliance
• appropriate reliance
• personality
• trust
• decision support system

1. Introduction

In interactions between human and decision support systems based on artificial intelligence (AI), oftentimes the human needs to evaluate the AI-generated decision and determine whether to incorporate or disregard the advice while making their own judgments. However, recent research has revealed that support from AI is not as helpful as hoped. On the one hand, humans can demonstrate an inability to disregard incorrect advice and instead display an overreliance on AI, while on the other hand disregarding correct advice and rejecting automation support (Cabitza et al., 2023; Schemmer et al., 2022). Especially in critical decision situations like healthcare, this inappropriate reliance can have problematic consequences (Bussone et al., 2015). Thus, the aim of human-AI interaction should be to foster appropriate reliance. Schemmer et al. (2022) define appropriate reliance as the human capability to differentiate between correct and incorrect AI advice and to be able to act upon this discrimination.

Despite studies acknowledging the significance of human factors for AI interaction (Knop et al., 2022) there remains a lack of comprehensive studies that investigate how specific human characteristics impact proper utilization of AI tools. Therefore, there is a crucial need for empirical investigation exploring the interplay between human factors and appropriate reliance to enable the creation of guidelines and development of user-centered and reliable support systems (Felmingham et al., 2021; Riedl, 2022).

Distinguishing between good and bad AI advice may depend on individual capabilities given that the end-users of AI systems are humans with diverse personal experiences, preferences, and decision-making styles. Consequently, the collaborative nature of human-computer interaction can significantly vary from one person to another. Understanding which psychological traits aid users in discerning between effective and ineffective AI advice is pivotal for user-centered system development to not only facilitate their adoption but, more importantly, maximize the benefits of human-AI interaction in decision-making through appropriate reliance. Some users reject AI assistance, preferring to trust their own task expertise (Logg et al., 2019), while others display a general tendency to trust technology and therefore show higher trust in system advice (Faulhaber et al., 2021). Identifying the user groups that require particular support facilitates the development of support systems that enable them to make better decisions.

Human factors previously connected to the interaction of humans and AI systems are trust, task expertise, technological expertise, as well as decision styles (Felmingham et al., 2021; Knop et al., 2022; Tschandl et al., 2020). In a meta-study Knop et al. (2022) list human and technological characteristics that generally influence the collaboration between humans and AI-enabled clinical decision support systems. However, this does not offer insights regarding the appropriateness of reliance on AI systems but can serve as a starting point to identify factors that might be important for appropriate reliance. Propensity to trust technology influences the initial disposition toward relying on AI-generated advice (Lee & See, 2004) and thus the openness to AI suggestions that potentially affects reliance behavior. Investigating humans with different levels of task expertise is crucial to get insights on how humans with extensive domain knowledge might demonstrate differences in reliance behavior compared to novices, especially since some level of task expertise is required to be able to judge the correctness of advice (Logg et al., 2019). Affinity for technology interaction and control beliefs in dealing with technology represent technology expertise, by portraying an individual’s comfort and familiarity with technology as well as feelings of control over technology, which can have an impact on trust or mistrust in the system. Furthermore, need for cognition represents an individual’s inclination to engage in effortful cognitive activities which can result in critical evaluation of advice (Waggoner & Kennedy, 2022), leading to differences in AI advice reliance. Beyond these psychological traits, one factor previously connected to the advice utilization is the confidence of the individual in its initial decision (Tschandl et al., 2020). Thus, confidence might influence reliance behavior beyond trust and the impact of psychological traits. Through the selection of these factors a comprehensive overview can be gained over which factors collectively shape an individual’s approach in interacting with AI systems, impacting trust and reliance. Our investigations aim to explore how psychological traits influence trust and appropriate reliance on AI, addressing the research gap of the intricate connection between individual psychological traits, confidence, trust in AI and the extend of reliance on the system.

To investigate these factors, our study employs a mixed-method approach, combining a quantitative within-factors experimental study with a qualitative think-aloud study. The qualitative think-aloud study serves to explore an individual’s reasoning behind trusting and following AI recommendations and to gain a deeper understanding of factors influencing decision-making, confidence, and trust in AI. We address the following research question: How do psychological traits, such as propensity to trust technology, task expertise, technology expertise and need for cognition, influence trust in and appropriate reliance on AI?

2. Literature review

When looking into how humans adopt and interact with AI systems different concepts must be considered and differentiated. Initially, much research has focused on the acceptance and adoption of support systems (Shibl et al., 2013; Venkatesh et al., 2012), followed by different studies specifically investigating trust in AI (Ferrario et al., 2020; Molina & Sundar, 2022; Schmidt & Biessmann, 2020; Thiebes et al., 2021). Recent studies highlighted the importance of appropriate reliance beyond acceptance and trust (Benda et al., 2021; Cabitza et al., 2023; Chiou & Lee, 2023; Ma et al., 2023; Schemmer et al., 2022, 2023; Talone, 2019). In order to make sense of these concepts, the following subchapter discusses definitions, differences and interconnections and explains which specific constructs are used for this research and how they are instrumentalized.

2.1. Trust, reliance, and appropriate reliance

Trust is commonly defined as “the attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability” (Lee & See, 2004, p. 51). This attitude is not a fixed trait but a state related to the specific system used.

A person’s reliance on a system is explained by trust (Chiou & Lee, 2023). Specifically, Lee and See (2004) define trust as an attitude towards the automation, while reliance is the behavior derived from it. Humans rely on automation that they trust and reject systems they do not (Lee & See, 2004).

However, too much trust can lead to over-reliance and too little trust to the rejection of the system advice (Lee & See, 2004). Thus, recent research places particular emphasis on calibrated trust and appropriate reliance. Calibrated trust refers to the adjustment of expectations towards automation regarding its reliability, thus, trusting a system more or trusting it less, depending on its perceived capabilities, and by doing so leading to appropriate reliance (Lebiere et al., 2021; Okamura & Yamada, 2020; Y. Zhang et al., 2020). Thus, calibrated trust and appropriate reliance are interconnected and sometimes argued to be conceptually equal (Schemmer et al., 2022).

As described previously, appropriate reliance is defined as the human capability to differentiate between correct and incorrect AI advice and being able to act upon this discrimination (Schemmer et al., 2022). Collaborative interaction between humans and decision support systems should be the overarching objective. The final decision-maker, the human, needs to be able to decide when to accept and when to reject AI advice, to together achieve the highest accuracy possible (Schemmer et al., 2023). Schemmer et al. (2023) conceptualized relative AI reliance and relative self-reliance. They signify the percentage of cases in which users beneficially rely on AI or themselves when faced with system advice that contradicts their own assessment and therefore capture the appropriateness of reliance.

If humans would be able to distinguish between receiving correct and incorrect advice, they would rely more strongly on correct advice and would not follow incorrect advice (Schemmer et al., 2022). We formulate the following hypothesis to test for differences in the strength of reliance depending on whether the AI advice received is correct or incorrect. Therein, we get an overview of the general ability of participants to differentiate and appropriately rely on AI advice in our dataset.

H1. There is a difference in reliance on the system between receiving correct and incorrect AI advice.

Humans rely on automation that they trust and reject systems they do not trust (Lee & See, 2004), which is represented in trust attitudes predicting behavioral reliance (Chiou & Lee, 2023). Additional to testing whether this assumption can be replicated in our data based on the strength of AI advice utilization, we hypothesize that trust in the system will have a positive influence on relative AI reliance and accordingly a negative influence on relative self-reliance. This is based on decision making and advice taking research showing that a judges’ trust in an advisor is positively related to advice utilization (Bonaccio & Dalal, 2006). Further, high levels of trust have been connected to blind reliance and simply accepting all AI advice received (Bansal et al., 2021), which would translate to a negative influence on relative self-reliance. Thus, as a prerequisite for our investigations regarding psychological traits, we investigate the following hypothesis:

H2. Self-reported trust in the system has a positive influence on reliance on the system.

One important factor influencing whether humans consider advice, different from fixed traits, is their confidence in their initial decision. Confidence is defined as the belief of a person “that a specific statement, opinion, or decision is the best possible” (Sniezek & Van Swol, 2001, p. 290). Research on general advice taking has shown, that individuals who are less confident in their initial decision are more likely to seek and follow advice (Bonaccio & Dalal, 2006), while confidence in the initial decision leads to advice discounting (Wang & Du, 2018). Tschandl et al. (2020) gained similar insights regarding human-computer interaction, showing that participants receiving system advice adjusted their response less often when they were confident in their initial decision, compared to participants being less confident in their decision. Kopka et al. (2022) presented similar results, but they found a difference between the influence of confidence on self-reported and behavioral trust. Behavioral trust decreased with higher decision certainty, while there was no significant influence on self-reported trust. We propose the following hypothesis, assuming that whether participants trust the system and incorporate its advice in their final decision is dependent on their confidence in their initial decision.

H3. The influence of self-reported trust in the system on reliance on the system is moderated by the confidence in the initial decision.

2.2. Psychological traits

The subsequent subchapters will consider human characteristics relevant to technology acceptance and their role in shaping trust in and reliance on AI. Since trust has a central role in influencing reliance and has a crucial role in any human-AI interaction (Hengstler et al., 2016), we continuously place self-reported trust in a mediating role influencing the effect of different human factors on reliance. This is based on previous research that identified trust as a mediating factor (Cai et al., 2023), that specifically influences the relationship between individual differences in innovation resistance and system adoption (Handrich, 2021). The following factors were chosen based on previous research identifying them as important factors influencing system interaction (Knop et al., 2022). The selection should give a comprehensive overview to be able to understand how different psychological traits specifically shape trust and reliance to serve as a basis for creating more tailored and effective AI systems that cater to diverse user profiles and needs.

2.2.1. Propensity to trust

Propensity to trust technology, unlike the previous concepts of self-reported trust and reliance, is not a state but instead a fixed trait that predicts trust behavior. Propensity to trust is defined as “the stable, trait-like tendency to trust or not trust others” (Merritt & Ilgen, 2008, p. 195). In this research, propensity to trust technology describes an individual’s expectation that automation can be trusted. The propensity to trust can be assumed to be connected to self-reported trust (Faulhaber et al., 2021) as well as reliance. It is seen as a crucial factor for trust development (Lee & See, 2004; Schaefer et al., 2016). Lee and See (2004) connect high levels of propensity to trust to a better understanding of situations in which automation advice should be trusted or not. The following hypotheses propose propensity to trust as an influencing factor for reliance on AI systems’ advice, additionally mediated by the user’s self-reported trust in the system.

H4a. Propensity to trust technology has a positive influence on self-reported trust in the system.

H4b. Propensity to trust technology has a positive influence on reliance on the system.

H4c. The influence of propensity to trust technology on reliance on the system is mediated by self-reported trust in the system.

2.2.2. Task expertise

Especially when investigating over-reliance on automation, task expertise has been listed as one of the most critical factors for following AI advice within the expertise area (Cabitza et al., 2023; Gaube et al., 2022; Inkpen et al., 2023). In fact, Noga and Arnold (2002) found that users with a lower level of task expertise were more likely to blindly follow decision support advice. Sniezek and Van Swol (2001) reported that users with less expertise showed higher trust in a judge-advisor system, based on difficulties in making an independent choice. On the other hand, Yaniv (2004) found, that more knowledgeable individuals generally are more likely to discount advice. For human AI interaction, Logg et al. (2019) observed that in their forecasting experiment, participants with expertise in forecasting exhibited less over-reliance than non-expert users and instead rejected AI advice, even when the AI was correct. Based on these previous investigations, we hypothesize that high task expertise is connected to less self-reported trust and reliance on AI advice.

H5a. Task expertise has a negative influence on self-reported trust in the system.

H5b. Task expertise has a negative influence on reliance on the system.

H5c. The influence of task expertise on reliance on the system is mediated by self-reported trust in the system.

2.2.3. Affinity for technology interaction

Technology expertise is regularly listed as an influencing factor for system trust and reliance, but with different concepts, such as: perception towards automation (Asan et al., 2020), experience with computers (Sousa et al., 2015), innovativeness in IT (Handrich, 2021), past IT experience and ability to control a new technology (Fan et al., 2020).

Some of these factors, for example, perception towards automation and innovativeness in IT, can be read as an affinity towards automation and technological progress. Thus, we want to operationalize technology expertise as the affinity for technology interaction. Affinity for technology interaction is defined as “the tendency to actively engage in intensive technology interaction, as a key personal resource for coping with technology” (Franke et al., 2019, p. 456). Affinity for technology interaction specifically has been shown to be a positive predictor of trust in AI (Agrawal et al., 2023; Tolmeijer et al., 2021). Thus, we hypothesize the following:

H6a. Affinity for technology interaction has a positive influence on self-reported trust in the system.

H6b. Affinity for technology interaction has a positive influence on reliance on the system.

H6c. The influence of affinity for technology interaction on reliance on the system is mediated by self-reported trust in the system.

2.2.4. Control beliefs in dealing with technology

To represent the intricacies of technology expertise better and represent the other category of factors, such as experience with computers and the ability to control new technology, we use a second measure to quantify technological expertise: Control beliefs in dealing with technology describe the perception of mastery a person has over a technology (Beier, 1999). This is particularly important regarding Sharan and Romano (2020), who mention that control over technology may increase trust, since participants feel less anxious handling the system. Furthermore, Solberg et al. (2022) connect a person’s locus of control and the perceived control over AI decision aids to the trust in the system. Experience in handling technology was proposed as an important factor for system adoption and reliance in several studies (Asan et al., 2020; Aw et al., 2023; Felmingham et al., 2021). Thus, we complement the hypotheses regarding affinity for technology interaction with these three hypotheses:

H7a. Control beliefs in dealing with technology have a positive influence on self-reported trust in the system.

H7b. Control beliefs in dealing with technology have a positive influence on reliance on the system.

H7c. The influence of control beliefs in dealing with technology on reliance on the system is mediated by self-reported trust in the system.

2.2.5. Need for cognition

Personality aspects are mentioned as key factors influencing the adoption of and trust in decision support systems (Felmingham et al., 2021; Knop et al., 2022; Waggoner & Kennedy, 2022). In this work, we will investigate the impact of an individual’s need for cognition on trust in and appropriate reliance on AI advice. The need for cognition is a personality trait describing the tendency to engage in effortful cognitive tasks and enjoy it (Cacioppo & Petty, 1982; Lins De Holanda Coelho et al., 2020). People with varying levels of need for cognition differ in their cognitive styles and thus have different preferences for information processing (Haugtvedt et al., 1992; Lin et al., 2011). Understanding in which ways the need for cognition influences appropriate reliance can help tailor systems to user needs. Brennan et al. (2019) observed that more analytical thinkers, and therefore people potentially high in need for cognition changed their decision more frequently after interaction with an algorithm. Waggoner and Kennedy (2022) show that individuals with a high need for cognition were more likely to follow algorithmic advice as well as being more trusting towards automation. Thus, we propose the following hypotheses:

H8a. Need for cognition has a positive influence on self-reported trust in the system.

H8b. Need for cognition has a positive influence on reliance on the system.

H8c. The influence of need for cognition on reliance on the system is mediated by self-reported trust in the system.

3. Methods

This research aimed to investigate psychological traits that influence the impact of AI advice on decision-making. In the following section, we present our sample and the procedure of two studies that were conducted. Both studies used the same design and questionnaire. Study I was a quantitative online study providing data to uncover the impact of psychological traits on the appropriateness of reliance. Study II was a qualitative think-aloud study that we additionally conducted to explore the individuals’ understanding and reasoning behind trusting and following AI advice, therein gaining additional insights into and deeper understanding of decision-making, confidence in the decision, and trust in AI.

3.1. Study design

The studies were conducted from June to August 2023 and were available to English-speaking participants aged 18 years and older. In the experimental study part participants had to classify pictures concerning their art period based on the study from Vodrahalli et al. (2022). The study included an experimental within-subjects design, manipulating the correctness of AI advice on the classification task in order to gain insights about reliance behavior. Recruitment for the quantitative sample was aided by the crowdsourcing website Prolific. This research was previously approved by the ethics committee of the Department of Computer Science and Applied Cognitive Science of the Faculty of Engineering of the University of Duisburg-Essen (ID 2305SPKA7060), and Study I was preregistered on OSF (Küper, 2023).

3.2. Procedure

Participants were informed about the procedure, data collection and anonymization on the landing page of the study and informed consent was obtained. In the experimental part of the study, participants were presented with 24 binary classification tasks depicting art pieces that had to be sorted into two possible art periods (“Which art period is this painting from?”) from a selection of four art periods: Renaissance, Baroque, Romanticism, and Modern. Participants submitted ratings on a continuous sliding scale from 0 to 100 with two end points labelled “Definitely Art Period 1” e.g., Romanticism and “Definitely Art Period 2” e.g., Modern. They went through a two-stage decision process, first making an initial unaided decision. In the second step, they then received AI advice, which was said to be 80% accurate, based on Vodrahalli et al. (2022), and was presented on the sliding scale next to their initial response. They had the option to change their initial response to a new decision. Based on the accuracy of 80%, five out of the 24 classifications by the AI and thus the advice given was manipulated to be incorrect. This enabled comparison of reliance behavior between receiving correct and incorrect advice. The second step of the classification task is visualized in Figure 1.

Figure 1. Visualization of classification task with AI advice. Note. At the bottom the decision slider ranging from “Definitely Romanticism” to “Definitely Modern” with the participant’s slider position at 70% and the AI prediction at 80% indicating a classification into the Modern art period. The participant’s slider could be adjusted.

3.3. Dependent variables and confidence measure

3.3.1. Weight of advice

From the data collected during the experiment, including the initial response, the second response after receiving AI advice and the AI advice itself, the Weight of Advice (WoA) can be calculated. The WoA is a widely used measure to instrumentalize advice utilization (Yaniv, 2004) and is defined as follows: $$\mathit{\text{WoA}}=\frac{{\mathit{\text{response}}}_2-{\mathit{\text{response}}}_1}{\mathit{\text{advice}}-{\mathit{\text{response}}}_1},$$

WoA was clipped at −1 to 1 for a maximum magnitude of 100% adjustment to the advice. This represents the percentile shift of judgement after receiving advice. A positive value of WoA implies that the participants changed their response towards the advice, while a negative value suggests that they moved further away from it. A value closer to 1 or −1 represents a stronger shift in judgement.

3.3.2. Relative AI reliance & relative self reliance

Beyond WoA, we wanted to conceptualize appropriateness of reliance to put the shift in judgement into an additional context. Therefore, we utilized Relative AI Reliance (RAIR) and Relative Self-Reliance (RSR) introduced by Schemmer et al. (2023). These calculations are based on the correctness of the first and second response in relation to the AI advice received. RAIR is defined as “the ratio of the number of cases … in which humans rightfully change their mind to follow [correct AI advice]” (Schemmer et al., 2023, p. 413). It is calculated by dividing all instances of the participant being wrong, receiving correct AI advice and changing their response to the correct decision through all instances in which the participant was wrong initially and the AI gave correct advice, regardless of the final decision. RSR, on the other hand, refers to moments of correct self-reliance, when the AI advice was incorrect. It is calculated by dividing all instances in which the participant was correct, the AI advice incorrect and the final response stayed correct through all instances in which the participant initially was correct, and the AI was incorrect. Both measures reach from 0 to 1 and translate to percentages, where a value closer to 1 indicates more appropriate reliance (Schemmer et al., 2023).

3.3.3. Confidence

Confidence in the initial decision is hypothesized as an important moderating factor for the influence of trust on reliance. Based on Dreiseitl and Binder (2005) we did not specifically ask for confidence in the initial decision and instead utilized the placement of the decision slider as proxy information for confidence. It was instrumentalized as a binary measurement, 0 for not confident and 1 for confident in the initial decision. This was calibrated based on the results of the think-aloud study, with high confidence in the area of 0 to 20 and 80 to 100 on the decision scale, and low confidence in between.

3.4. Questionnaires

After the classification task, participants filled in a questionnaire assessing sociodemographic data (age, gender, education) and validated questionnaires measuring trust as a mediator and propensity to trust, art interest, affinity for technology interaction, control beliefs in dealing with technology, and need for cognition as independent variables. The questionnaires used will be introduced in the following section including the calculation of reliability measures for every questionnaire on the data of this study.

3.4.1. Trust

To measure trust, the global trust scale from Wischnewski et al. (2023) was used. The scale consists of five items that are referring to the previously used AI system and are rated on a five-point Likert scale from strongly agree to strongly disagree. An example item is “I trust the system.”, Cronbach’s alpha is .85.

3.4.2. Propensity to trust technology

Propensity to trust technology was measured using the questionnaire by Schneider et al. (2017). This questionnaire consists of six items rated on a five-point Likert scale ranging from strongly disagree to strongly agree. An example item is “I think it’s a good idea to rely on technology for help.”. Cronbach’s alpha for this questionnaire is .79.

3.4.3. Task expertise

To measure task expertise for the art classification task, the Vienna Art Interest and Art Knowledge Questionnaire from Specker et al. (2020) was selected. We opted to use the art interest sub-section since the art knowledge part appeared too specialized for a general audience and may not be applicable to the vast majority of respondents. The questionnaire uses seven items such as “I enjoy talking about art with others.” to measure agreement on a seven-point Likert scale from not at all to very much, as well as frequency on a seven-point Likert scale from less than once per year to once per week or more often with four items such as “How often do you read books, magazines or catalogues about art?”. Cronbach’s alpha for the complete scale is .92.

3.4.4. Technology expertise

The Affinity for Technology Interaction scale from Franke et al. (2019) was used with nine items being rated on a six-point Likert scale from completely disagree to completely agree. An example item is “I like testing the functions of new technical systems.”. Cronbach’s alpha is .85.

Control beliefs in dealing with technology was measured using the German “Kontrollüberzeugung im Umgang mit Technik (KUT)” (Beier, 1999) questionnaire, which was back and forth translated into English language. It consists of eight items on a five-point Likert scale from strongly disagree to strongly agree. An example item is “I can solve many technical problems that I face on my own.”. Cronbach’s alpha for this questionnaire is .81.

3.4.5. Need for cognition

To measure the need for cognition, this study employed the short six-item scale developed by Lins De Holanda Coelho et al. (2020). Items such as “I really enjoy a task that involves coming up with new solutions to problems.” are rated on a five-point Likert scale from extremely uncharacteristic of me to extremely characteristic of me. Cronbach’s alpha is .78.

3.5. Sample

Study I was an online survey which was completed by 263 participants. An a priori power analysis expecting a medium effect size with a coefficient of determination of R² = .13, a statistical power of .9, and a significance level of α = .05, resulted in a sample size of n = 124 required for a significant overall model with 6 predictors (Cohen, 1988; Hemmerich, 2019). The final sample contained 244 datasets after excluding all participants who failed two attention checks placed throughout the questionnaire. Of these participants, 121 (49.90%) identified themselves as male, 122 as female and one participant as gender-fluid. Age ranged from 18 to 73 years, with a mean of 29.91 years (SD = 8.82). Most participants (43.85%) had an undergraduate degree, followed by a graduate degree (22.13%) and a high school diploma (18.44%).

Study II consisted of a think-aloud study that was conducted with nine German-speaking participants (6 male, 2 female, 1 non-binary) who were fluent in English. Nonetheless, an additional German introduction and explanation of the experimental setup was provided to ensure complete comprehension. Participants were debriefed about the process of a think-aloud study and informed consent was obtained. The age of the qualitative sample ranged from 24 to 67 years (M = 33.11, SD = 13.00).

4. Results

4.1. Quantitative analysis

All quantitative analyses and the performance of the path analysis were conducted using IBM SPSS Statistics Version 28 and IBM SPSS Amos Graphics Version 28. Means and standard deviations of all variables, as well as correlations between them, are presented in Table 1.

Table 1. Descriptive statistics and pearson product-moment correlation for study variables.

Variable	M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14
1. Trust	3.54	0.76	–
2. Propensity to trust	4.02	0.59	.43**	–
3. Art interest	3.99	1.34	.05	.11	–
4. Affinity for technology interaction	4.22	0.87	.19**	.29**	.14*	–
5. Control beliefs in dealing with technology	3.81	0.64	.15*	.27**	.01	.67**	–
6. Need for cognition	3.63	0.71	.09	.20**	.24**	.48**	.44**	–
7. Confidence in initial decision	0.63	0.22	.05	.14*	.12	.12	.21**	.11	–
8. Accuracy before AI advice	59.82	12.10	.03	−.08	.08	−.05	−.05	−.06	.20**	–
9. Accuracy after AI advice	65.20	11.77	.15*	−.02	.04	−.01	−.08	−.04	.06	.70**	–
10. Weight of advice	0.16	0.18	.18**	.13*	−.01	.10	.03	.03	−.17**	−.15*	.43**	–
11. Weight of correct advice	0.15	0.18	.18**	.12	−.02	.09	.02	.05	−.18**	−.20**	.43**	.98**	–
12. Weight of incorrect advice	0.19	0.25	.15*	.14*	.02	.10	.03	−.01	−.11	.03	.35**	.85**	.74**	–
13. Relative AI reliance	0.31	0.35	.16*	.07	−.01	.05	−.02	−.03	−.12	.00	.58**	.88**	.87**	.75**	–
14. Relative self-reliance	0.67	0.42	−.14*	−.11	−.04	−.04	−.01	.03	.10	.03	−.28**	−.81**	−.72**	−.88**	−.74**	–

Note. N = 244 (n = 241 for relative self-reliance, because of exclusion in case of division by 0). * p < .05, ** p < .01.

4.1.1. Accuracy and reliance patterns

The accuracy of each participant’s first and second responses was determined by considering responses correct if they fell on the correct side of the continuum within a maximum range of 40% away from the slider extremes. This decision was informed by qualitative data, which highlighted a range of undecidedness around the 50% mark. The mean accuracy for the first decision point (percentages of correct choices) lies at 59.82% (SD = 12.10%), ranging from 25% to 95.83% accuracy. The mean accuracy for the second and final decision with AI advice lies at 65.20% (SD = 11.77%), with a range from 33.33% to 91.67% accuracy. Participant’s accuracy on average was improved by 5.38% (SD = 9.18%) with AI advice, with a maximum change of 37.50%. Differences in accuracy changes when receiving correct versus incorrect advice can be viewed in Figure 2, with a positive change of 11.32% for correct advice and a negative change of 17.22% for incorrect advice. Higher accuracy at the second decision point correlated significantly with self-reported trust in the AI (r(242) = .15, p = .020). There was no correlation between art interest and the initial accuracy.

Figure 2. Accuracy in percentages between the different conditions.

Depending on the first decision of the participant and the correctness of the AI advice there are different types of reliance that can be observed. These eight reliance patterns are presented in Table 2 based on the classification of Cabitza et al. (2023). Additionally, one column showcases the percentages of reliance patterns observable in our dataset. Through these classifications we calculated a RAIR of 30.55% (± 35.17) and a RSR of 66.58% (± 42.30), showing that participants were self-relying to a high degree and less likely to decide to follow AI advice. For RSR, calculations were made with 241 instead of 244 cases since the calculation included division by 0 in three cases, which were therefore excluded.

Table 2. Definition of possible reliance patterns based on Cabitza et al. (2023) and percentages of cases in study I.

Human decision	AI advice	Final decision	Reliance pattern	Percentage of cases (%)
0	0	0	Detrimental reliance	10.16
0	0	1	Beneficial under-reliance	0.00
0	1	0	Detrimental self-reliance	20.88
0	1	1	Beneficial over-reliance	9.14
1	0	0	Detrimental over-reliance	3.59
1	0	1	Beneficial self-reliance	7.09
1	1	0	Detrimental under-reliance	0.17
1	1	1	Beneficial reliance	48.98

Note. In the decision and advice columns, 0 signifies an incorrect decision point, and 1 represents a correct decision point.

4.1.2. Model fit

A model aligning with our pre-registered analysis plan was explored; however, the model did not meet the predefined fit criteria and was not accepted. The model fit was estimated based on established fit criteria. The Chi-square value was at 12.79 and is therefore above the acceptable 2.00. The SRMR was at 0.03 (should be below 0.08), CFI was 0.98 (should be over 0.90) and RMSEA was above 0.08 (should be below 0.08) (Hu & Bentler, 1999). Hence, to ensure the robustness of our research, we chose to exclude the structural equation model from our analysis plan. Instead, we opted for specific mediation analyses to gain insights into our proposed hypotheses.

4.1.3. Analysis of appropriate reliance, trust, and confidence

Hypothesis 1 predicted a difference in reliance between receiving correct and incorrect advice from the AI. The t-test between WoA for correct and incorrect advice was significant, t(243) = −3.68, p < .001, d = −0.235. Reliance was greater for incorrect advice (M = 0.187, SD = 0.249) compared to correct advice (M = 0.147, SD = 0.179). H1 is supported, showing a significant difference between reliance behavior when receiving correct versus incorrect AI advice.

Hypothesis 2 predicted that trust influences reliance. It was tested using linear regression analysis. Self-reported trust significantly predicted reliance b = .74, t(242) = 54.25, p = .005. 3% of variance of reliance was explained by self-reported trust, R² = .03, F(1, 242) = 7.90, p = .005. Additional investigations examining the influence of self-reported trust on RAIR and RSR showed a significant positive effect of self-reported trust on RAIR (b = .07, t(242) = 2.51, p = .013) explaining 2% of variance (R² = .02, F(1, 242) = 6.30, p = .013) and a significant negative influence of self-reported trust on RSR (b = −.08, t(242) = −2.24, p = .026) explaining 2% of variance (R² = .02, F(1, 242) = 5.02, p = .026). Hypothesis 2 is supported, showing that trust in the system predicts reliance on the system. Specifically, high trust predicts high relative AI reliance and low relative self-reliance.

Hypothesis 3 stated that the influence of self-reported trust on reliance is moderated by the confidence in the initial decision. The overall model was significant, 9.04% of the variance was predicted, F(3, 240) = 5.99, p < .001. Moderation analysis showed that confidence moderated the effect between self-reported trust and reliance significantly, B = −.18, t(240) = −2.98, p = .003. Simple slopes are depicted in Figure 3. Additional investigations showed that confidence also negatively moderates the influence of self-reported trust on RAIR (B = −.37, t(240) = −2.98, p = .003), while positively influencing the effect of self-reported trust on RSR (B = .32, t(240) = −2.27, p = .024). Hypothesis 3 is supported, showing that confidence moderated the influence of trust on reliance on the system, further leading to lower relative AI reliance and higher relative self-reliance.

Figure 3. Simple slopes of H7.

4.1.4. Mediations for psychological traits

Mediation analyses for the following hypotheses were conducted using the PROCESS macro by Hayes (2013), using ordinary least square regressions with unstandardized path coefficients for total, direct, and indirect effects. Confidence intervals and inferential statistics were computed using bootstrapping with 5,000 samples. If the confidence interval did not include zero, the effects were deemed significant.

Hypotheses 4a, b, and c cumulated into one mediation predicting that the influence of the propensity to trust technology on reliance on the system is mediated by trust in the system. Beta coefficients and significance of the model paths are presented in Figure 4. The relationship of propensity to trust on reliance was fully mediated by self-reported trust, with a significant indirect effect of .020, 95%-CI[.003, .039]. Furthermore, a significant indirect effect of propensity to trust on RAIR was found (B = .040, 95%-CI[.004, .078]), while there was no significant effect for RSR (B = −.042, 95%-CI[-.080, .009]). Hypotheses 4a, b, and c are supported, showing that the influence of the propensity to trust on reliance on the system is mediated by trust in the system, resulting in more AI reliance and less self-reliance.

Figure 4. Mediation model for H4. Note. Reported beta coefficients are significant at *p < 0.05 and **p < 0.01. c is the direct effect without the mediator, while c’ is the effect for the mediation model. The change in significance shows a full mediation.

Hypotheses 5a, b, and c were also analyzed by mediation. It hypothesized that the influence of task expertise, in this case art interest, on reliance on the system is mediated by trust in the system. Art interest had no significant total effect on reliance, B = −.001, p = .889. Art interest further did not significantly predict self-reported trust, B = .028, p = .500. In this mediation model the influence of self-reported trust on reliance did show significant results, B = .043, p = .005. Since the direct effect of art interest on reliance was not significant and art interest did not significantly predict self-reported trust, and yet self-reported trust significantly predicts reliance, this suggests that there is no mediation effect. Art interest furthermore showed no significant direct or indirect effect on RAIR (B = −.002, 95%-CI[-.004, .009]) or RSR (B = −.002, 95%-CI[-.011, .004]). Hypotheses 5a, b, and c are not supported, showing no significant influence of task expertise on any reliance behavior.

Beta coefficients and significance values for the simple mediation for hypotheses 6a, b, and c, whether affinity for technology interaction has an influence on reliance mediated by self-reported trust, are shown in Figure 5. There is no significant effect of affinity for technology interaction on reliance, B = .021, p = .087. The model shows a full mediation with a significant indirect effect of .007, 95%-CI[.001, .016]. Furthermore, affinity for technology interaction showed a significant indirect effect on RAIR (B = .012, 95%-CI[.001, .028]) and RSR (B = −.012, 95%-CI[-.029, −.001]). H6a is supported, showing that affinity for technology interaction influence trust on the system positively. H6b is not supported. The mediation of H6c is supported, showing that the influence of affinity for technology interaction on reliance on the system is mediated by trust in the system, further leading to more relative AI reliance and less relative self-reliance.

Figure 5. Mediation model for H6. Note. Reported beta coefficients are significant at *p < 0.05 and **p < 0.01. c is the direct effect without the mediator, while c’ is the effect for the mediation model.

Hypotheses 7a, b, and c, which postulated that the influence of control beliefs in interacting with technology on reliance on the system is mediated by trust in the system, were again analyzed by a simple mediation. Beta coefficients and significance of the model paths are presented in Figure 6. Control beliefs in interacting with technology did not significantly predict reliance on the system, B = .008, p = .671. The complete model showed a significant full mediation with an indirect effect of .007, 95%-CI[.001, .017]. However, this significant indirect effect could not be replicated for RAIR (B = .013, 95%-CI[-.001, .032]) or RSR (B = −.013, 95%-CI[-.035, .001]). H7b was not supported. For WoA however, H7a and H7c are supported, showing that generally the influence of control beliefs in interacting with technology on reliance on the system is mediated by trust in the system.

Figure 6. Mediation model for H7. Note. Reported beta coefficients are significant at *p < 0.05 and **p < 0.01. c is the direct effect without the mediator, while c’ is the effect for the mediation model.

The mediation for Hypotheses 8a, b, and c hypothesizing that the influence of the need for cognition on reliance on the system is mediated by trust in the system showed no significant total effect of the need for cognition on reliance, B = .009, p = .570. The need for cognition did not significantly predict self-reported trust, B = .092, p = .222. The influence of self-reported trust on reliance was significant, B = .043, p = .005. There were no significant indirect effects found for RAIR (B = .007, 95%-CI[-.005, .021]) and RSR (B = −.008, 95%-CI[-.024, .005]) either. Since the direct effect of need for cognition on reliance was not significant and the need for cognition did not significantly predict self-reported trust, hypotheses 8a, b, and c are not supported. Therefore, no significant influence of the need for cognition on reliance mediated by trust was found.

4.2. Qualitative analysis

The aim of study II was to gain a deeper understanding of the reasoning behind following and not following AI advice, its interplay with trust and the confidence of the participant. The think-aloud protocol voice- and screen-recordings of study II were transcribed including information about the presented art piece, the received AI advice, and the position of the decision slider. In total, nine participants went through 24 classification tasks, resulting in 216 classifications with two decision points. Answers were manually grouped into categories using an inductive coding scheme investigating our areas of interest. All textual components related to trust, incorporation of and thus reliance on AI advice, and confidence were systematically extracted from the data and subsequently categorized in sub-codes, if applicable. Content analysis was conducted using MAXQDA 2022 version 22.7.0, and coding and content analysis processes were carried out manually.

4.2.3. Confidence

The code system for confidence was divided into statements pertaining to “not confident at all” (15 times), “in between confident and not confident” (45 times) and “very confident”. High confidence was indicated in 40 statements, when values from 0 to 20 or 80 to 100 were selected, representing the endpoints of the scale, accompanied by exclamations such as “I know this” (P04) or “This is definitely…” (P01, P05, P07). In contrast, 60 statements were connected to values from 21 to 79 and could either be classified as guesswork or are described explicitly as “gut feeling” (P01), thus reflecting uncertainty and low confidence in the initial decision. Differentiations between low confidence and statements classified as “in between” seemed difficult for participants and only high confidence was clearly identifiable. Therefore, the content analysis indicates that a binary measurement based on Dreiseitl and Binder (2005) is reasonable.

4.2.4. Trust

Trust is an important mediator for reliance on system advice. In addition to self-reported trust, we wanted to gain further insights from the qualitative study into what constitutes trust and distrust in the AI system and its advice during interaction. Trust was coded into statements signifying trust or dis-trust. The hypothesized relationship between trust and confidence is represented in the qualitative data.

Every participant mentioned trusting or distrusting the AI advice at some point throughout the experiment. 22 out of 46 statements about trust were specifically connected to a lack of human confidence in the initial response and, e.g., believing that the AI is better trained in image classification than they are (P04, P06). In the absence of better knowledge, participants decide to trust the AI’s advice. Out of eleven statements where high confidence was connected to contradictory advice from the AI only one led to small adjustment of the response at the second decision point, which is reflected in the weight of advice (“Since I was not a 100% sure and the AI usually agreed with me before, when I was relatively confident, I will adjust to the AI. Not 100% agree but move in the AI’s direction”, P06). The other ten cases stuck to their first response and did not change it according to AI advice. Thus, participants who felt confident in their initial decision or who took the time to identify the factors that specifically led to their decision were more hesitant to follow AI advice. When participants were initially confident in their decision, they clearly communicated their opposition (“No, I do not believe you. This is not modern.”, P02; “AI does not agree with me. Honestly, in this case I really do not care. Because I believe, this is romanticism.”, P04).

Six out of nine participants asked for explanations as to why the AI decided differently or stated that they “would trust the AI more, if it had more than 80% accuracy” (P04). When confronted with advice conflicting with their assessment, seven out of nine participants expressed uncertainty and reduced confidence.

4.2.5. Reliance

To better understand the measurement of WoA and relate it to reliance behavior, the code system consisted of codes indicating agreement or disagreement with the AI advice and actions (change or no change) resulting from it. Disagreements between participants and AI advice did not lead to actively moving the decision slider away from the advice of the AI. Instead, the decision was either adjusted towards the AI advice by a small margin (6.02%), or most times not changed at all (18.98%). Especially in the qualitative data set, it became clear that the final response, which is crucial for the calculation of the WoA, was often not adjusted at all: out of 216 decision situations in total, the decision was unchanged in 71.75% of the cases, independent of whether participants voiced agreement or disagreement with the AI advice. In 114 cases (52.77%) where the initial decision was placed on the same decision-half and thus congruent with the AI advice, participants explicitly stated that they did not need to revise their response, as they already agreed with the AI advice. This was independent of where exactly the slider was placed in the initial decision, which represents confidence in the decision. In comparison, only two of the nine participants integrated congruent advice into their initial response, resulting in an increase in confidence in their decision (“The AI said so too? Then I am completely confident (moving slider position to 100%).”, P02; “Alright, moving it to 100%, since I feel validated by the AI.”, P09). However, this only occurred a total of four times.

5. Discussion

Recent research on AI highlighted that mindlessly relying on decision support systems is not the right approach. Instead, the aim of AI development should be to build for appropriate reliance (Benda et al., 2021; Cabitza et al., 2023; Chiou & Lee, 2023; Ma et al., 2023; Schemmer et al., 2022, 2023; Talone, 2019). While there is plenty research investigating the influence of individual factors on human-computer interaction and system acceptance (Asan et al., 2020; Cabitza et al., 2020; Felmingham et al., 2021; Knop et al., 2022; Tschandl et al., 2020), no study specifically examined how these factors relate to the appropriateness of reliance on AI systems. Understanding the characteristics of different user groups is crucial to be able to design systems that play into the strengths and weaknesses of its userbase and maximize the benefit of human-AI collaboration.

This mixed-method study aimed to investigate how different psychological traits influence both self-reported trust in an AI system as well as behavioral reliance on AI advice to detect factors leading to appropriate reliance. It answered the research questions, in what way the propensity to trust technology, task expertise, technology expertise and need for cognition influence appropriate reliance on AI. The findings of this study offer valuable insights into the factors that affect individuals’ decision to trust AI advice.

In this research a quantitative online study served as the basis for investigation concerning the influence of human factors on the acceptance and integration of AI advice during participants’ decision-making, while also exploring which factors caused relative AI reliance and which led to relative self-reliance. By additionally utilizing the think-aloud method, it was possible to examine the reasoning behind participants’ decision to incorporate or reject AI guidance in their decision-making processes in more detail.

5.1. Reliance, trust, and confidence

Firstly, there was a significant difference between reliance when receiving correct versus incorrect advice. However, this result must be interpreted with caution. While it was expected that participants would follow correct advice more than incorrect advice, the results indicate that reliance was stronger when participants received incorrect advice. This is also representative in the accuracy change of the classification, showing that correct advice enhanced participants accuracy, while incorrect advice lowered the accuracy of participants.

This outcome may be attributed to a variety of factors. Most importantly, our measure of reliance, weight of advice (WoA), measures the adherence to received advice on a percentage level (Yaniv, 2004). When the initial advice of the participant is correct and therefore placed far away from the incorrect advice of the AI system, changing the decision in compliance with the AI leads to a more significant percentile shift. Assuming that participants classified the artwork correctly initially but adhered to incorrect AI advice, would result in a higher WoA compared to participants classifying correctly and then not needing to change their decision, because it matches the AI advice. This is also represented in the percentile shift of decision accuracy in cases of incorrect advice, moving from 51.23% to 34.01% correctness.

To distinguish between correct and incorrect advice a high level of task expertise is required. This conclusion is supported by qualitative insights, pointing to a hesitancy to dispute AI recommendations in areas of limited expertise. Thus, when AI systems are paired with users with low expertise, these users are especially at risk of exhibiting overreliance. Next to low task expertise this may be connected to humans preferring the option with the least amount of cognitive effort (Skitka et al., 1999) and thus tending to accept AI advice.

The analyses revealed that self-reported trust significantly predicts reliance on the AI system, which is consistent with previous research (Chiou & Lee, 2023; Lee & See, 2004). Our study expands these results by looking at the appropriateness of this reliance. Here, it was shown, that a higher level of trust in the system resulted in higher relative AI reliance (RAIR), but lower relative self-reliance (RSR). Participants with a higher level of trust were misled by the AI advice and changed their decision to the wrong classification, not able to appropriately self-rely.

These results reinforce the significance of properly calibrating trust and practicing the ability to critically evaluate the correctness of AI advice received (Schemmer et al., 2022). The qualitative analysis supports these results. Participants talk about their reasons for trusting the AI before adjusting their estimate in compliance with the AI advice, showing behavioral reliance. However, a lack of trust in the system, possibly due to reservations regarding its accuracy in classifications, resulted in disregarding the AI advice, which reflects previous research (Lee & See, 2004). A base level of trust is necessary to enable beneficial human-AI interaction, reflected in the total final accuracy being higher after consideration of AI advice. This is further supported by self-reported trust correlating with final accuracy.

The analysis revealed that confidence plays a moderating role in the relationship between self-reported trust and reliance. This finding was reinforced by the results of the think-aloud study, which indicated that participants tended to heed AI advice when relying on their intuition to make an initial estimate but were less inclined to integrate AI advice into their decision-making processes when they felt confident in their original judgments. The effect of self-reported trust on reliance varies with different levels of confidence. With increased confidence in the initial decision, the relationship between self-reported trust and the reliance on AI advice weakens. Individuals still display trust in the AI system but are less inclined to follow AI advice when they have more confidence in their decision-making abilities (Tschandl et al., 2020).

Interconnection between confidence and following advice has been demonstrated in previous studies (Kopka et al., 2022; Ma et al., 2023; Tschandl et al., 2020). This study further explored the relationship by examining self-reported trust and revealing a negative connection between confidence and RAIR but a positive influence on RSR. Thus, when individuals are too confident in their decision, they fail to recognize when they should rely on AI advice instead. This shows a need for AI systems to present additional information to aid users in reassessing their flawed decisions, specifically when they feel confident in them.

5.2. Psychological traits

The ability to evaluate the correctness of AI advice can be connected to human characteristics making users more or less susceptible to AI recommendations. Therefore, exploring human factors is crucial, as trust in and reliance on a system are highly dependent on human characteristics (Felmingham et al., 2021; Knop et al., 2022). These previous assumptions can partly be supported and extended by the results of this study. We specifically looked into propensity to trust technology, since it was proposed as a crucial factor for trust development (Lee & See, 2004; Schaefer et al., 2016).

Task expertise is an important factor to judge the correctness of advice (Logg et al., 2019) and was instrumentalized as art interest in this study. Affinity for technology interaction and control beliefs in interacting with technology serve as a representation of technology expertise. Need for cognition was looked into as a trait crucial for advice utilization (Waggoner & Kennedy, 2022). The results showed that propensity to trust technology, affinity for technology interaction, and control beliefs in interacting with technology are all influencing factors for reliance on system advice, moderated by trust in the system. The following segments will discuss these results in detail.

Results show that the propensity to trust technology directly affects reliance behavior. This influence becomes even more pronounced when introducing the mediator self-reported trust in the system. Self-reported trust explains a significant portion of the relationship between propensity to trust technology and reliance. These findings indicate that humans with a greater propensity to trust in technology are more likely to be influenced by AI advice, which is further reinforced by their self-reported trust in the particular AI system.

This extends the findings of previous research identifying propensity to trust as a predictor of self-reported trust (Buçinca et al., 2021; Kopka et al., 2022; Sharan & Romano, 2020) by highlighting the mediating function of trust in the system on reliance. Furthermore, the characteristic of propensity to trust in technology is the only human factor that showed differing significant values when comparing receiving correct and incorrect AI advice. Individuals with a higher propensity to trust in technology exhibited a significant influence in following incorrect AI advice, but not for following correct advice. This is further reflected in the propensity to trust indirectly affecting RAIR, but not RSR. This challenges previous assumptions stating that individuals with high propensity to trust are better at distinguishing when to place their trust in automation, since they are better at understanding the capabilities of a system (Lee & See, 2004). Instead, our findings suggest that those who have a high propensity to trust in technology are especially at risk of being misled by incorrect AI advice, represented by over-trusting and not being able to correctly self-rely. This highlights the previously proposed necessity for AI systems to give additional information and training to cultivate appropriate trust and critical evaluation, primarily when used by individuals who are more likely to put their trust in automation (Pop et al., 2015).

Furthermore, this emphasizes the significance of investigating factors influencing the appropriateness of reliance and distinguishing between RAIR and RSR as Schemmer et al., (2022) intended. While consistent reliance on AI, regardless of its correctness, may lead to correlations between RAIR and RSR, our research underscores the crucial distinction between these constructs. The propensity to trust influenced relative AI reliance but did not shape relative self-reliance, emphasizing the need for a nuanced understanding of individual factors influencing reliance dimensions. It is essential to distinguish between RAIR and RSR, particularly as they measure different reliance types, each contributing unique and incremental elements to appropriate reliance.

Regarding the influence of art interest on WoA, RAIR, or RSR, there were no significant findings. Additionally, there was no correlation between art interest and accuracy for the initial estimate. Therefore, we conclude that art interest as a proxy for task expertise is unsuitable because it is not able to capture the expertise necessary for art period classification. This is indicated by art interest not significantly correlating with the initial accuracy of participants. Previous research has emphasized the significance of task expertise as an inhibitor of accepting advice from automation (Vodrahalli et al., 2022) and as a potential factor in distinguishing between correct and incorrect advice (Logg et al., 2019). Furthermore, a lack of experience is associated with over-reliance (Tschandl et al., 2020).

Our results indicate that a mere interest in art is insufficient to accurately classify artworks into their respective art periods due to the depth of knowledge required. The task itself was too difficult, resulting in participants struggling to reject incorrect advice even after initially identifying the correct classification. However, analysis of the qualitative results supports the presumed relationship between task expertise, trust, and reliance. Participants reported trusting and following AI advice due to their lack of knowledge, which aligns with previous research (Yin et al., 2019; Q. Zhang et al., 2022). Furthermore, they reasoned that the AI is better trained in these classifications, identifying the AI as having more expertise in the field and because of that following its advice. This is in line with advice-taking research (Sniezek et al., 2004). It was only overridden by their confidence in their initial decision, previously gained by effortful studying the art piece. Therefore, further research must be conducted on task expertise involving participants knowledgeable in the classification task to obtain a clear understanding of how experts may vary in their interaction with AI advice. After all, beneficial collaborative human-AI decision-making requires complementary expertise (Q. Zhang et al., 2022).

Affinity for technology interaction has been identified in our results as a significant factor in following AI advice, which is further reinforced by self-reported trust. In the same way as the propensity to trust technology, the affinity for technology interaction tends to make people more trusting of AI advice and thus indirectly more reliant on it. The impact of affinity for technology interaction on trust aligns with previous research (Agrawal et al., 2023), which could be expanded showing the mediating effect of trust on behavioral reliance. Additionally, our findings revealed that affinity for technology interaction indirectly leads to increased RAIR but a decrease in RSR.

This research demonstrates that individuals who enjoy interacting with technology are more likely to trust AI in the correct instances, but also prone to over-trusting AI and thus require support to develop appropriate reliance. Affinity for technology interaction served as a stand-in for technology expertise. However, it is important to emphasize, that an affinity implies a benevolence towards technology that results in a greater willingness to trust AI, whereas technological expertise can also include a mastery of technology.

Thus, in addition to measuring individuals’ affinity for technology interaction we also assessed control beliefs in interacting with technology to instrumentalize technology expertise comprehensively. Our analysis revealed that control beliefs do not have a significant direct effect on reliance, RAIR, or RSR. However, control beliefs do predict self-reported trust significantly and, as such, indirectly impact reliance. Thus, if a human perceives themselves to have the ability to control technology, they are more likely to trust this particular system, such as the AI employed in this study, and behave accordingly when integrating AI advice. This compares to earlier research by Sharan and Romano (2020) stating that having control over technology increases trust, given that users feel less anxious when interacting with the system.

The participants’ need for cognition had no significant influence on trust or reliance. This is contrary to the significant change in the decision for highly analytical thinkers when interacting with an algorithm identified by Brennan et al. (2019). Moreover, there was no connection between the need for cognition and trust in the system, as proposed based on research by Waggoner and Kennedy (2022). Potentially, since this study only presented a perceptual classification task and no sequential problem solving and the AI only presented its classification without any additional explanations, the relevance of the need for cognition may have been overestimated in this study.

The qualitative analysis, however, indicated that participants who thoroughly analyzed the picture before coming to a conclusion and therefore potentially exhibited a high need for cognition, asked for additional explanations as to why the AI advice differed from their judgment. It should be acknowledged that the think-aloud study situation made participants more conscious of their thinking process, resulting in a different study atmosphere compared to participating in the study without supervision and reflecting on their thoughts. Therefore, no consistent insights can be inferred regarding the need for cognition.

In summary, this research showed that especially traits related to affinity towards technology, such as the propensity to trust technology, affinity for technology interaction, and control beliefs in interacting with technology have an influence on reliance on AI systems. This connection is further strengthened by trust in the system, which is only damped by confidence in the initial decision. Furthermore, the propensity to trust and the affinity for technology interaction indirectly influence appropriate RAIR positively, while the affinity for technology interaction influences RSR negatively.

Considering especially propensity to trust and technology expertise during the design and development of future AI system will enable the realization of the full range of benefits associated with human-AI interaction, resulting in smoother integration and an end product that is both safe and effective. These factors are especially important, when designing products for target groups enthusiastic about technology usage and interaction. Ensuring that these users are provided with transparent information about the system’s limitations may hinder over-reliance. Supplying explanations within AI systems could lend them the tools to discern when it is appropriate to place a high level of trust (Yang et al., 2020) and when it is necessary to reevaluate and not mindlessly rely on AI advice.

5.3. Limitations and future research

This study has some limitations that should be addressed in future studies. Firstly, art interest as a stand-in for task expertise did not serve the required purpose. There was no significant correlation between art interest and initial decision accuracy, self-reported trust in the AI system, confidence in the initial decision, or the final reliance on AI. As this contradicts previous research highlighting the importance of expertise in accepting or refusing advice received, we conclude that the art interest subscale of the Vienna Art Interest and Art Knowledge Questionnaire scale (Specker et al., 2020) was insufficient in capturing the level of expertise required for our experimental task. Thus, further studies should focus on tasks relevant to user groups with different levels of expertise to make informed inferences regarding the role of task expertise and its influence on self-reported trust as well as reliance, additionally investigating specialist knowledge and potential high-risk decision making.

Furthermore, the required task involvement needed for investigations of the need for cognition was not guaranteed in the quantitative online study. While the qualitative analysis showed a connection between effortfully analyzing the art pieces and consequently relying less on AI advice, need for cognition was not found to be a meaningful predictive factor in the quantitative analysis. This may be due to a lack of cognitive engagement with the presented tasks. This study was based on perceptual classification and thus did not require problem-solving which might be responsible for the seemingly low relevance of need for cognition. Additional investigations incorporating AI explanations might lead to different results and are worth investigating further, since differences in advice integration depending on presented information have been connected to need for cognition in previous studies (Buçinca et al., 2021).

A more in-depth exploration into system adjustments, including diverse forms of explainability, tailored to varying experience levels and personal preferences, could unveil strategies to foster appropriate reliance.

Moreover, it is necessary to debate the appropriateness of WoA as a metric. The think-aloud study showed that participants rarely adjusted their second response. While the sum scores of the quantitative study demonstrated percentage shifts, it is crucial to clarify that, even within the quantitative dataset, instances frequently occurred where the WoA for individual decisions was 0. This occurred when participants deemed no adjustment necessary, especially when their initial decision aligned with the AI advice. Furthermore, negative scores in the WoA, arising from situations in which the participant agreed with the AI advice, moving the slider closer to the endpoint the advice suggested but therein away from the slider position of the AI advice, appeared detrimental to the calculation of sum scores. Contrasting the WoA value, the qualitative analysis demonstrated that participants always interpreted moving the slider away from the AI advice, but closer to the endpoint the advice suggested, as agreement with the AI advice rather than contradicting it. However, additional calculations, after removing any negative values arising from these situations from the dataset, exhibited the same substantial outcomes. Hence, we felt confident in still interpreting the WoA and additionally employed RAIR and RSR as measures to contextualize our findings. WoA, RAIR, and RSR further prove to be valuable tools when applied to decision situations with a clear ground truth. Future investigations, however, may find valuable insights by delving into fuzzier areas of decision-making.

In our approach, we opted to use the position on the decision slider as a proxy for a binary confidence measurement. This decision was informed by insights from qualitative data. We acknowledge, however, that directly querying participants about their confidence could potentially provide a more nuanced measurement, given the nature of participants’ decisions, which were made without explicit consideration of confidence when placing the slider.

6. Conclusion

The aim of this study was to examine different psychological traits and their influence on trust and appropriate reliance on advice provided by an AI-enabled decision support system. Previous studies discussed personality and trust in and acceptance of AI advice (Felmingham et al., 2021; Knop et al., 2022; Tschandl et al., 2020) but failed to connect this to the appropriateness of reliance. This research uncovered that the propensity to trust in technology, affinity for technology interaction, and control beliefs in interacting with technology are crucial factors for trust in the system, which in turn influences the appropriateness of reliance.

These findings need to be considered when designing and developing AI systems for collaborative human-AI decision-making, as the results indicate that individual characteristics influence the level of trust and subsequent integration of AI advice. It is worth noting that excessively trusting a system can be just as harmful as not trusting it enough, as demonstrated by the percentage of participants who exhibited detrimental self-reliance. Consideration must be given to the expected propensity to trust technology and the level of technological expertise among user groups when designing systems that aim to promote suitable degrees of trust and appropriate reliance.

Disclosure statement

The authors report there are no competing interests to declare.

Data availability statement/data deposition

The data of this study can be accessed via the following OSF-Link: https://osf.io/87tpu/

Additional information

Funding

This work was supported by a PhD grant from the DFG Research Training Group 2535 Knowledge- and data-based personalization of medicine at the point of care (WisPerMed), University of Duisburg-Essen, Germany.

Notes on contributors

Alisa Küper

Alisa Küper, a research assistant at the Chair of Social Psychology, Media and Communication of the Department of Human-centered Computing and Cognitive Science, University of Duisburg-Essen, holds a master’s in applied cognitive and media science. As a Ph.D. candidate, her research focus includes human-computer interaction and Explainable AI.

Nicole Krämer

Nicole Krämer is Professor of Social Psychology, Media and Communication at the University of Duisburg-Essen, Germany, and co-speaker of the Research Center “Trustworthy Data Science and Security”. She completed her PhD in Psychology at the University of Cologne in 2001. Dr. Krämer´s research focuses on human-technology-interaction and computer-mediated-communication.