Investigating the Effects of Perceived Teammate Artificiality on Human Performance and Cognition

Abstract

Teammates powered by artificial intelligence (AI) are becoming more prevalent and capable in their abilities as a teammate. While these teammates have great potential in improving team performance, empirical work that explores the impacts of these teammates on the humans they work with is still in its infancy. Thus, this study explores how the inclusion of AI teammates impacts both the performative abilities of human-AI teams in addition to the perceptions those humans form. The current study found that participants perceiving their third teammate as artificial performed worse than those perceiving them as human. Furthermore, these performance differences were significantly moderated by the task's difficulty, with participants in the AI teammate condition significantly outperforming participants perceiving a human teammate in the highest difficulty task, which diverges from previous human-AI teaming literature. Alternatively, no significant effect of perceived teammate artificiality was found on shared mental model similarity. However, it did significantly affect participants' levels of perceived team cognition. Individual performance on medium difficulty maps also mediated the effect of perceived teammate artificiality on perceived team cognition. These results further build on the current understanding of how AI teammates impact perceptions of individual human teammates and how those perceptions subsequently impact their objective performance, which is critical in building more effective AI teammates to incorporate alongside humans.