Empirical Impacts of Independent and Collaborative Training on Task Performance and Improvement in Human-AI Teams

Abstract

With improving AI technology, human-AI teams are becoming increasingly common in research. Within these teams, humans and AI can work collaboratively to complete shared tasks. However, continuing research efforts highlight that humans are ill-prepared to work in human-AI teams. As such, recent efforts have called for training to become a greater focus. This paper reports on an empirical in-person experiment that explored the impact of individual and collaborative task-focused training in human-AI teams. Participants were tasked with either training together or separately prior to collaborative working in a human-AI team. Results show that having humans train together prior to joining a human-AI team can negatively impact their performance and ability to improve at a task when they begin working in a human-AI team. As such, results suggest that human-AI teams need to identify ideal ways to collaboratively train humans on task-related skills in human-AI teams.