Towards a typology of prompts for human-AI interaction: Mapping intent and complexity with lay users

Abstract

Large Language Models enable broader access to AI for end users with diverse backgrounds and varying levels of expertise. A significant and growing proportion of the population is interacting with highly sophisticated technology. This opens questions related to the nature, dynamics, and effects of such interactions, specially among lay users. To understand these dynamics, we must first identify the types of interactions that may take place according to the user’s intent and their complexity, since LLMs allow unprecedented freedom to be used in different contexts. To date, no categorisation of prompts with comparable complexity levels has been developed from the user's perspective, avoiding confounding variables when studying human-AI interactions. To address this gap in prompts, we applied three sequential methodological approaches. First, we explored prompt categories and complexity levels through iterative queries with ChatGPT. The prompts were written by GPT itself. Second, we analysed these textual data using a thematic qualitative approach and curated a pre-set of 34 prompts with comparable complexity. Prompts were classified into two main categories: “task-oriented” and “reflexive”, with two additional controls: “both” and “none”. Third, we conducted a validation study with 28 lay users from different countries through an online survey. “Task-oriented” prompts achieved a mean category confirmation rate of 62% (Max = 82%), and “reflexive” prompts reached 52% (Max = 71%). Complexity levels averaged near the central point of the scale (M =4.10). A smaller set of 12 prompts with at least 60% of category agreement was obtained. This study lays an empirical foundation for investigating complexitycomparable types of interaction between lay users and LLM-powered conversational agents. Ultimately, this study contributes to advancing research in human-AI interaction, offering a validated set of prompts suitable for investigating trust dynamics, emotional responses, and other key constructs in HCI.