Studying the GenAI’s impact on learning and critical thinking: A multi-site field experiment

Abstract

Generative AI (GenAI) tools are transforming higher education, but robust evidence of their pedagogical value is limited. This multi-site field experiment, based on Situated Cognition Theory, explores the impact of GenAI on learning and critical thinking at ten universities and polytechnics. Around 270 students were randomly assigned to an experimental group using licensed ChatGPT for a supervised task, or a control group using traditional resources. Tasks, aligned with local learning outcomes, were evaluated using a rubric assessing critical thinking, accuracy, coherence, and originality. Classroom observations and a post-task questionnaire further enriched the analysis. Preliminary results indicate that ChatGPT enhanced content comprehension and motivation, but many students struggled with evaluating AI reliability and creating effective prompts.