ChatGPT's performance in university admissions tests in mathematics

Abstract

This study comprehensively analyses the performance of the artificial intelligence (AI)-based language model, ChatGPT 4.0, in solving Spanish university admission tests in applied mathematics in social sciences. Using exams taken at public universities in Madrid, we have analysed ChatGPT's answers and concluded that its performance varies significantly across different areas of mathematics, excelling in probability and statistics exercises, but performing significantly worse in algebra and calculus. When compared with students, ChatGPT clearly outperforms them in all areas except algebra. Despite the model's limitations in interpreting complex mathematical ideas, in some cases its responses are positively surprising, indicating its potential as a valuable tool in certain mathematical problem-solving scenarios. Our results suggest significant potential for the introduction of these AI-based systems into the classroom. Despite the progress made, much remains to be explored regarding the efficient integration of chatbots into course development and the subsequent impact on education.