Authors: Siyuan Guo and Gautam Biswas
CoSMoAgent is embedded in the Collaborative Computational STEM (C2STEM) learning environment, where students use block-based programming to model realworld scenarios. They connect scientific concepts with computational constructs to model scientific processes. Although this approach supports science learning through construction, students often struggle to implement these constructs computationally to simulate system behavior. CoSMoAgent acts as a knowledgeable peer that asks useful questions, offers hints on appropriate scientific and computational constructs, helps investigate unexpected results, and suggests next steps when students are stuck. This role reflects students’ preference for knowledgeable yet approachable, supportive interaction.
A central motivation for CoSMoAgent is to use generative AI in ways that promote critical thinking and productive struggle while avoiding over-reliance and cognitive offloading. In science and computing, students need opportunities to conduct inquiry, test ideas, and explain why something works. We therefore explored how an AI partner could support this process while keeping students actively involved in thinking and decision-making.
A key development challenge has been enabling the agent to understand where students are struggling. A message such as “this is not moving” provides little information on its own. CoSMoAgent considers students’ models, recent actions, and current task context to provide cognitive and metacognitive guidance that supports learning and task completion. Behind the conversation, specialized agents collaborate to interpret student strategies, assess progress, and retrieve relevant curriculum content. This enables dialogic support that responds to work completed in the learning environment, including difficulties students may not be able to describe.
Through development, classroom deployments, and interviews with students and educators, we learned that students and educators do not always define “helpful” in the same way. Some students valued questions that encouraged closer examination of their models. Others became frustrated when they wanted a direct answer but received a probing question. Educators helped us interpret this tension, including how students’ prior AI experiences shaped their expectations. This raises an ongoing design question: when should the agent intervene, and how much support should it provide? It must offer enough assistance for students to trust and engage with it, while preserving opportunities for them to reason and make decisions independently. Too little assistance can leave students stuck, whereas too much can shift intellectual work to the agent.
Classroom experience also highlights the importance of human relationships. In one documented interaction, a pair of students struggled to explain their difficulty to the agent. Their teacher helped them understand and articulate the problem more clearly and, in doing so, helped them address it. When the pair later interacted with CoSMoAgent using this clearer understanding, the agent provided a more targeted suggestion. The students made a change, tested their model, and saw it work. Such moments illustrate how teachers, peers, and AI can provide distinct forms of support within the same learning process.
In a six-week classroom study involving 33 pairs of high school students, we observed that the agent’s support changed as students progressed. As task mastery increased, CoSMoAgent’s responses shifted from probing understanding to suggesting actions and encouraging students to extend their thinking. Stronger task performance was associated with clearer explanations of computational models and fewer requests for agent support. CoSMoAgent also linked evidence from students’ activities to its decisions and responses, making its feedback more interpretable. The study revealed different patterns of agent use: some students primarily sought confirmation, while others engaged in more detailed discussions. In an exploratory comparison of two pairs, these patterns were associated with broadly similar learning gains.
These experiences have shaped how we think about student learning. We seek to understand whether students make thoughtful decisions, test ideas, explain their reasoning, and recognize when they need help. Our goal is for CoSMoAgent to support these learning behaviors while helping students develop confidence in their ability to work through challenging problems.
