Supporting Instructional Decision Making

About |

Collaborative Research: Supporting Instructional Decision Making: Potential of An Automatically Scored Three-dimensional Assessment System

This project studies the utility of a machine learning-based assessment system for supporting middle school science teachers in making instructional decisions based on automatically generated student reports (AutoRs). The assessments target three-dimensional (3D) science learning by requiring students to integrate scientific practices, crosscutting concepts, and disciplinary core ideas to make sense of phenomena or solve complex problems. Led by collaborators from University of Georgia, Michigan State University, University of Illinois at Chicago, and WestEd, the project team will develop computer scoring algorithms, a suite of AutoRs, and an array of pedagogical content knowledge supports (PCKSs). These products will assist middle school science teachers in the use of 3D assessments, making informative instructional changes, and improve students’ 3D learning. The project will generate knowledge about teachers’ uses of 3D assessments and examine the potential of automatically scored 3D assessments. 


Sponsor: National Science Foundation

Funding: $2,999,996

Timeline: 2021-2025



Our Team

Institutional Leads

Xiaoming Zhai, Principal Investigator, University of Georgia

Xiaoming Zhai is an Assistant Professor of Science Education and an Adjunct Assistant Professor of Artificial Intelligence. His research focuses on two areas: (a) The first looks at using innovative assessment to examine complex constructs in science learning and teaching as one means of supporting teachers, approaching the topic through a variety of methodologies and technologies such as machine learning. Particularly, he is interested in machine learning-based automatic scored assessments in science. (b) His second research area of interest concerns the application of assessment results in science teaching and learning.

Joseph Krajcik, Principal Investigator, Michigan State University

Joe is the Lappan-Phillips Professor of Science Education and Director of the CREATE for STEM Institute at Michigan State University. During his career, Joe has focused on working with science teachers to reform science teaching practices to promote students’ engagement in and learning of science. He focuses on developing and studying the outcomes of project based learning environments that will engage students in making sense of phenomena or finding solutions to problems that learners find important. His hope is that such environments will provide optimal learning situations for diverse learners that will help them develop the capabilities to live fruitful lives.

Yue Yin, Principal Investigator, University of Illinois Chicago

Yue Yin is a professor in the Department of Educational Psychology whose research interests and expertise include assessments, science education, research design, survey design, and applied measurement/statistics. Dr. Yin is passionate about using assessment to help improve instruction and learning. She has conducted research on various assessments, such as performance assessment, concept mapping assessment, formative assessment, computational thinking assessment, and assessments used in makerspace. In her research, she uses learning theory as a foundation, measurement/statistics as tools, to explore and examine ways of using assessments to improve learning and instruction.

Gary Weiser, Principal Investigator, WestEd

Gary Weiser is a Research Associate for the STEM program at WestEd. He researches the efficacy of novel science curricula and new educational tools, while continuing his interests in science assessment and environmental sustainability. Prior to WestEd, Gary received his PhD in Science Education at Teachers College, Columbia University with a focus on developing assessments considerate of crosscutting concepts as part of a joint effort with Indiana University and the Educational Testing Service.


Members

Christopher Harris, Co-Principal Investigator, WestEd

Christopher is Senior Director of Research in Science & Engineering Education at WestEd. He leads large-scale multiyear, multi-institutional research, development, and evaluation projects and has been involved in developing scalable approaches to address the NGSS through assessments and curricula that help teachers make learning engaging, interactive, and accessible for students of diverse backgrounds and abilities. Christopher’s recent work in assessment has focused on developing instructionally-supportive technology-based assessments for the elementary and middle grades that require students to use and apply knowledge to make sense of phenomena and design solutions to problems.

Sania Zahra Zaidi, Co-Principal Investigator, the University of Illinois at Chicago

Sania Zaidi is a Research Assistant Professor at LSRI. Sania’s research interests are in the design and evaluation of learning environments and assessment interventions in K-12 science. Her research focuses particularly on technology-based formative assessments in science. She has worked on designing NGSS-aligned formative assessments for elementary and middle school students. Sania has also worked with the College Board in the redesign of the Advanced Placement Biology assessments for high school students. At LSRI, Sania is part of the Next Generation Science Assessment (NGSA) group and works on projects focussed on both the middle and the upper elementary grade bands. On the NGSA projects, Sania co-leads the development of assessment tasks, the design of rubrics for research use and practitioner use, and conducts classroom-based research on the use of the designed assessment tasks. Sania works closely with researchers and teachers in the design of the assessment products. Sania is also part of the team that is tasked with designing large scale assessments for the state of Illinois. Sania leads the middle school teams in the design of these assessments.

Namsoo Shin, Project Manager, Michigan State University

Namsoo Shin is an Associate Research Professor in Counseling, Educational Psychology & Special education at Michigan State University. Her research focuses on the design and study of learning environments to promote students’ conceptual understanding and complex thinking skills for investigating and resolving issues and problems in real life. She is engaged in complex research and evaluation of various projects and in designing classroom activities, learning technology tools and assessment.

Peng He, Post-doc, Michigan State University

Peng He, Ph.D., is a Research Associate at CREATE for STEM, Michigan State University. His research focuses on crafting science learning environment to engage students in science learning, especially designing, testing, implement curriculum materials, and assessment tasks development and validation using both quantitative and qualitative methods.

Sheng Li, Senior Personnel, University of Georgia

Sheng Li is an assistant professor in the Department of Computer Science, University of Georgia. Dr. Li’s research interest in in areas of Machine learning (unsupervised learning, transfer learning, deep learning) with applications to big data analytics, computer vision, causal inference, user modeling, natural language processing, bioinformatics, etc.

Jim Pellegrino, Senior Personnel, University of Illinois at Chicago

Jim is Liberal Arts and Sciences Distinguished Professor at the Uiversity of Illinois at Chicago and Co-director of UIC’s interdisciplinary Learning Sciences Research Institute. His research and development interests focus on children’s and adult’s thinking and learning and the implications of cognitive research and theory for assessment and instructional practice.

Holly Amerman, Research Assistant and Project Manager, University of Georgia.

Holly is a Ph.D. student in science education, who is interested in K-12 STEM outcomes and in the application of AI and Machine Learning in education.

Xinyu He, Research Assistant at the University of Georgia.

Xinyu is a Ph.D. student in science education, who is interested in classroom assessments.

Ehsan Latif, Research Assistant at the University of Georgia.

Ehsan is a Ph.D. student, who is interested in addressing localization challenges of multi robotic systems and developing technological solutions for STEM education.

Sahrish Salim Panjwani, Research Assistant at the University of Georgia.

Sahrish is a Ph.D. student in science education. She also works as an Instructional Coach supporting teachers in Gwinnett County Public Schools. Her research interests focus on supporting science teachers working with students with disabilities in inclusive classes. 

Katherine (Katy) Nilsen, Research Associate at WestEd

Katy is a Research Associate in Science Education in WestEd’s STEM Program. Katy currently works on several research projects examining the implementation of the Next Generation Science Standards (NGSS).


Publications

Zhai, X., Haudek, K. C., Wilson, C., & Stuhlsatz, M. (2021). A Framework of Construct-Irrelevant Variance for Contextualized Constructed Response Assessment. Frontiers in Education6. https://doi.org/10.3389/feduc.2021.751283 (Click Here for PDF)