Supporting Instructional Decision Making
Collaborative Research: Supporting Instructional Decision Making: Potential of An Automatically Scored Three-dimensional Assessment System
This project studies the utility of a machine learning-based assessment system for supporting middle school science teachers in making instructional decisions based on automatically generated student reports (AutoRs). The assessments target three-dimensional (3D) science learning by requiring students to integrate scientific practices, crosscutting concepts, and disciplinary core ideas to make sense of phenomena or solve complex problems. Led by collaborators from University of Georgia, Michigan State University, University of Illinois at Chicago, and WestEd, the project team will develop computer scoring algorithms, a suite of AutoRs, and an array of pedagogical content knowledge supports (PCKSs). These products will assist middle school science teachers in the use of 3D assessments, making informative instructional changes, and improve students’ 3D learning. The project will generate knowledge about teachers’ uses of 3D assessments and examine the potential of automatically scored 3D assessments.
Sponsor: National Science Foundation
Xiaoming Zhai is an Assistant Professor of Science Education and an Adjunct Assistant Professor of Artificial Intelligence. His research focuses on two areas: (a) The first looks at using innovative assessment to examine complex constructs in science learning and teaching as one means of supporting teachers, approaching the topic through a variety of methodologies and technologies such as machine learning. Particularly, he is interested in machine learning-based automatic scored assessments in science. (b) His second research area of interest concerns the application of assessment results in science teaching and learning.
Joe is the Lappan-Phillips Professor of Science Education and Director of the CREATE for STEM Institute at Michigan State University. During his career, Joe has focused on working with science teachers to reform science teaching practices to promote students’ engagement in and learning of science. He focuses on developing and studying the outcomes of project based learning environments that will engage students in making sense of phenomena or finding solutions to problems that learners find important. His hope is that such environments will provide optimal learning situations for diverse learners that will help them develop the capabilities to live fruitful lives.
Yue Yin is a professor in the Department of Educational Psychology whose research interests and expertise include assessments, science education, research design, survey design, and applied measurement/statistics. Dr. Yin is passionate about using assessment to help improve instruction and learning. She has conducted research on various assessments, such as performance assessment, concept mapping assessment, formative assessment, computational thinking assessment, and assessments used in makerspace. In her research, she uses learning theory as a foundation, measurement/statistics as tools, to explore and examine ways of using assessments to improve learning and instruction.
Gary Weiser is a Research Associate for the STEM program at WestEd. He researches the efficacy of novel science curricula and new educational tools, while continuing his interests in science assessment and environmental sustainability. Prior to WestEd, Gary received his PhD in Science Education at Teachers College, Columbia University with a focus on developing assessments considerate of crosscutting concepts as part of a joint effort with Indiana University and the Educational Testing Service.
Christopher is Senior Director of Research in Science & Engineering Education at WestEd. He leads large-scale multiyear, multi-institutional research, development, and evaluation projects and has been involved in developing scalable approaches to address the NGSS through assessments and curricula that help teachers make learning engaging, interactive, and accessible for students of diverse backgrounds and abilities. Christopher’s recent work in assessment has focused on developing instructionally-supportive technology-based assessments for the elementary and middle grades that require students to use and apply knowledge to make sense of phenomena and design solutions to problems.
Sania Zaidi is a Research Assistant Professor at LSRI. Sania’s research interests are in the design and evaluation of learning environments and assessment interventions in K-12 science. Her research focuses particularly on technology-based formative assessments in science. She has worked on designing NGSS-aligned formative assessments for elementary and middle school students. Sania has also worked with the College Board in the redesign of the Advanced Placement Biology assessments for high school students. At LSRI, Sania is part of the Next Generation Science Assessment (NGSA) group and works on projects focussed on both the middle and the upper elementary grade bands. On the NGSA projects, Sania co-leads the development of assessment tasks, the design of rubrics for research use and practitioner use, and conducts classroom-based research on the use of the designed assessment tasks. Sania works closely with researchers and teachers in the design of the assessment products. Sania is also part of the team that is tasked with designing large scale assessments for the state of Illinois. Sania leads the middle school teams in the design of these assessments.
Namsoo Shin is an Associate Research Professor in Counseling, Educational Psychology & Special education at Michigan State University. Her research focuses on the design and study of learning environments to promote students’ conceptual understanding and complex thinking skills for investigating and resolving issues and problems in real life. She is engaged in complex research and evaluation of various projects and in designing classroom activities, learning technology tools and assessment.
Peng He, Ph.D., is a Research Associate at CREATE for STEM, Michigan State University. His research focuses on crafting science learning environment to engage students in science learning, especially designing, testing, implement curriculum materials, and assessment tasks development and validation using both quantitative and qualitative methods.
Sheng Li is an assistant professor in the Department of Computer Science, University of Georgia. Dr. Li’s research interest in in areas of Machine learning (unsupervised learning, transfer learning, deep learning) with applications to big data analytics, computer vision, causal inference, user modeling, natural language processing, bioinformatics, etc.
Jim is Liberal Arts and Sciences Distinguished Professor at the Uiversity of Illinois at Chicago and Co-director of UIC’s interdisciplinary Learning Sciences Research Institute. His research and development interests focus on children’s and adult’s thinking and learning and the implications of cognitive research and theory for assessment and instructional practice.
Holly is a Ph.D. student in science education, who is interested in K-12 STEM outcomes and in the application of AI and Machine Learning in education.
Xinyu is a Ph.D. student in science education, who is interested in classroom assessments.
Ehsan is a Ph.D. student, who is interested in addressing localization challenges of multi robotic systems and developing technological solutions for STEM education.
Sahrish is a Ph.D. student in science education. She also works as an Instructional Coach supporting teachers in Gwinnett County Public Schools. Her research interests focus on supporting science teachers working with students with disabilities in inclusive classes.
Katy is a Research Associate in Science Education in WestEd’s STEM Program. Katy currently works on several research projects examining the implementation of the Next Generation Science Standards (NGSS).
Zhai, X., Haudek, K. C., Wilson, C., & Stuhlsatz, M. (2021). A Framework of Construct-Irrelevant Variance for Contextualized Constructed Response Assessment. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.751283 (Click Here for PDF)