The Use of Cognitive Diagnostic Modeling in the Assessment of Computational Thinking

In order to achieve broadening participation in computer science and other careers related to computing, middle school classrooms should provide students opportunities (tasks) to think like a computer scientist. Researchers in computing education promote the idea that programming skill should not be a pre-requisite for students to display computational thinking (CT). Thus, some tasks that aim to deliberately elicit students’ CT competency should be stand-alone tasks rather than coding fluency-oriented tasks. Guided by this approach, this assessment design process began by examining national standards in CT. A Q-matrix (i.e., item–attribute alignment table) was then developed and modified using (a) literature in CT, (b) input from subject-matter experts, and (c) cognitive interviews with a small sample of students. After multiple-choice item prototypes were written, pilot-tested, and revised, 15 of them were finally selected to be administered to 564 students in two middle schools in the Mid-western US. Through cognitive diagnostic modeling, the estimation results yielded mastery classifications or subscores that can be used diagnostically by teachers. The results help teachers facilitate students’ mastery orientations, that is, to address the gap between what students know and what students need to know in order to meet desired learning goals. By equipping teachers with a diagnostic classification based assessment, this research has the capacity to inform instruction which, in turn, will enrich students’ learning experience in CT.