Designing and Validating Multiple Assessment Options with Equivalent Difficulty for Introductory Programming

Abstract

Introductory programming (i.e., CS1) is a key subject in any computing degree, as students who do not succeed are significantly more likely to discontinue their studies. As such, maintaining motivation and engagement is critical to increasing student’s chance of success. In this paper we explore the potential for increasing introductory programming student’s motivation and engagement by offering multiple assessment options. These options are designed to cover the same concepts, and be of equivalent difficulty, but are themed differently. Our goal is to increase the likelihood that each student will find at least one option personally relatable and motivating. In order to achieve this, we also investigate methods for estimating the difficulty of a programming task based on an exemplar solution, so that we can ensure the assessment options are equivalent. We found that students reacted positively to being given assessment options, and that there were no negative impacts to student’s learning outcomes based on the assessment theme they chose. We also demonstrate the potential of several static code metrics in estimating the difficulty of introductory programming tasks.