Search
9 resources
-
Nischal Ashok Kumar, Andrew Lan|Feb 10th, 2024|preprintNischal Ashok Kumar, Andrew LanFeb 10th, 2024
In computer science education, test cases are an integral part of programming assignments since they can be used as assessment items to test students' programming knowledge and provide personalized feedback on student-written code. The goal of our work is to propose a fully automated approach for test case generation that can accurately measure student knowledge, which is important for two reasons. First, manually constructing test cases requires expert knowledge and is a labor-intensive...
-
Zichao Wang, Andrew Lan, Richard Baraniu...|Dec 15th, 2021|conferencePaperZichao Wang, Andrew Lan, Richard Baraniu...Dec 15th, 2021
-
Mengxue Zhang, Neil Heffernan, Andrew La...|Jun 1st, 2023|preprintMengxue Zhang, Neil Heffernan, Andrew La...Jun 1st, 2023
Automated scoring of student responses to open-ended questions, including short-answer questions, has great potential to scale to a large number of responses. Recent approaches for automated scoring rely on supervised learning, i.e., training classifiers or fine-tuning language models on a small number of responses with human-provided score labels. However, since scoring is a subjective process, these human scores are noisy and can be highly variable, depending on the scorer. In this paper,...
-
Nigel Fernandez, Alexander Scarlatos, An...|Jul 22nd, 2024|preprintNigel Fernandez, Alexander Scarlatos, An...Jul 22nd, 2024
Automated teaching assistants and chatbots have significant potential to reduce the workload of human instructors, especially for logistics-related question answering, which is important to students yet repetitive for instructors. However, due to privacy concerns, there is a lack of publicly available datasets. We introduce SyllabusQA, an open-source dataset with 63 real course syllabi covering 36 majors, containing 5,078 open-ended course logistics-related question-answer pairs that are...
-
Mengxue Zhang, Neil Heffernan, Andrew La...|Jun 1st, 2023|preprintMengxue Zhang, Neil Heffernan, Andrew La...Jun 1st, 2023
Automated scoring of student responses to open-ended questions, including short-answer questions, has great potential to scale to a large number of responses. Recent approaches for automated scoring rely on supervised learning, i.e., training classifiers or fine-tuning language models on a small number of responses with human-provided score labels. However, since scoring is a subjective process, these human scores are noisy and can be highly variable, depending on the scorer. In this paper,...
-
Jaewook Lee, Digory Smith, Simon Woodhea...|May 1st, 2024|preprintJaewook Lee, Digory Smith, Simon Woodhea...May 1st, 2024
Multiple choice questions (MCQs) are a popular method for evaluating students’ knowledge due to their efficiency in administration and grading. Crafting high-quality math MCQs is a labor-intensive process that requires educators to formulate precise stems and plausible distractors. Recent advances in large language models (LLMs) have sparked interest in automating MCQ creation, but challenges persist in ensuring mathematical accuracy and addressing student errors. This paper introduces a...
-
Hunter McNichols, Wanyong Feng, Jaewook ...|Dec 15th, 2023|journalArticleHunter McNichols, Wanyong Feng, Jaewook ...Dec 15th, 2023
Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable format in both assessments and practices. An important aspect of MCQs is the distractors, i.e., incorrect options that are designed to target specific misconceptions or insufficient knowledge among students. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content...
-
Hunter McNichols, Wanyong Feng, Jaewook ...|Jan 11th, 2024|preprintHunter McNichols, Wanyong Feng, Jaewook ...Jan 11th, 2024
Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable form of assessment. An important aspect of MCQs is the distractors, i.e., incorrect options that are designed to target specific misconceptions or insufficient knowledge among students. To date, the task of crafting high-quality distractors has largely remained a labor-intensive process for teachers and learning content designers, which has limited...
-
Nigel Fernandez, Aritra Ghosh, Naiming L...|Dec 15th, 2022|preprintNigel Fernandez, Aritra Ghosh, Naiming L...Dec 15th, 2022
Automated scoring of open-ended student responses has the potential to significantly reduce human grader effort. Recent advances in automated scoring often leverage textual representations based on pre-trained language models such as BERT and GPT as input to scoring models. Most existing approaches train a separate model for each item/question, which is suitable for scenarios such as essay scoring where items can be quite different from one another. However, these approaches have two...