Search
9 resources
-
Christopher Ormerod|Oct 23rd, 2022|journalArticleChristopher OrmerodOct 23rd, 2022
We introduce a regression-based framework to explore the dependence that global features have on score predictions from pretrained transformer-based language models used for Automated Essay Scoring (AES). We demonstrate that neural networks use approximations of rubric-relevant global features to determine a score prediction. By considering linear models on the hidden states, we can approximate global features and measure their importance to score predictions. This study uses DeBERTa models...
-
Christopher Ormerod|Feb 23rd, 2022|preprintChristopher OrmerodFeb 23rd, 2022
We investigate the effectiveness of ensembles of pretrained transformer-based language models on short answer questions using the Kaggle Automated Short Answer Scoring dataset. We fine-tune a collection of popular small, base, and large pretrained transformer-based language models, and train one feature-base model on the dataset with the aim of testing ensembles of these models. We used an early stopping mechanism and hyperparameter optimization in training. We observe that generally that...
-
Kai North, Christopher Ormerod|Oct 23rd, 2025|conferencePaperKai North, Christopher OrmerodOct 23rd, 2025
-
Christopher Michael Ormerod, Alexander K...|Jul 2nd, 2024|preprintChristopher Michael Ormerod, Alexander K...Jul 2nd, 2024
Current research on generative language models (GLMs) for automated text scoring (ATS) has focused almost exclusively on querying proprietary models via Application Programming Interfaces (APIs). Yet such practices raise issues around transparency and security, and these methods offer little in the way of efficiency or customizability. With the recent proliferation of smaller, open-source models, there is the option to explore GLMs with computers equipped with modest, consumer-grade...
-
Amy Burkhardt, Sherri Woolf, Christopher...|Oct 23rd, 2025|conferencePaperAmy Burkhardt, Sherri Woolf, Christopher...Oct 23rd, 2025
-
Joshua Wilson, Corey Palermo, Paul Deane...|Oct 23rd, 2025|presentationJoshua Wilson, Corey Palermo, Paul Deane...Oct 23rd, 2025
-
Susan Lottridge, Amy Burkhardt, Christop...|journalArticleSusan Lottridge, Amy Burkhardt, Christop...
Every year, millions of middle-school students write argumentative essays that are evaluated against a scoring rubric. However, the scores they receive don’t necessarily offer clear guidance on how to improve their essay or what they’ve done well. With advancements in natural language processing technology, we now have the capability to provide more detailed feedback. At this juncture, we’ve developed an artificial intelligence-supported editing tool to assist students in revising their...
-
The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical ChallengesOkan Bulut, Maggie Beiting-Parrish, Jodi...|Jun 27th, 2024|preprintOkan Bulut, Maggie Beiting-Parrish, Jodi...Jun 27th, 2024
The integration of artificial intelligence (AI) in educational measurement has revolutionized assessment methods, enabling automated scoring, rapid content analysis, and personalized feedback through machine learning and natural language processing. These advancements provide timely, consistent feedback and valuable insights into student performance, thereby enhancing the assessment experience. However, the deployment of AI in education also raises significant ethical concerns regarding...
-
The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical ChallengesOkan Bulut, Maggie Beiting-Parrish, Jodi...|Oct 23rd, 2024|preprintOkan Bulut, Maggie Beiting-Parrish, Jodi...Oct 23rd, 2024
The integration of artificial intelligence (AI) in educational measurement has revolutionized assessment methods, enabling automated scoring, rapid content analysis, and personalized feedback through machine learning and natural language processing. These advancements provide timely, consistent feedback and valuable insights into student performance, thereby enhancing the assessment experience. However, the deployment of AI in education also raises significant ethical concerns regarding...