Search
702 resources
-
Yan Zhuang, Qi Liu, Zhenya Huang|Jun 28th, 2022|journalArticleYan Zhuang, Qi Liu, Zhenya HuangJun 28th, 2022
Computerized Adaptive Testing (CAT) refers to an efficient and personalized test mode in online education, aiming to accurately measure student proficiency level on the required subject/domain. The key component of CAT is the "adaptive" question selection algorithm, which automatically selects the best suited question for student based on his/her current estimated proficiency, reducing test length. Existing algorithms rely on some manually designed and pre-fixed informativeness/uncertainty...
-
Qiao Wang|Jun 21st, 2022|journalArticleQiao WangJun 21st, 2022
This study searched for open-source semantic similarity tools and evaluated their effectiveness in automated content scoring of fact-based essays written by English-as-a-Foreign-Language (EFL) learners. Fifty writing samples under a fact-based writing task from an academic English course in a Japanese university were collected and a gold standard was produced by a native expert. A shortlist of carefully selected tools, including InferSent, spaCy, DKPro, ADW, SEMILAR and Latent Semantic...
-
David W. Dorsey, Hillary R. Michaels|Jun 9th, 2022|journalArticleDavid W. Dorsey, Hillary R. MichaelsJun 9th, 2022
In this concluding article of the special issue, we provide an overall discussion and point to future emerging trends in AI that might shape our approach to validity and building validity arguments.
-
Inioluwa Deborah Raji, Peggy Xu, Colleen...|Jun 9th, 2022|preprintInioluwa Deborah Raji, Peggy Xu, Colleen...Jun 9th, 2022
Much attention has focused on algorithmic audits and impact assessments to hold developers and users of algorithmic systems accountable. But existing algorithmic accountability policy approaches have neglected the lessons from non-algorithmic domains: notably, the importance of interventions that allow for the effective participation of third parties. Our paper synthesizes lessons from other fields on how to craft effective systems of external oversight for algorithmic deployments. First, we...
-
Anirudh Goyal, Abram L. Friesen, Andrea ...|May 24th, 2022|preprintAnirudh Goyal, Abram L. Friesen, Andrea ...May 24th, 2022
Most deep reinforcement learning (RL) algorithms distill experience into parametric behavior policies or value functions via gradient updates. While effective, this approach has several disadvantages: (1) it is computationally expensive, (2) it can take many updates to integrate experiences into the parametric model, (3) experiences that are not fully integrated do not appropriately influence the agent's behavior, and (4) behavior is limited by the capacity of the model. In this paper we...
-
Nicol Turner Lee, Samantha Lai|May 17th, 2022|webpageNicol Turner Lee, Samantha LaiMay 17th, 2022
Stakeholders in artificial intelligence must trace back to the roots of the problems, which lie in the lack of diversity in design teams and data that continues to carry on trauma and discrimination of the past, Nicol Turner Lee and Samantha Lai write.
-
Mark D. Shermis|May 15th, 2022|journalArticleMark D. ShermisMay 15th, 2022
One of the challenges of discussing validity arguments for machine scoring of essays centers on the absence of a commonly held definition and theory of good writing. At best, the algorithms attempt to measure select attributes of writing and calibrate them against human ratings with the goal of accurate prediction of scores for new essays. Sometimes these attributes are based on the fundamentals of writing (e.g., fluency), but quite often they are based on locally developed rubrics that may...
-
Now Is the Time to Build a National Data Ecosystem for Materials Science and Chemistry Research DataEva M. Campo, Sadasivan Shankar, Alexand...|Apr 13th, 2022|journalArticleEva M. Campo, Sadasivan Shankar, Alexand...Apr 13th, 2022
-
Norah Almusharraf, Hind Alotaibi|Apr 5th, 2022|journalArticleNorah Almusharraf, Hind AlotaibiApr 5th, 2022
-
Derek Justice|Apr 27th, 2022|conferencePaperDerek JusticeApr 27th, 2022
-
Jill Burstein, Geoffrey T. LaFlair, Anto...|Mar 23rd, 2022|reportJill Burstein, Geoffrey T. LaFlair, Anto...Mar 23rd, 2022
The Duolingo English Test is a groundbreaking, digital-first, computer-adaptive English language proficiency test intended to support stakeholder admissions decisions at English-medium institutions. The test measures four key constructs for university English language proficiency: Speaking, Writing, Reading, and Listening (SWRL), and is aligned with the Common European Framework of Reference for Languages (CEFR) proficiency levels and descriptors. As a digital-first assessment, the test...
-
Riordan Brennan, Debbie Perouli|Mar 21st, 2022|conferencePaperRiordan Brennan, Debbie PerouliMar 21st, 2022
-
Amber Dood, Blair Winograd, Solaire Fink...|Mar 21st, 2022|conferencePaperAmber Dood, Blair Winograd, Solaire Fink...Mar 21st, 2022
-
Mohammadreza Tavakoli, Abdolali Faraji, ...|Mar 21st, 2022|conferencePaperMohammadreza Tavakoli, Abdolali Faraji, ...Mar 21st, 2022
-
Long Ouyang, Jeff Wu, Xu Jiang|Mar 4th, 2022|preprintLong Ouyang, Jeff Wu, Xu JiangMar 4th, 2022
Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users. In this paper, we show an avenue for aligning language models with user intent on a wide range of tasks by fine-tuning with human feedback. Starting with a set of labeler-written prompts and prompts submitted through...
-
Christopher Ormerod|Feb 23rd, 2022|preprintChristopher OrmerodFeb 23rd, 2022
We investigate the effectiveness of ensembles of pretrained transformer-based language models on short answer questions using the Kaggle Automated Short Answer Scoring dataset. We fine-tune a collection of popular small, base, and large pretrained transformer-based language models, and train one feature-base model on the dataset with the aim of testing ensembles of these models. We used an early stopping mechanism and hyperparameter optimization in training. We observe that generally that...
-
Okan Bulut, Alexander MacIntosh, Cole Wa...|Oct 27th, 2022|bookSectionOkan Bulut, Alexander MacIntosh, Cole Wa...Oct 27th, 2022
-
The Promises and Challenges of Artificial Intelligence for Teachers: a Systematic Review of ResearchIsmail Celik, Muhterem Dindar, Hanni Muu...|Jul 27th, 2022|journalArticleIsmail Celik, Muhterem Dindar, Hanni Muu...Jul 27th, 2022
Abstract This study provides an overview of research on teachers’ use of artificial intelligence (AI) applications and machine learning methods to analyze teachers’ data. Our analysis showed that AI offers teachers several opportunities for improved planning (e.g., by defining students’ needs and familiarizing teachers with such needs), implementation (e.g., through immediate feedback and teacher intervention), and assessment (e.g., through automated essay scoring) of their...
-
David W. Dorsey, Hillary R. Michaels|Sep 27th, 2022|journalArticleDavid W. Dorsey, Hillary R. MichaelsSep 27th, 2022
-
David W. Dorsey, Hillary R. Michaels|Sep 27th, 2022|journalArticleDavid W. Dorsey, Hillary R. MichaelsSep 27th, 2022
Abstract We have dramatically advanced our ability to create rich, complex, and effective assessments across a range of uses through technology advancement. Artificial Intelligence (AI) enabled assessments represent one such area of advancement—one that has captured our collective interest and imagination. Scientists and practitioners within the domains of organizational and workforce assessment have increasingly used AI in assessment, and its use is now becoming more common in...