The Duolingo English Test is a groundbreaking, digital-first, computer-adaptive English language proficiency test intended to support stakeholder admissions decisions at English-medium institutions. The test measures four key constructs for university English language proficiency: Speaking, Writing, Reading, and Listening (SWRL), and is aligned with the Common European Framework of Reference for Languages (CEFR) proficiency levels and descriptors. As a digital-first assessment, the test leverages “human-in-the-loop AI” from end to end for test security, automated item generation, and scoring of test-taker responses. This paper presents a novel theoretical assessment ecosystem for the Duolingo English Test. It is a theoretical representation of language assessment design, measurement, and test security processes, and the test-taker experience factors that contribute to the test validity argument and test impact. The test validity argument is constructed with a digitally-informed chain of inferences that addresses digital affordances applied to the test. The ecosystem is composed of an integrated set of complex frameworks: (1) the Language Assessment Design Framework, (2) the Expanded Evidence-Centered Design Framework, (3) the Computational Psychometrics Framework, and (4) the Test Security Framework. Test-taker experience (TTX) is a test priority throughout the test-taking pipeline, such as low cost, anytime/anywhere, and shorter testing time. The test’s expected impact is aligned with Duolingo’s social mission to lower barriers to education access and offer a secure and delightful test experience, and to provide a valid, fair and reliable test score. The ecosystem leverages principles from assessment theory, computational psychometrics, design, data science, language assessment theory, NLP/AI and machine learning, and test security.

Citation Key: burstein2022