Probable Cause

Developing Warrants for Automated Scoring of Essays

Authored by: David M. Williamson

Handbook of Automated Essay Evaluation

Print publication date:  April  2013
Online publication date:  July  2013

Print ISBN: 9781848729957
eBook ISBN: 9780203122761
Adobe ISBN: 9781136334801


 Download Chapter



When the idea of scoring an essay with a computer was first introduced (Page, 1968) it was a radical concept, but now such an application of technology almost seems mundane when contrasted with computer systems that can translate text across dozens of languages (, best the top human competitors in the game show Jeopardy (Ferrucci et al., 2010), and perform personal assistant services from verbal commands ( If computers can perform these kinds of tasks, then why shouldn’t they be able to score a basic academic essay typical of those used in standardized testing programs? The potential benefits of using computers to score essays include improving the quality of scores by eliminating idiosyncratic behaviors of human raters (e.g., halo effects, fatigue, central tendency), reducing the time for score reporting, allowing for immediate performance feedback, and reducing the cost and coordination effort of recruiting and managing human raters. An automated essay scoring 2 (AES) system that provides performance feedback could also encourage broader use of essays in learning and assessment, which would facilitate greater alignment between the kinds of tasks that appear in assessments and the kinds of performances most valued in education. Of course, there are potential drawbacks to automated scoring as well, including the cost of development and validation and uncertainty regarding the extent to which these systems produce scores in the same way as human raters.

Search for more...
Back to top

Use of cookies on this website

We are using cookies to provide statistics that help us give you the best experience of our site. You can find out more in our Privacy Policy. By continuing to use the site you are agreeing to our use of cookies.