Probable Cause

Developing Warrants for Automated Scoring of Essays

Authored by: David M. Williamson

Handbook of Automated Essay Evaluation

Print publication date:  April  2013
Online publication date:  July  2013

Print ISBN: 9781848729957
eBook ISBN: 9780203122761
Adobe ISBN: 9781136334801

10.4324/9780203122761.ch10

 Download Chapter

 

Abstract

When the idea of scoring an essay with a computer was first introduced (Page, 1968) it was a radical concept, but now such an application of technology almost seems mundane when contrasted with computer systems that can translate text across dozens of languages (http://translate.google.com/), best the top human competitors in the game show Jeopardy (Ferrucci et al., 2010), and perform personal assistant services from verbal commands (http://www.apple.com/iphone/features/siri.html). If computers can perform these kinds of tasks, then why shouldn’t they be able to score a basic academic essay typical of those used in standardized testing programs? The potential benefits of using computers to score essays include improving the quality of scores by eliminating idiosyncratic behaviors of human raters (e.g., halo effects, fatigue, central tendency), reducing the time for score reporting, allowing for immediate performance feedback, and reducing the cost and coordination effort of recruiting and managing human raters. An automated essay scoring 2 (AES) system that provides performance feedback could also encourage broader use of essays in learning and assessment, which would facilitate greater alignment between the kinds of tasks that appear in assessments and the kinds of performances most valued in education. Of course, there are potential drawbacks to automated scoring as well, including the cost of development and validation and uncertainty regarding the extent to which these systems produce scores in the same way as human raters.

 Cite
Search for more...
Back to top

Use of cookies on this website

We use cookies, see our privacy and cookies policy for more information. If you are okay with cookies click below or continue to browse.