Scaling and Norming for Automated Essay Scoring

Authored by: Kristin L. K. Koskey , Mark D. Shermis

Handbook of Automated Essay Evaluation

Print publication date:  April  2013
Online publication date:  July  2013

Print ISBN: 9781848729957
eBook ISBN: 9780203122761
Adobe ISBN: 9781136334801

10.4324/9780203122761.ch12

 Download Chapter

 

Abstract

In this chapter, we provide an overview of different methods used in scaling and norming essay scores starting with a general comparison of holistic and analytic rubrics. Next, scales applied for rating the quality of writing samples are reviewed with a focus on standardized writing development scales established for application in automated essay evaluation (AEE) providing for an examinee’s writing ability to be meaningfully tracked over time and across essay prompts. Methods for formation of scores in AEE are then overviewed. Common standard-setting methods are summarized and, finally, differential item functioning methods are discussed. Throughout the chapter, we raise validity issues that continue to exist today, and make recommendations for future work in the effort of producing meaningful scores in the process of scaling and norming essay scores in general and in AES.

 Cite
Search for more...
Back to top

Use of cookies on this website

We are using cookies to provide statistics that help us give you the best experience of our site. You can find out more in our Privacy Policy. By continuing to use the site you are agreeing to our use of cookies.