Scaling and Norming for Automated Essay Scoring

Authored by: Kristin L. K. Koskey , Mark D. Shermis

Handbook of Automated Essay Evaluation

Print publication date:  April  2013
Online publication date:  July  2013

Print ISBN: 9781848729957
eBook ISBN: 9780203122761
Adobe ISBN: 9781136334801

10.4324/9780203122761.ch12

 Download Chapter

 

Abstract

In this chapter, we provide an overview of different methods used in scaling and norming essay scores starting with a general comparison of holistic and analytic rubrics. Next, scales applied for rating the quality of writing samples are reviewed with a focus on standardized writing development scales established for application in automated essay evaluation (AEE) providing for an examinee’s writing ability to be meaningfully tracked over time and across essay prompts. Methods for formation of scores in AEE are then overviewed. Common standard-setting methods are summarized and, finally, differential item functioning methods are discussed. Throughout the chapter, we raise validity issues that continue to exist today, and make recommendations for future work in the effort of producing meaningful scores in the process of scaling and norming essay scores in general and in AES.

 Cite
Search for more...
Back to top

Use of cookies on this website

We use cookies, see our privacy and cookies policy for more information. If you are okay with cookies click below or continue to browse.