Essay Grading Software

by Evelyn Beck

As an English teacher who has devoted countless nights and weekends, not to mention my formerly keen eyesight, to grading student papers, I have met the news about essay-grading software with a mixture of joy and unease. Is it really possible, I wondered, that a computer might take over a chore that is often, admittedly, pure drudgery? Can a computer really evaluate a student’s writing as carefully and helpfully as I do? And if that is possible, does it mean I’m expendable?

Several companies have developed essay-grading software. Their products include IntelliMetric by Vantage, Intelligent Essay Assessor by Pearson Knowledge Technologies, SAGrader from Qualrus, and Criterion from ETS.

What’s exciting about these developments is how they give students immediate feedback on their writing, suggesting areas of strength and weakness. This can help a student who’s stuck midway through an essay figure out how to push through to the end, it can direct a student on how to polish a rough draft, and it can also reinforce and support a teacher’s analysis of a student’s writing. This software could also work well in courses where the content of the writing is most important, for it offers a holistic look at each student’s essay. For example, if you usually give objective tests, but have considered essay exams or your institution is encouraging writing across the curriculum, this software might be a good solution to the challenge of trying to manage this more demanding kind of assignment. It is also a very useful tool for a student preparing for a standardized test that includes a writing sample, such as the SAT, GRE, or AP. And from an institutional standpoint, these are great labor-saving devices for evaluating placement tests.

Most of these products, though, do not provide word-by-word, line-by-line, or even paragraph-by-paragraph analysis. Most do not suggest specific ideas for revision or development beyond noting overall kinds of omissions or weaknesses. And even though feedback might address the student by name, none of the software, understandably, is personal in the way that an instructor might be. Can you imagine a computer telling a student, “I laughed until my side hurt when I read about your vacation disaster” or “I am so proud of the way your writing has developed this semester?”

The ways that essays are evaluated differ by company. Pearson’s Intelligent Essay Assessor offers scores ranging from 0 (inadequate) to 5 (excellent) in four categories: overall, content, style, and mechanics. Vantage provides feedback in the form of a paragraph that briefly evaluates content, organization, language, and mechanics, along with a score ranging from 1 to 6. For example, a sample essay scored 4 received this comment: “An adequate essay with an organizational pattern that is evident and shows reasonable development of ideas. Word choice is adequate. The writer shows an adequate command of the language, with some errors in usage and/or mechanics.” Qualrus’ SAGrader scores essays from 1 to 20, focusing exclusively on content, specifically the identification and explanation of key terms related to a topic, such as criminal theories of deviance.

From my perspective, the most impressive essay-grading software is Criterion from ETS. Like the other products, this one awards each submitted essay a holistic score, in this case ranging from 1 to 6. What it offers that the others don’t is detailed feedback on a student’s essay, including pointing out individual errors. It also keeps a running tally of the number of papers in which certain kinds of errors occurred—something that I would find particularly helpful as an English instructor working to help students identify areas they need to work on. Unfortunately, to take full advantage of these tools, instructors must assign a topic already built into the software. It’s possible to create your own prompt, though the student will not receive a holistic score when writing in response to an instructor’s original topic.

After viewing the free demos of these products, I’m excited about the extra help available to students in improving their writing. (Of course, this help comes at a cost to the institution.) I’m also, frankly, relieved that these tools are only teacher’s helpers and not a substitute for the real thing, which so far is irreplaceable.