Would-be education reformers love to chide teachers. I’ve chided at times myself. But a recent New York Times article on artificial intelligence-based, computerized essay grading reminded me of one criticism that I find especially irritating . . . all the more since it generally comes from people whose proposals I otherwise support.
The criticism goes like this: Teachers and schools aren’t taking advantage of the huge breakthroughs in educational technology because they’re stuck in an old paradigm. To teach effectively with technology, teachers must teach entirely differently. Off with the old, on with the new. Or, for those of you who remember your Sunday school lessons, we teachers are accused of trying to put old wine in new wineskins. Toss out that old wine . . . however much better it may taste.
Well, I’ll admit that I’m an aging wineskin – or, as I prefer to call myself, a paleoteacher. I’m not convinced that old ways are bad, or that we’re going to improve a lot on, say, Socrates. My old-fashioned prejudices notwithstanding, I think these critics often miss the mark. Technology doesn’t necessarily require teachers to learn an entirely new set of tricks. Often it helps us do what we’ve always done . . . only faster, better, and yes, more cost-effectively.
Since beginning to teach online I have become, almost inadvertently, a heavy user of computer technology. I like to think that internet-based technology has made me a better teacher. I don’t think it’s made me a radically different teacher, or, more importantly, that it should.
Essay grading is a perfect example of where technology can enhance, but should not replace, traditional teaching skills. Almost everyone involved with education acknowledges that students need to write more. They need also need to write with greater analytical rigor; they need to have not only their argumentation but also their grammar and usage graded rigorously; and, ideally, they need to develop an individual voice and some genuine compositional flair. All of this demands an enormous amount of teacher time, and, realistically, a fair amount of teacher skill.
Can computers help teachers meet this challenge? You bet. I’m currently, for example, in the throes of grading a set of AP U.S. history document-based essays on the Cold War. My students submitted these essays to me over turnitin.com, which enables me both to identify plagiarism (okay, not with 100% reliability) and to grade more thoroughly. Since I’ve graded this topic before, I can turn to a library of detailed comments on the specific question, the College Board’s grading rubricfor this question, and multiple aspects of this historical era. I probably spent 15 hours composing these comments the first time I graded this assignment, but are now they’re available at the click of my mouse. (These comments, which show up right in the essay, are also legible. As any of my pre-online teaching students can attest, handwritten McConnell comments can be tough to deciper, especially after I’ve wielded my red pencil on a dozen or so essays.) Using the same computerized grading program, I can identify common grammar and usage issues (pronoun/antecedent agreement, tense shift, etc.) and correct them with another click. I can enter individual comments responding to the student’s particular arguments or persistent writing quirks. Finally, and this may be the best feature of all, when students submit a second draft I can quickly (okay, this takes 3 clicks) see precisely what they did or did not change from the first draft.
So yes, I can grade essays both faster and much more thoroughly with computer technology. To use a term from my Pentagon days, the computer is a force multiplier: I can hit faster, harder, and, in my case, from a longer distance
Can artifical intelligence produce these kinds of comments, or correct these kinds of errors? The New York Times article describes a program that claims to do just that:
Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the “send” button when you are done and receiving a grade back instantly, your essay scored by a software program.
And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade.
EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks.
The article goes on to describe some critics’ response:
Skeptics say the automated system is no match for live teachers. One longtime critic, Les Perelman, has drawn national attention several times for putting together nonsense essays that have fooled software grading programs into giving high marks. He has also been highly critical of studies that purport to show that the software compares well to human graders.
Well sure, competent and committed live teachers almost certainly grade more thoroughly and thoughtfully than a computer program. Still, as the article goes on to point out, computer-graded essays are better than no assigned essays at all. Since we have considerable evidence that students are writing fewer papers, especially fewer long papers, maybe better grading technology will produce more, better, and more demanding assignments.
But do we really have to choose? Isn’t it possible that these programs, like turnitin.com, will turn out to be a force multiplier that still demands, well, a strong teaching force?
Artificial intelligence algorithms can apparently make increasingly sophisticated judgments. I’m guessing that now or at least sometime in the near future these programs will be able, for example, to identify whether a paragraph stays more or less on point, and whether an argument is supported by relevant evidence. The professor who could regularly fool the computer program with calculated nonsense could probably accomplish this precisely because he knows enough about the topic to include all sorts of information that the computer recognized as relevant. A student with that much knowledge in his or her mental hard drive might just as well go ahead and write a good essay.
But I’m skeptical that this system would work well without a teacher checking over and tweaking the grading. Instead of viewing the computer as an alternative grader, why not view it as a grading assistant? Let the program flag as many problems as possible, and save the teacher valuable time. If a computer wanted first crack at my students’ essays, I’d be happy to read whatever report it generated. . . as long as I could still weigh in with a more refined judgment and a final grade. In the same way, I think that once teachers have created a careful rubric and thoughtful comments in an online grading program such as turnitin.com, intelligent paraprofessionals (maybe college students trying to reduce their debt?) could use these tools to grade essays online. Teachers could assign more essays to more students, and maybe even have a little time left over to spend with their families. Everybody would benefit.
So bring on the new wineskins, by all means. But I really don’t see that we should throw out that valuable old wine.