Is educational technology a force for radical change . . . or a force multiplier?

Would-be education reformers love to chide teachers. I’ve chided at times myself. But a recent New York Times article on artificial intelligence-based, computerized essay grading reminded me of one criticism that I find especially irritating . . . all the more since it generally comes from people whose proposals I otherwise support.

http://www.nytimes.com/2013/04/05/science/new-test-for-computers-grading-essays-at-college-level.html?pagewanted=all&_r=0

The criticism goes like this: Teachers and schools aren’t taking advantage of the huge breakthroughs in educational technology because they’re stuck in an old paradigm. To teach effectively with technology, teachers must teach entirely differently. Off with the old, on with the new. Or, for those of you who remember your Sunday school lessons, we teachers are accused of  trying to put old wine in new wineskins. Toss out that old wine . . . however much better it may taste.

Well, I’ll admit that I’m an aging wineskin – or, as I prefer to call myself, a paleoteacher. I’m not convinced that old ways are bad, or that we’re going to improve a lot on, say, Socrates. My old-fashioned prejudices notwithstanding, I think these critics often miss the mark. Technology doesn’t necessarily require teachers to learn an entirely new set of tricks. Often it helps us do what we’ve always done . . . only faster, better, and yes, more cost-effectively.

Since beginning to teach online I have become, almost inadvertently,  a heavy user of computer technology. I like to think that internet-based technology has made me a better teacher. I don’t think it’s made me a radically different teacher, or, more importantly, that it should.

Essay grading is a perfect example of where technology can enhance, but should not replace, traditional teaching skills. Almost everyone involved with education acknowledges that students need to write more. They need also need to write with greater analytical rigor; they need to have not only their argumentation but also their grammar and usage graded rigorously; and, ideally, they need to develop an individual voice and some genuine compositional flair. All of this demands an enormous amount of teacher time, and, realistically, a fair amount of teacher skill.

Can computers help teachers meet this challenge? You bet. I’m currently, for example, in the throes of grading a set of AP U.S. history document-based essays on the Cold War. My students submitted these essays to me over turnitin.com, which enables me both to identify plagiarism (okay, not with 100% reliability) and to grade more thoroughly. Since I’ve graded this topic before, I can turn to a library of detailed comments on the specific question, the College Board’s grading rubricfor this question, and multiple aspects of this historical era. I probably spent 15 hours composing these comments the first time I graded this assignment, but are now they’re available at the click of my mouse. (These comments, which show up right in the essay, are also legible. As any of my pre-online teaching students can attest, handwritten McConnell comments can be tough to deciper, especially after I’ve wielded my red pencil on a dozen or so essays.) Using the same computerized grading program, I can  identify common grammar and usage issues (pronoun/antecedent agreement, tense shift, etc.) and correct them with another click. I can enter individual comments responding to the student’s particular arguments or persistent writing quirks. Finally, and this may be the best feature of all, when students submit a second draft I can quickly (okay, this takes 3 clicks) see precisely what they did or did not change from the first draft.

So yes, I can grade essays both faster and much more thoroughly with computer technology. To use a term from my Pentagon days, the computer is a force multiplier: I can hit faster, harder, and, in my case, from a longer distance

Can artifical intelligence produce these kinds of comments, or correct these kinds of errors? The New York Times article describes a program that claims to do just that:

Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the “send” button when you are done and receiving a grade back instantly, your essay scored by a software program.

And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade.

EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks.

The article goes on to describe some critics’ response:

Skeptics say the automated system is no match for live teachers. One longtime critic, Les Perelman, has drawn national attention several times for putting together nonsense essays that have fooled software grading programs into giving high marks. He has also been highly critical of studies that purport to show that the software compares well to human graders.

Well sure, competent and committed live teachers almost certainly grade more thoroughly and thoughtfully than a computer program. Still, as the article goes on to point out, computer-graded essays are better than no assigned essays at all. Since we have considerable evidence that students are writing fewer papers, especially fewer long papers, maybe better grading technology will produce more, better, and more demanding assignments.

But do we really have to choose? Isn’t it possible that these programs, like turnitin.com, will turn out to be a force multiplier that still demands, well, a strong teaching force?

Artificial intelligence algorithms can apparently make increasingly sophisticated judgments. I’m guessing that now or at least sometime in the near future these programs will be able, for example, to identify whether a paragraph stays more or less on point, and whether an argument is supported by relevant evidence. The  professor who could regularly fool the computer program with calculated nonsense could probably accomplish this precisely because he knows enough about the topic to include all sorts of information that the computer recognized as relevant. A student with that much knowledge in his or her mental hard drive might just as well go ahead and write a good essay.

But I’m skeptical that this system would work well without a teacher checking over and tweaking the grading. Instead of viewing the computer as an alternative grader, why not view it as a grading assistant? Let the program flag as many problems as possible, and save the teacher valuable time. If a computer wanted first crack at my students’ essays, I’d be happy to read whatever report it generated. . . as long as I could still weigh in with a more refined judgment and a final grade. In the same way, I think that once teachers have created a careful rubric and thoughtful comments in an online grading program such as turnitin.com, intelligent paraprofessionals (maybe college students trying to reduce their debt?) could use these tools to grade essays online. Teachers could assign more essays to more students, and maybe even have a little time left over to spend with their families. Everybody would benefit.

So bring on the new wineskins, by all means. But I really don’t see that we should throw out that valuable old wine.

 

3 comments

  1. Stephanie Sawyer

    Human beings are relational creatures (well, most are). While we can use technology to be a force multiplier, we risk ignoring what makes us human. A personal note from teacher to student (if said student reads it) is powerful. I believe that it is the student-teacher relationship that really promotes learning, but I could be wrong. Won’t be the first time!
    If I may digress off-topic, I would say that this whole “redo so you can get a better grade” really smacks of treating grades as a commodity rather than a measure.

    • Mary McConnell

      I agree that human beings are relational – but remember that for today’s students, relationships advance via computer all the time. A personal comment on a computer is still a personal comment, if it responds to that particular student’s essay.

      Rewriting is an absolutely critical element of the writing process. Yes, grades often serve as a necessary motive for rewriting. . . a step students tend to resist. In fact, I DROP a student’s grade if he or she fails to submit a second draft. (I drop it even more if he or she submits the first draft as a second draft – a trick that turnitin.com catches beautifully.)

      For what it’s worth, I’ve found the rewriting process to be self-reinforcing. Sure, students rewrite in part because they want to raise their grade, but they also see that they can improve their performance. Another element of human nature is our desire to better ourselves, and teachers should always tap into that desire.

    • Emily Deckenback

      “If I may digress off-topic, I would say that this whole “redo so you can get a better grade” really smacks of treating grades as a commodity rather than a measure.”

      I think it would be a fair argument to say that grades ARE a commodity, not a measure. Or rather that in education, in many cases they are one and the same.

      Assigning a grade isn’t just about measuring how well a student has learned certain material. It’s also about how hard that student has worked, and what lessons she has learned about how to work hard. Take, for example, two completely different types of B-student. One is brilliant but unfocused, only turns in first or even rough drafts of papers, doesn’t study for tests, and only puts serious effort into assignments she finds interesting. Contrast this student with another, who may have the exact same percentage grade in the class, but has rewritten every assignment, studied for every test, and spent countless extra hours with the teacher trying to figure out a difficult concept. One student is settling for a B, and the other is working hard for it. But the letter grade is the same. What is that grade really measuring? If it’s just a percentage of accuracy on an exam and homework assignments turned in, then it doesn’t teach students to strive for excellence. They have no opportunity to demonstrate that they’ve learned from their mistakes, nor to show that they have the ability to follow through. In life, the rough draft of a paper is never going to be acceptable, and we are doing a disservice to our children if we give them the message that the work they do just goes away after it’s turned in.

      Having a teacher interact with an assignment in a meaningful way – and again, for this generation, the internet is meaningful – tells students that their work doesn’t just disappear into the void. Someone is actually reading it. Someone has an opinion on it, and is taking the time to share that opinion. That sets the stage for the expectation that someday, the boss will actually read the proposal that’s been drafted, or customers will really interact with a message being shared. Many of those interactions will never take place in person, either – but the fact that they are technological in origin doesn’t make them any less important to a business’s bottom line.

Leave a comment

DeseretNews.com encourages a civil dialogue among its readers. We welcome your thoughtful comments.

*