A window into teachers’ thinking

One of the regular commentators on this blog – and a heartfelt critic of standardized testing – sent me an Education Week article that highlights teacher skepticism about the value of such tests. As the article reports,

Most teachers do not believe standardized tests have significant value as measures of student performance, according to a new report published jointly by Scholastic and the Bill & Melinda Gates Foundation.

The report, based on a survey of more than 10,000 public school teachers, finds that only 28 percent of educators see state-required standardized tests as an essential or very important gauge of student achievement. In addition, only 26 percent of teachers say standardized tests are an accurate reflection of what students know.

http://www.edweek.org/tm/articles/2012/03/20/gates.html?tkn=OSCCmfZcp2txYaWrR3rWmk0nS13v1G9z0d%2BA&cmp=clp-sb-teacher

This finding doesn’t surprise me, but I’d note that most colleges and universities with selective admissions haven’t abandoned standardized tests. Also, as the same Education Week article notes,

Many policymakers believe that value-added analysis of tests scores scores offers a critical window on individual teachers’ effectiveness, a position that some research at least tentatively supports.

I assume this is a reference to the recent study that surprised its own authors by finding that value-added teacher assessments (based on test scores) had surprisingly strong, and long term, predictive power. I’m reposting a link to the New York Times article about this story.

http://www.nytimes.com/2012/01/06/education/big-study-links-good-teachers-to-lasting-gain.html?_r=1&pagewanted=all

It’s easy to spin the Education Week account as a clueless teachers or clueless policymakers story, but I really don’t want to go there. It strikes me that the survey  actually reveals a fair amount of agreement, and suggests a lot of room for compromise, on this issue. It’s worth reading on to some of the study’s other findings:

• Teachers generally believe that they should be evaluated and observed, through a variety of methods, more frequently than they are now.

• Large majorities of teachers also favor tying tenure decisions to evaluations of teachers’ effectiveness and having tenure status reassessed at regular intervals.

Overall, according to the report, teachers see ongoing formative assessments, class participation, and performance on class assignments as much more important measures of student learning. At the same time, most teachers (85 percent) agree that their students’ growth over the course of the year should contribute significantly to evaluations of their own performance.

I find that 85% figure very encouraging. It seems to me that the ball has moved on teacher evaluations, from resisting evaluation to trying to make sure evaluations are done right. This – I hope – suggests that we’re increasingly in a place where teachers, administrators, parents and policymakers can work together to find better ways to evaluate both teacher performance and student learning. My guess is that testing will be part of the compromise, and so will tests that do a better job of measuring higher order reasoning skills.

Again, from the same Education Week article:

Teacher-effectiveness authority Charlotte Danielson added that “not a single one of the 21st-century skills can be assessed on a multiple-choice test.” She said that the appeal of standardized test scores is that they “give you a number” but that teaching is too complex to be captured in that way.

Danielson and other panelists suggested, however, that the Common Core State Standards Initiative, adopted by all but four states, may present an opportunity to develop more nuanced types of assessment.

Whether we’ll get good news from such assessments is another – really big – question. Here’s my prediction: if we really do start testing higher order reasoning skills, we’ll encounter some disappointing results and unpleasant surprises. If these cause all of us – teachers, parents, administrators, policymakers – to think about whether we are really preparing students to succeed in the twenty-first century, well, all to the good. But don’t expect it to be pretty.

14 comments

  1. Fred 44

    Mary,

    Just read the New York Times article, and I am sure that this will not surprise you, but before I accept these findings, I would like to see if and how the researchers account for variables in students. Things such as two parent family, religious values taught in the home, whether both parents worked, socioeconomic situation of family and community, education level of parents and other siblings, individual student IQ, etc, etc, etc.

    Good research isolates the variables and provides data to show a clear cause and effect to its conclusions. I am not sure that the researchers can show a cause and effect relationship between quality teachers and teen age pregnancy. I am very confident there are many other variables that will impact which students became parents as teenagers. Intuitively I would not necessarily argue that quality teachers make a difference in students lives. I could share pages of anecdotal information about how teachers have changed students lives, many times for what the did outside of the classroom as much or more than what they did inside the classroom. Research and anecdotal information are two entirely different things.

    I have taught for 27 years, and I would say that some children in my classes failed to learn at the expected standard even with great instruction and my very best efforts. At the same time, other children’s learning far outreached my teaching, because they were gifted students and had highly involved parents. Each child that walks into a classroom (sometimes that is only once a week) is different, and each child will take something different away from the class.

    I must take exception with one of your statements. You said; “It seems to me that the ball has moved on teacher evaluations, from resisting evaluation to trying to make sure evaluations are done right.” Teachers and teachers associations have been accused of “resisting evaluation” for many years. I have worked in my local association for years and I can assure you that at the local state and national level teachers associations have never “resisted evaluations” and have always worked to have evaluations “done right”. Our efforts to have them “done right have been seen as “resistance” by some in the media and politicians. This “resistance” has been used as an excuse not to invite teachers to the discussion. A few states have finally invited teachers to be part of the evaluation discussion and in Utah that has led to a well thought out new evaluation law passed in this last legislative session.

    By the way the new evaluation procedure for Utah will have a “value added component” but not the simplistic year end test; rather it will look at the students improvement over the course of their school years as well as comparisons to other students in schools of similar makeup. This is what can happen when educators are allowed to be part of creating their evaluation process.

    Finally if we are truly concerned about education reform, that reform cannot focus on only area. The education of a child is a three legged stool with three critical stakeholders, the child, the parents, and educators. If we truly want reform, we must stop putting ALL of the responsibility for educating a child on the teacher, and we must hold parents and students accountable as well.

    • Mary McConnell

      I agree with a lot of this. Parents share accountability for student performance. Test-based evaluations should be sophisticated, and tests should never be the only basis for evaluation. And above all, no teacher or student evaluation system will work if teachers aren’t involved in its design and implementation. I’ve said all these things . . . repeatedly.

      I kind of hate to keep fighting over the teacher association issue, since the whole point of my last posting is that we’re all increasingly on the same page. I DO think that the acceptance of some test-based evaluation is a relatively new position for the NEA. Here’s a link to an earlier blog posting on this issue, which cites an Education Week article about last year’s NEA convention, where a change of policy was announced (and by some, disputed.) http://educatingourselves.blogs.deseretnews.com/2011/07/01/another-bite-at-the-nea-position-on-test-based-teacher-evaluations/

      But – let me repeat that this isn’t where I want to go with this discussion. In his excellent book on education reform, The Same Thing Over and Over, Rick Hess makes the following statement:

      “The teachers unions, in particular, seem to have a knack for driving their critics to distraction. As a result, unions become an all-purpose whipping boy, used to excuse inept system management and making it more difficult to distinguish real problems from the litany of complaints. The sensible course is to call unions out on their self-interest and their defense of problematic arrangements, but to do so in a fashion that acknowledges legitimate concerns and does not blame unions for inept school and school district management.

      Too often, criticism of unions can morph into shrill cariacture, and critics can too readily slip into ad hominem, demonizing language that distracts attention from the need to update creaky systems. . . Similarly, the claim that unions are to blame for the plight of our schools has too often allowed superintendents and school boards to skate by, freeing them of responsibility to rework problematic contracts or overhaul sputtering school systems.”

      In this blog, I’ve tried to take this warning to heart.

  2. denverite

    My experience with testing is–those that know something (and thus do well on them) have no problem with them–and even welcome them. Those who know or suspect they don’t know much (and thus will do badly) criticize testing like all get out with all sorts of excuses to hide their own incompetence not only from others, but from themselves.

    It is true that nearly every test in everything has 1 or 2 questions that you kinda wonder about — and those are the ones that make headlines and generate complaints. But the rest of the questions are things that you either ought to know about the given subject –if you actually know the given subject–or you should be able to figure out from the given information if you don’t know it already.

    Testers spend ample amounts of time and money to make sure the test does what it’s supposed to do–and generally speaking, they do very well.

    • Yak_Herder

      “Testers spend ample amounts of time and money to make sure the test does what it’s supposed to do–and generally speaking, they do very well.”

      Going on the assumption that “testers” refers to SAT or ACT kind of organization and quality…

      College entrance exams have a very specific purpose. That purpose is NOT to assess how well a student has absorbed the content outlined by a given set of standards (as do Utah’s CBTs for the core subjects) nor their proficiency with a given set of skills (as do Utah’s competency exams for CTE subjects). The idea that State-developed tests accomplish their objectives as well the college entrance exams achieve theirs is laughable. The State tests in Utah are handicapped first by generally poor standards (Common Core exceptions duly noted) and second by the process used for creating the actual test questions.

      If the increased level of attention Utah’s tests are being given creates the motivation to improve in those areas, we’ve all won. If not, and they become a basis for meaningful student OR teacher evaluation, we all lose.

    • Mary McConnell

      I’ve had similar experience with the AP tests, which, for all their faults, are a good model for how testing can test both knowledge and higher order skills. There has always been a strong correlation with how hard students work in my class and how well they perform on the AP tests. And yes, there are always a few questions that make me shake my head and wonder what was in the water supply that day.

      As we begin trying to devise tests to assess the common standards, I hope that we’ll really try to capture the higher learning skills that students need. I worry a little, frankly, that once tests begin uncovering just how weak many students are at writing coherent essays, interpreting data, etc., we’ll see better, sophisticated tests come under attack as well.

  3. Beth Elmer

    If I must be evaluated by students’ test scores, please let them look at value added. That makes much more sense than simply looking at the student’s raw or scaled scores. Some of my students came in a year behind…I want to see how much progress they have made under my tutelage. Value added does this.

    • Mary McConnell

      I agree, absolutely. This was why the value-added approach was developed in the first place. We need to see if teachers have helped their students move forward . . . wherever they began.

      Back when I was teaching AP Language and Composition, I had a colleague who created a special remedial course for juniors and seniors who had clearly fallen behind in their language arts skills. Many spoke English as a second language. I admired (and still admire) her tremendously for taking on this challenge and working so assiduously to help these kids. I’ll bet her value-added scores would have been higher than mine, and deservedly so.

      • Yak_Herder

        Yup.

        I think we all sense that things have changed since teachers offered instruction to multiple levels of students in one room schoolhouse. Students probably did tend to move on more of an individual pace likely got more one-on-one attention. Whether true or not, those things sound like something we would want for our children today, don’t they?

        Once we figure out a reliable way to measure “value-added” instruction (we haven’t, yet) we’re quickly going to see the fundamental incompatibility between those ideals and the way we have allowed class sizes to get out of control. If your answer to that is to replace the teacher with a computer, okay, but that comes with a package of consequences all its own.

  4. Laura Ann

    I’ve always thought that the best way to see if my students had learned a required subject was to give them a test at the beginning of the year that contained what was needed to be taught that school year. The tests would probably be best given by someone else than the teacher, as that might affect the way a certain subject is taught. I would then know where I needed to improve, but this would take money which the state does not have at this time. It would also show me how affective I had been as an educator. I always want to be better at what I do.

    • Yak_Herder

      The Administration at our school has made it abundantly clear that any teacher who doesn’t pre-test students as a matter of routine would be in a world of hurt in the near future. Any good teacher does it, and not just at the beginning of the year.

      Here’s the rub:
      The door for gamesmanship has opened. If pre-test/post-test data is going to be used to evaluate teachers, how hard to you have to think before you realize that the practice of “sand-bagging” works to your advantage?

      I am NOT saying we shouldn’t pre-test. We should. I am saying that the purpose is easily corrupted when you attach them to personal incentives.

      • Mary McConnell

        Is your concern that teachers will write hard pre-tests and easy post-tests, to improve their “value-added”? I guess I don’t know exactly what “sandbagging” means.

        At any rate, I think I understand your basic concern. I’ve always been very grateful that my principal expected some students to fail AP exams – indeed, recognized that a zero failure rate meant that not enough students were being encouraged to take the risk of attempting a more challenging class. One of the gravest harms inflicted by No Child Left Behind (a law that I like better than most of my readers) is setting up 100% proficiency as a goal. Impossible goals promote cheating, no question about that.

  5. Yak_Herder

    Yes, that would be one form of sandbagging. Another method, harder to identify but also very effective, is simply the way a teacher presents each test. I wouldn’t doubt a shift in the average of 10% could be achieved by just working that angle a bit. The only protection against that sort of stuff is the personal integrity of the teacher. We can butress that line of defense by not tempting them with poor wages and incentives that conflict with one another.

    • Yak_Herder

      Either I’m psychic, or it’s just that obvious.

      From today’s paper:
      deseretnews.com/m/article/765562801

    • Yak_Herder

      And right on cue…

      An article in today’s edition of the Deseret News:
      “Report finds suspicious test scores in schools across nation”

Leave a comment

DeseretNews.com encourages a civil dialogue among its readers. We welcome your thoughtful comments.

*