How can we predict K-12 teacher effectiveness?

(Cross-posted at BlogHer)

In the U.S., public educational systems select teacher candidates based on test scores, credentials, and university degrees. But those, says Malcolm Gladwell in a recent New Yorker article, are not good predictors of teacher success. In fact, he writes, there may be no good predictors at all.

Gladwell frames his argument with stories about how two other industries, the National Football League and elite financial advising companies, recruit their professionals. He points out that there may be absolutely no way to tell if a college quarterback will succeed in the NFL--short of letting him play quarterback for an NFL team--because playing college ball barely resembles playing for the NFL. Only time in the league will demonstrate whether a quarterback will succeed.

Gladwell argues that selecting teachers is similarly difficult, that there appears to be little connection between formal teacher preparation regimes (e.g. teaching credential programs, Master's of Education degrees) and the ability to reach all learners in a classroom.

Gladwell says one solution is to change the way teachers are recruited, and he recommends a model embraced by financial advising firm North Star Resource Group. For every thousand people recruiters from North Star interview, they find fewer than 50 they think might succeed. These semifinalists attend a four-month, intensive training camp. Fewer than half of those who were invited to the training camp were then hired as apprentice advisers. Three to four years later, North Star retains 30 to 40 percent of those hired as apprentices.

This model goes against the typical teacher recruitment proceedings, which require little beyond a college degree, a teaching credential, and a passing score on one or more state-mandated teacher certification tests. As Dave Saba points out, "most state teacher tests have pass rates over 95%" and schools of education don't exactly have the reputation of being the most rigorous of our nation's professional training grounds. And Will Wilkinson is wondering why teachers even need college degrees.

Gladwell writes that there is a characteristic that the best teachers demonstrate, what he terms "withitness": the ability to understand and respond to classroom dynamics while still promoting a learning agenda. Teachers who have "withitness" know almost intuitively, in Gladwell's examples, when to let young kids squirm (because it may actually be a marker of learning) and how to shut down student-generated distractions before they snowball and affect the entire class.

The education blogosphere has been broad and mostly supportive of Gladwell's analysis, although there are a few points with which some bloggers take umbrage. For example, Laura Vanderkam presents some interesting data on teachers and standardized tests:

other things being equal, a teacher who scores a 700 on the SAT math section is going to be a more effective teacher than one who scores a 500. The higher score tends to indicate that the teacher is better able to figure things out quickly. This ability to solve problems quickly is a key component of the "withitness" that Gladwell notes is a common attribute with good teachers.

Christopher Sessums wonders how we would fund Gladwell's model of teacher selection:

While I have regularly enjoyed Gladwell's contributions, he seems to gloss over the overall costs associated with his plan. Training all comers will cost more money than the system has. Even if we switched to an apprentice-based pay scale (i.e., paying apprentices a smaller salary while compensating top-tier educators appropriately), finding the money to do so will more than likely outstrip our current salary systems. This is not to say such systems could not be constructed. Instead, I am suggesting further studies need to be considered before we can realistically consider such a move.

Greg Anrig has similar concerns, and offers an alternative model of teacher training:

[T]here are only a few dozen professional quarterbacking jobs in contrast to some 7 million public school teachers, with significant shortages in many cities and subjects. Whatever your position on the appropriate credentialing of teachers, most urban school administrators don't have the luxury of selecting 25 percent of applicants after some sort of closely monitored trial period.

Much more germane to the teaching profession is developing a far more effective, teamwork approach so that instead of relying entirely on the talents of individual teachers to instruct their students in isolation, they can learn from each other on an ongoing basis how to better connect with their students.

J M Holland highlights the differences in salary incentives between football and education. In football, players tend to get paid better when they perform well; the same isn't true for teachers in public schools:

Teaching isn't hierarchical in its demands and schools are not organized so that the same type of practice is needed to be successful in each. The truth of the situation is that in some schools you can teach like a high school quarterback and be fine and in others you have to teach like professional quarterback to be successful. The real difference is that you get paid better in professional football if you are successful whereas in teaching the high school quarterbacks and the professional quarterbacks all get paid the same.

The History Enthusiast believes teachers need to have more than, in Gladwell's words, "a pulse and a college degree":

I did, however, take issue with one statement in particular: "They [reports, new evidence, etc.] suggest that we shouldn’t be raising standards. We should be lowering them, because there is no point in raising standards if standards don’t track with what we care about. Teaching should be open to anyone with a pulse and a college degree—and teachers should be judged after they have started their jobs, not before."

My problem with this is really with the implication here (and elsewhere) that "book smarts" don't really matter in this equation. Now, I'm not saying that every person with book smarts is automatically a good teacher. I'm simply saying that when you consider subjects taught in high school, for instance, teachers should be expected to be book smart. That seems fair to me.

For instance, I have a few education majors in my courses (which fulfills a requirement) who are barely passing. They do not understand basic historical facts. They write at a very basic level with significant grammatical problems. They may end up becoming fabulous teachers who've mastered all the pertinent pedagogical techniques and who can really relate to students. I don't know. But, shouldn't they also be expected to really understand their subject matter? If they are relatively uninformed when it comes to basic historical concepts, shouldn't we find that troublesome? And more importantly, shouldn't ALL teachers be able to write well (i.e. coherently and without noticeable errors)?

Eduwonkette takes Gladwell to task for not looking at the larger contexts in which teachers learn their trade and children learn about the world:

It was surprising to see Gladwell focus so heavily on the potential of the individual player or teacher, given that he just penned a book about the importance of social contexts and chance in producing human greatness. As he put it, "The tallest oak in the forest is the tallest not just because it grew from the hardiest acorn; it is the tallest also because no other trees blocked its sunlight, the soil around it was deep and rich, no rabbit chewed through its bark as a sapling, and no lumberjack cut it down before it matured."

So where's the "forest" for a quarterback or teacher? It's a team. Or a school. Even the most gifted quarterbacks end up with pretty crappy pass completion stats if their teammates consistently miss the ball. And a great quarterback doesn't look so great if he's a poor fit for the team he's playing with. The same goes for teachers. So my fingers are crossed that the Gladwell who recognizes the importance of the environments - not just individuals - wins this match.

My primary problem with the Gladwell article is that he uses standardized test scores as the measure of the "value added" by the teacher. He says, for example, that teacher quality can be objectively measured by the class's aggregated improvement on standardized test scores during the year. Therefore, a teacher who pushes her students to perform at the seventieth instead of the fiftieth percentile on a national standardized test is a better teacher than one who only advances his students ten percentile points.

Both as someone who is a lousy test taker and as an instructor now seeing the effects on college students of the high-stakes testing regimen ushered in by No Child Left Behind, I take issue with the use of standardized tests as measures of learning. Increasingly, teachers "teach to the test," and the result is a focus on content absorption and regurgitation rather than on developing the skills people need to succeed in college and in life: interpretation, analysis, synthesis, and evaluation, to name a few. If standardized tests consistently measured students' ability in these skill sets, then I might find them to be better measures of teacher effectiveness.

Want to learn more? Check out Addofio's thoughtful overview of teacher effectiveness research.

What are your thoughts on teacher effectiveness?