The evidence disconnect in education policy

Is good evidence winning or losing the battle over education policy? I sat in on an interesting panel discussion on reading last week hosted by the New America Foundation, where the conclusion was, in essence, good evidence isn’t winning often enough. In the case of reading, where there is relatively abundant research on what works, very few school districts and classrooms actually implement tried-and-true methods, the panelists said.

The problem goes beyond reading, of course. The education world is notorious for the relatively weak research base that informs what happens in schools and classrooms. Recently, the Obama administration has tried to encourage more practices based on good research through its Investing in Innovation Fund, where groups and school districts competed for federal money to expand education reform programs that were supported by research.

But shaky or conflicting evidence is still often the norm in many areas of education. Several stories this week highlight this problem: On Monday, Sam Dillon of The New York Times wrote about the debate over class size, which has become shriller as budgets have shrunk and districts are being forced to increase student-teacher ratios. Educators are divided over what the research says on whether small class sizes matter — or matter enough to make up for the extra costs. (See what we’ve written on the issue, here.)

Also in the news, highlights a new report that finds that merit pay — a favorite among education reformers, including Education Secretary Arne Duncan — didn’t work to improve test scores in a New York City experiment, and was actually connected with dropping test scores among middle-school students. A Vanderbilt University study in September 2010 found largely the same thing, that offering middle-school math teachers bonuses up to $15,000 did not produce gains in student test scores.

There are many other examples. Does the preschool research show enough of a lasting benefit for children to justify its (often high) expense? As states decide whether to divert more resources to charter schools, which of the various charter  studies are to be believed — the big national studies that found that charters are mediocre on average, or the smaller studies that have found that in cities, at least, they perform better than their public school counterparts?

These questions over what policies are supported by research — and which research is best — are likely to get more heated as districts prioritize in tight financial times. Will tighter wallets force schools and policymakers to pay more attention to the evidence to ensure that we get better bang for our buck, or will evidence continue to get short shrift?

Comments & Trackbacks (18) | Post a Comment

Bernard Schuster

The news that merit pay didn’t produce gains makes me wonder if it can demoralize teachers with no chance of earning it, so they try less hard with an attitude that they will now lose out more than before.

Its tempting to say the system will get more of what it rewards, thinking that way leads to the conclusion that merit pay is important. However there may also be a “curtailment of range” effect … where the undeserving teachers quit, and the surviving ones figure they all deserve a reward. It’s an interesting paradox.

Bernard Schuster

M. Manos

Education is and will remain a disaster so long as our culture perpertuates the idea that ‘one size fits all’ in the classroom, school, or district level.

Each student is unique and has different strengths and weaknesses to offer. No student will ever be fully served by the notion of ‘comprehensive’ education and standardized tests as a measurement of achievment.

As it stands now, we prohibit the policy of grouping/tracking based on academic achievement levels because students might have their feelings hurt if they realize that they are in the ‘dumb’ class. Really? So education moved to heterogeneous grouping so students of all levels are in the same class. Now their feelings might not be hurt, but I’ll venture to say that there is no teacher who can fully serve the needs of 25 students who are all at different academic levels. STOP THE INSANITY! It is ok to not be an honors student. Let’s get smart and start catering to the needs of all students.

Can’t anyone figure that out and, more importantly, have the balls to say it out loud? Surely I can’t be the only person to have put the pieces of this puzzle together?!?


An article that might just “amen” me.

Jeffrey Pflaum

During the early 70′s my district (NYCDOE) was involved with the M.E.S.(More Effective Schools) project which kept class sizes in inner-city schools to a max of 21. After several years of the program, they found that reading and math scores actually went down–and dropped the program. In my experience, it’s not so much about how many kids are in the classroom, but the educator-the communicator-who stands in front of them. Teaching is about college undergraduate and graduate programs in education and their products–teachers. College programs need to take a hard look at themselves and what they are teaching. They’re missing key strategies, approaches, and most important, fundamental skills for learning how to learn. I took 12 credits in the ITT course (Intensive Teacher Training)at Queens College in NY and knew little about teaching when I first started. I wound up creating my own projects and curricula in reading, writing, thinking, poetry, and Emotional Intelligence. I became a developer-researcher-experimenter in my classrooms and made things up, and found, that the materials compared favorably with the research, especially the concept of intrinsic motivation in reading to inspire adolescents to read and reflect about their reading and reading lives. I learned to think, or better yet, in educational jargon, to use meta-cognition for my teaching and communication skills. How much communication is really going on in the world of education today?

Lauri Lee

We live in a society that is focused on ascribing blame, rather than taking responsibility. Schools have students for an average of 6 – 6.5 hours per day – and students have far less instructional time than that, once you figure in transition times, specials, lunch, and recess (all of which are important!). Research has shown for years that parents have a far greater impact upon their children’s achievement and school success than any other single factor. When will we redesign our educational system to support parents, starting with mandatory human development classes in middle and high schools, and creating schools that reflect a true partnership between families and educators?


Thank you, M. Mantos!!!!!

I think many of us have thought it, but no one listens to teachers…..
Being a teacher that feels beaten down every day in the media, I come to work and take care of my kids as best I can given what I am supposed to teach them.
We are, according to a professor of The University of North Carolina at Greensboro, the 2nd most directed state in the Nation…Only 2nd to Hawaii, some one needs to look at what they tell us to teach and how we are supposed to deliver it.

Teachers are very good at following directions…If we stay in the business for a while we become very good at it…..But some research should be done in what we are being told to do and HOW we are expected to do it. We do what we are told, but too many times our directions are being created by those who have a little or no time in the classroom and are at their present level until the next rung on the ladder opens up.

And for Gods sake quit telling us to teach the test.

Gary Galluzzo

According to the panel, teachers don’t use “proven” reading strategies. According the Ms. Garland, policymakers don’t use evidence to make policy (I happen to be on her side on this one). We can’t decide on whether Head Start is worth the money. People insist on arguing about class size 25 years after a well-designed study (STAR in TN) found that it does matter.

I think the answer to her question about whether evidence matters is: bringing research evidence to a policy debate is akin to bring a penknife to a gunfight.

Don castle

We can measure and research all the variables until we are blue in the face but it will not matter at all until we start a viable and important discussion on the role of the family and parenting and their effects on children’s educational success. It is not teaching it is the expectations of the family that is most important for a student’ success.

George Peternel

The notion that merit pay would result in improved educational outcomes has always puzzled me. The hidden assumption about merit pay that nobody talks about is that teachers are “sandbaggers” who will only expend the time and effort to do their best if their pay envelopes are sweetened. Nonsense.

Being a principal for 30 years taught me a few things about how to get the quality teaching from teachers. I believe that teachers need support from their administrators and peers. They need criticism and staff development that will help them improve, and occasional doses of inspiration. In some cases, they need smaller class sizes or the removal of other obstacles to learning in their schools. And yes, some need to be weeded out before they get tenure.

Education Prof in NM

While I believe, intuitively, that there is some benefit from decreased class sizes, I can live with the results from the research…to a point. But what is being measured? While academic achievement is critical, what about classroom management issues, motivation, and that elusive mentoring relationship that often occurs between teacher and student? These are not measurable, yet they are real. Most teachers agree, for example, that there is a fine line, in terms of classroom management, between a class of, say 22-24 and a class of 29. In fact, anecdotally, many teachers agree that up to 28 is a class and 29+ is, well, more zoo-like. The nature of the classroom, the ability of the students to learn more than academic knowledge from the teacher, the possibility of a student becoming excited about a subject, are all being ignored in the discussion of class size. So, are we just interested in minimum competencies for our children?

Philip Cooper

It seems obvious that if a teacher doesn’t have too many students she can spend more time improving her craft, planning her lessons, and less time grading, logging, etc.

I think most teachers have to make a trade off between correcting student work and preparing lessons. Am I right?

Philip Cooper

One thing that struck me about the University of Chicago study (on 9th grade merit payments shown in Freakonomics,) is that more contact (probably more than ever,) was being made with parents. That might be the cheapest way to “move the needle.”

Tom Nedreberg

Let’s look at research about smoking, second hand smoke, the ozone layer, global warming, or any number of other topics and there will be group of scientists saying something is bad and other group of scientists doing everything they can to debunk the original science. It’s the same with education research. Look at the money that funds the research, look at who’s doing the research, check their political or religious affiliation, and finally see if the research is truely peer reviewed and maybe we’ll have something we could use. But, just because someone says it’s research doesn’t make it so, unless of course I agree with the conclusions.


There is an incredible disconnect and worse yet, the measurement tool for “success in our schools is painfully flawed” Some of my thoughts on this subject can be found in this article.

The Emperor has no clothes -
We can’t afford to continue the path we are on


Policy makers are not making education policy decisions based on the expected results of the policies. They are making those decisions based on their political situation. Can they trade their vote on the education issue for someone else’s vote on another issue? How will their vote on the issue affect their re-election hopes? How it will affect the children is far enough down the list that it rarely gets any attention.

Jim Holifield

One thing that bothers me about merit pay is that it assumes that teachers/administrators are somehow shirking their duties and that they won’t perform well without an extra incentive. I don’t know of one educator that says “You know, I would like to do better by my students but I just don’t feel enough incentive to do so…” It’s simplistic and downright insulting to the profession.

Erika Burton

It is not surprising that implementing what policy makers and not educators offer to close the academic achievement gap does not work. We need educators making decision based on being in the trenches and seeing the needs of students to properly rebuild our education system in the US. We need to develop thinkers and not test takers. Other countries are passing up by due to our incessant focus on testing in reading and math and not focusing on applying them to real situations and making learning meaningful.
Erika Burton, Ph.D.
Stepping Stones Together, Founder
Empowering parental involvement in early literacy programs

Mandy Summers

Why do students in the suburbs get a better education? I have been a substitute in 4 different districts of differing (since I haven’t found a job yet in this economy), and I have found that the teachers mostly teach the same. The difference? The parental support. Most of the inner city students’ parents never finished high school, most of the suburb students’ parents finished college! There need to be programs to teach parents how to parent and how to encourage their child! It all starts in the home. And not just when the students are 3 and 4, like in Headstart. It needs to be continuous.

Your email is never published nor shared.