On this week’s Education Gadfly Show podcast, David Houston, an assistant professor at George Mason University, joins Mike and David to discuss how presidents polarize voters when they weigh in on education debates. Then, on the Research Minute, Amber examines a new study investigating whether performance incentives improve teacher skills and so the academic growth of their students.
Recommended content:
“Polling data: Presidents split the public on schools” —Kevin Mahnken, The 74
“Let’s talk about bad teachers” —Michael Petrilli, Fordham Institute
David Houston and Alyssa Barone, “How the engagement of high-profile partisan officials affects education politics, public opinion, and polarization,” Annenberg Institute at Brown University (March 2024).
Eric Taylor, “Employee evaluation and skill investments: Evidence from public school teachers,” Annenberg Institute at Brown University (May 2024).
Feedback Welcome: Have ideas for improving our podcast? Send them to Daniel Buck at dbuck@fordhaminstitute.org.
This transcript was created using AI software.
Michael Petrilli:
Welcome to the Education Gadfly Show. I'm your host Mike Petrilli of the Thomas B. Fordham Institute. Today, David Houston, an assistant professor at George Mason University, joins us to discuss how presidents polarized voters when they weigh in on education policy debates. Then on the research minute, Amber reports on a new study investigating whether performance incentives improve teacher skills and their value added contributions to students. All this on the Education Gadfly Show.
Hello. This is your host, Mike Petri of the Thomas B. Fordham Institute here at the Education Gadfly Show and online at fordhaminstitute.org. And now please welcome our special guest for this week, David Houston. David, welcome back to the show.
David Houston:
Thank you so much for inviting me, Michael.
Michael Petrilli:
Yeah, also joining us. The other David, my co-host, David Griffith.
David Griffith:
Hey, Mike. Always a pleasure.
Michael Petrilli:
Yeah, well to David's this week, we'll see if we can get through the complications of that. I think we should be able to. David Houston is an assistant professor of education at George Mason University. David has a new paper out that looks at a polling from the Education Next poll, which was something David has been involved with for many years as well. So let's talk all about that on Ed Reform update. Okay, David, so tell us about this new paper you've got, and I believe it is about the role that presidents play in shaping public opinion on education, which is perhaps not always a good role. Go for it.
David Houston:
Yeah, I'd be happy to chat a little bit about the paper. So a little bit of the origin story for a few years there, I was the survey director of the Education next poll at Harvard University. And for the two years prior to that, I was a postdoctoral fellow there in that group. And the next poll has been in, it was run annually from 2007 to 2022. And some of my predecessors liked to insert what they referred to as these presidential endorsement experiments. This was the ED Next Polls and nationally representative survey. We'd ask folks about their opinions about charter schools and testing and spending and teacher salaries and things all along that type. But some folks, we would randomly assign them to learn. As it turns out, Barack Obama supports charter schools before we ask them to give their opinions on charter schools, for example.
And so this is a really nice way of identifying what are the effects of information about a president's policy, endorsements on support or opposition to a given policy issue. Now every year we tend to report out the results of these experiments, but this is an attempt to gather all of that information together and see if we can identify some larger patterns. What happens when presidents weigh in on K 12 education debates? Are they able to shift public opinion in the direction of their position? Do they push people away? We wanted to get under the hood and see what's happening here.
Michael Petrilli:
No, that's great. Okay, so tell us, I guess I already hinted at this, that the results are not always great and that the paper is about polarization. It sounds like in most cases, president takes a position and you tell a survey respondent about that position before asking them about their view. And what if they're of the same party? They tend to agree. If they're the opposite party, they tend to disagree. Is that how it goes?
David Houston:
That's a really good description of it. If you were to look at the average effect, what's the effect of learning that Donald Trump opposes the common core on positions about the common core? If you look at the average effect, it's one big fat zero. A whole lot of nothing. Presidents across many issues across many years across. Many presidents tend to be relatively unsuccessful at shifting the public as a whole towards their position. But that null average effect conceals some really important heterogeneity. And it's that polarization that you're talking about. Presidents tend to bring along members of their own party. They tend to push away members of the opposing party, and the really reliable result is greater polarization among the public on that issue.
Michael Petrilli:
And when we say presidents, again, this goes back to 2007, so we've really got a sample of three. We've got Obama, Trump, and Biden. It sounds like for Trump and Biden, they are taking positions that are generally traditional positions for Republicans or Democrats to take. Trump was anti-Common core, which unfortunately is the typical Republican position, or I assume pro school choice, pro charter schools, Biden very close with the teachers unions in favor of spending more money. So it's not surprising to me that conservatives like hearing Trump support something that's already conservative, that maybe they're already inclined to support and vice versa with Democrats. But tell us about the Obama counter example here,
David Houston:
And you've put your finger on the interesting exception here, but it's an exception in name only. So what my co-author Alyssa Baroni, a brilliant PhD student here at Mason, what we found is that there was one deviation from the larger pattern, and that's when presidents endorse a policy that's more traditionally favored by members of the other party. And the cases that we have here are all instances in which Obama supported centrist or center right positions like charter schools or evaluating teachers based in part on student test score growth or strengthening the rigor of academic standards, testing, things like that, positions that were more popular among Republicans in the control group than their counterparts, their democratic counterparts. What happens when folks learn about what we refer to as these cross-party cues? Well, as it turns out, the same phenomenon happens. It pushes members of the opposing party away. In fact, when Republicans learn that Obama supports charter schools, they become a little less supportive of charter schools, but it pulls members of the president's own party with them that Democrats become more supportive of charter schools. And because the distribution of opinion in the control group is reversed, this has some non-trivial depolarizing effects. And the shift toward the policy is a little bit larger than the countervailing push away from it. So that on average it does nudge aggregate opinion modestly in the direction of the position when a president takes an ideologically unexpected stance.
Michael Petrilli:
And look, I would think we see this. I would think on other issues too. You see Trump, for example, taking this anti-free trade position, which is at odds with the traditional Republican position, and he seems to have dragged a lot of Republican voters along with them.
David Houston:
Yes, that's exactly right. So this is not unique to education. And in fact, this story I think would be really unsurprising to many folks in political scientists who study American political behavior. It's pretty consistent with what we see in a lot of policy domains, and it's just worth bringing it to the surface and finding a real clear, crisp example of it in K 12 education where sometimes, or at least historically speaking, the party lines of what is conventionally the democratic position and what is conventionally the Republican position gets a little muddied over time. And so there's an opportunity to explore this phenomenon in some depth and some subtlety here.
Michael Petrilli:
Okay. David Griffith, any thoughts about this kind of makes sense, right?
David Griffith:
It does kind of make sense. I actually have a couple questions. I think my first question, and these may be beyond the scope of your study, but my first question is do you think it's fair to consider these effects upper bounds for the real world effects? In other words, I mean, in practice everyone does not know that Barack Obama supports charter schools. So presumably his actual effects on public opinion was smaller than the effects in the poll. Is that fair, do you think?
David Houston:
Oh, it's a great question and it's a pervasive question with survey experiments writ large. So how generalizable are the effects? And there's a couple of reasons to take the generalizability of this paper really seriously. And I think there's a couple of reasons why a grain of salt may be valuable. So one of the common critiques of survey experiments is that the survey experimental context is itself artificial, that answering questions in an online survey is just not the same as decisions in real life. And so that we shouldn't expect to see the magnitude of the effects generalized from this artificial environment to a more realistic environment. I don't think that critique applies particularly well to this study because our object of inquiry is public opinion and the way that you measure public opinion is in these surveys. And so the context is in fact, as realistic as we want it to be, but the critique is to make it more of a steelman critique, not a straw man critique still holds some weight here.
So when folks encounter information in a realistic media environment, it's not not only learning about Trump's position, they're also learning about the counter position from his political opponents. In some sense, that may understate the effects that we're seeing in this study because in most cases, you're getting sort of party aligned information to find out that Donald Trump opposes the common core. For example, might be very consistent with, let's say we're in, well, there was a time when supporters for the common core decided to take a low profile. So maybe another example would be better here, let's talk about support for teachers unions, for example. So you would find out that Donald Trump was opposed to the teachers' unions, but in the same breadth you'd find out that Joe Biden was in support of teachers' unions, and that might sort of further strengthen these partisan gravitating towards the position of your party or being repelled by the position of the other party.
Michael Petrilli:
Yeah, no, and look, I think to David Griffith's question about does it matter? I mean, maybe public opinion matters, maybe it doesn't, but I think we saw real life effects back in the Trump administration when he polarized the country around charter schools and school choice. He made it very hard for democratic leaders at the state and local level to support charter schools. And so that was, I think, an example where that's both a direct impact, maybe on political elites, but maybe indirectly through public opinion as well.
David Houston:
And I think another really vivid example of that would be the first year of the pandemic where Trump came out pretty early and pretty hard on the desire to rapidly reopen schools for in-person instruction. And I think that really made it very difficult for Democrats to come out in favor of the same thing. I think their democratic base became really wary of supporting something that Trump supported. Yeah,
Michael Petrilli:
That's a great, that is a really great point. And boy did that have a real world impact, unfortunately a terrible one. Alright, well, David Houston, this is great stuff. We really appreciate you coming on and talking about it, folks. You can see all the information in the show notes, but you can also find it if you Google how the engagement of high profile partisan officials affects education, politics, public opinion, and polarization. Again, David Houston, assistant professor of education at George Mason University. David, thanks for joining us. Thanks very much, Michael. Alright, now it's time for everyone's favorite Amber's research minute. Amber, welcome back to the show.
Amber Northern:
Thanks, Mike.
Michael Petrilli:
It was another beautiful weekend in the Mid-Atlantic. Did you do anything fun?
Amber Northern:
It was a beautiful day and I spent it walking mind you not running a 5K to raise some money for colon cancer prevention. So I feel really good about that. Maybe our listers know we've been touched on our family by that heinous disease. I lost my husband, it's been nearly four years, but I had a nice group of friends that gathered and we raised 3000 and counting. So pretty good for our first year. We'll be trying to get that up in future years. But yeah, it was a beautiful day and hey Mike, we won the best costume award and I literally won a Japanese maple tree for our boxer shorts that were very inventive, shall I say
Michael Petrilli:
That warms my Fordham heart right there. We love costumes here at Fordham. That's great. You're not allowed to tell us more about the boxer shorts.
Amber Northern:
Well, I mean, mine had donuts because my husband loved donuts. My friends had black dogs, we had a black poodle. We had some pancakes because they liked pancakes. I don't know, it was just silly stuff on the boxers that reminded people of my husband and apparently it was a hit with the organizers of the run.
Michael Petrilli:
Well, that's great. I love it. That's fantastic. That's great that you did that. Alright, well what do you have for us this week? On the research front,
Amber Northern:
We have a new study from NBRs, Eric Taylor at Harvard. He's looking at how Tennessee's tenure policies went into effect in 2012. So we got to look backwards and how they impacted various elements of the evaluation and tenure program on job performance. So they're looking at the anticipatory effects, the what happened when the changes started at the end of the time period by when teachers would be impacted by those changes given how long they'd been teaching, and then what happened when they implemented all these new measures to evaluate teachers. So the new policy said instead of three years, you now are eligible for tenure after five years and to earn it, their annual level of effectiveness, I'm going to talk about that a bit more later, must be above or significantly above the cutoff, which is another cutoff that they figured out based on these measures in both year four and five.
So teachers who missed this cutoff can continue on a probationary contract in year six and beyond, but they only earn tenure after scoring a four greater than a four out of five on this cutoff in two consecutive years. But when you dig into the details in practice, few teachers are let go for not getting tenure. Alright. And then the sample ELA and math teachers are included grades four through eight since we've got annual tests in those grades and it's teachers who in the early years of their career, so they have to be in years one through seven, and that's when these incentives are most salient and impactful. They use difference in differences designs. Couple different of them depending on what outcome they're measuring. They're using various fixed effects where depending on what's examined, the treated teachers, they were the ones hired in 2010 or 11, had experienced some new element of this reform, whether it had been the announcement of the new tenure rules or the enactment of these new performance measures or the start of the tenure award process and so on.
And they spent several pages saying that the parallel trends assumption is satisfied. The comparison group were teachers hired in 2000 or nine or earlier before these evaluations started in 2012. And the treatment effect is growth in performance between the first and second year or second and third year of a teacher's career. But this is important. It's additional growth on top of the typical growth between those years that they're looking for. And the typical growth is the counterfactual estimate from the comparison group. So that's how they figure out how much more growth they need to be seeing based on what the comparison group has done That was teaching before 2012. The new system uses classroom observation data, value added growth, and an additional achievement or passing score from the state tests or another commercially available tests. They're looking at all three of those things. Then they'd figure out a way.
They standardize the scores on those things, they map 'em onto a five point scale and then they combine those different scores to come up with a teacher's LOE or teacher's level of effectiveness. That was a very quick summary of a lot of stuff, but I hope you get the gist. The results teacher performance began improving before any rewards or consequences in anticipation of the new policy. The new program, increased average value added by 0.05, standard deviation among teachers who were anticipating future enactment of the policy math and ELA teachers both improved though math teachers improved more. They're able to show that selection effects is not the driver with any of these findings, basically because there's a lot of discussion around selection and how it couldn't be the driver, the linking, and then they look further out because the linking of these performance measures to earning tenure begins in year four.
But they actually see little improvement in year four because the improvements have started earlier. Like I said, in anticipation of the policy, nearly two thirds of teachers earn tenure after year five and their scores in year six no longer count for tenure. So they say, okay, let's see what happens in year six. But they find in year six that the value added gains persist and do not decline. So they think, okay, well whatever they're learning or these skills that happened or continuing to occur and they're not seeing a big nose dive in year six, teachers keep performing in higher levels. Newly tenured teachers improve even further between year five and six by about 0.05 standard deviation. So they keep climbing a little bit in those latter years, and then they look finally at the teachers who didn't get tenure about a third of them, and they looked at 'em and they say, well, they continued in year six on this probational license, but the actual rules, these new tenure rules did not increase or decrease attrition. So you might think, well, maybe they decided they were going to leave because they haven't managed to get tenure at year six, but they don't see a pattern either way. So they say, this new policy didn't impact attrition. That is what I've got for us this week.
Michael Petrilli:
Okay. Interesting. I mean, it seems like just knowing that you are being evaluated might be having an effect even whether there's formal consequences tied to it or not. Right.
Amber Northern:
And that this is coming down the pike. Right? Yeah.
Michael Petrilli:
Well, and that's what I wonder. I mean, yes, it's coming down the pike, but also just that, but that you're starting to be, I mean, did they know in those early years that they were being watched in some new way?
Amber Northern:
So before they were evaluated, but it wasn't annually. I think it was every other year. And then when obviously the teachers who are more experienced, it's even less than that. Okay,
Michael Petrilli:
Gotcha. Alright. The tenure thing is interesting to me too. I'm a big fan. I've been writing about this, one of my many ideas that doesn't seem to ever get any traction, which is that more places should take the tenure approval process seriously. And so it sounds like Tennessee tried that. Of course New York City did this during the Bloomberg, Joel Klein years huge impact. I mean, they used to basically rubber stamp everybody and then overnight they approved fewer than half one year. And the deal was that didn't mean you were fired, it just meant that you didn't get tenure and then you could come back the next year and try again. But I do wonder if there's not enough of enough immediate benefit for getting tenure in the higher ed world, I think that means you get promoted, you get a better title and maybe a raise also. And it seems like there should be, at least it should come with something, some big benefit, not just this protection stuff. And maybe it should also be limited. Maybe there should only be so many number of tenured spots at every school, and we should make sure those spots are distributed evenly and that high poverty schools have as many as others. And so you basically, in higher ed, you force a little bit more of this sort of upper out process.
Amber Northern:
Although Mike isn't a guaranteed lifetime job a pretty big benefit.
Michael Petrilli:
Well, it is. No, absolutely. And so you're right, but maybe there's not enough of a downside under this system of not getting tenure, right? I mean, should it be you either get tenure or you're gone versus you can come back the next year? I don't know. I mean, look, this notion of trying to make it more serious is just because we are terrible at firing bad teachers once they have tenure. But we're not any much better firing bad teachers before they have tenure. At least we don't think we do. I mean, it's possible that some of the attrition we see in those early years is because people are getting pushed out and they just doesn't show up in the data because they aren't officially fired. They're just decide to leave on their own under pressure. We don't know. But I would like to make it more likely that in the early years we push out bad teachers.
David Griffith:
But Mike isn't the point of the study. I mean, okay, I don't necessarily disagree with any of that, but it seems like the point of the study is that maybe we talk a little bit too much about firing people and not enough about just holding everyone's feet to the fire. I don't know, Amber, have there been other studies that have shown this before? Because
Amber Northern:
I'm trying to think. I mean, obviously Mike talked about New York, but we haven't had a lot of places that have decided to push out tenure after five years. And that affected obviously everybody, like you're saying, not just, but no, I don't think we know enough about extending tenure guidelines, if that's what you mean, David.
Michael Petrilli:
But David is your point that because we're seeing improvements, right? But the pretty small, I mean they, does this add up to something real?
David Griffith:
Yeah, of course. I mean, everything adds up to something real if it applies to everybody. I mean, you can fire the worst 5% of teachers, or you can make a hundred percent of teachers 5% better. I'm butchering the math there, but you get the point, right? It doesn't have to be any improvement that applies to all teachers in perpetuity is a huge deal. And I mean, that was the part that I thought was interesting, and I don't think I've heard before from other studies. I'll just say if it's true, it's a huge deal, right? Because it sort of implies that even if these reforms are undone, you're going to have a wave of better than otherwise. It's expected teachers just passing through the system for 10, 20, 30 years anyway. It suggests that reformers aren't completely wasting their time, even if things are periodically undone.
Michael Petrilli:
And maybe we should try to get back to doing this teacher valuation thing and do it better the next time. Now I see your point, David. Of course. I would say both and right. Let's both get rid of the 5% or 10% or 20% worse teachers and help everybody else get better. Alright, well we will leave it there there.
I'll give myself the last word this time.
David Griffith:
Something new and different.
Michael Petrilli:
Oh, come on David. Come on. Alright, thanks Amber. That's an interesting one. Complicated but interesting. So thanks for bringing it and thanks again for your fundraising efforts. We absolutely appreciate that as well. Alright, well that is all the time we've got for this week. And so until next week,
David Griffith:
I'm David Griffith.
Michael Petrilli:
And I'm Mike Petri of the Thomas b Fordham Institute. Signing off.
#924: How presidents polarize education debates, with David Houston