On this week’s Education Gadfly Show podcast, Robert Pondiscio, a senior fellow at Fordham and the American Enterprise Institute, joins Mike and David to discuss the lack of curriculum oversight in American schools. Then, on the Research Minute, Amber examines a new study of whether aspiring teachers’ professional references predict their later performance.
Recommended content:
“How public schools became ideological boot camps” —Robert Pondiscio, The Free Press
“Taking curriculum implementation seriously” —Robert Pondiscio, Fordham Institute
Dan Goldhaber, Cyrus Grout, and Malcolm Wolff, “How well do professional reference ratings predict teacher performance?” Education Finance and Policy (March 2024).
Feedback Welcome: Have ideas for improving our podcast? Send them to Daniel Buck at dbuck@fordhaminstitute.org.
Michael Petrilli (00:01):
Welcome to the Education Gadfly Show. I'm your host Mike Petrilli, the Thomas B. Fordham Institute. Today Robert Pondiscio, a senior fellow at the American Enterprise Institute, joins us to discuss how little curriculum oversight exists in American schools. Then on the research minute, Amber reports on a new study investigating whether professional references predict teacher quality, all this on the Education Gadfly show.
(00:39):
Hello, this is your host, Mike Petrilli of the Thomas B. Fordham Institute here at the Education Gadfly Show and online@fordhaminstitute.org. And now please welcome our special guest for this week, Robert Panio. Robert, welcome back to the show.
Robert Pondiscio (00:53):
Thank you. And let the record show. Welcome back to Fordham's Offices. You are not here, but I'm actually in your office in dc. Mike, where are you?
Michael Petrilli (01:02):
I love it. The last time this happened, Lindsay Burke showed up at our office and that was due on a miscommunication and broke into our office and did the podcast from there. It was amazing. Okay.
Robert Pondiscio (01:14):
I did not break in. I was invited in. I was let in by one of your colleagues. It's all legit above board.
Michael Petrilli (01:21):
I will be coming in later this afternoon, but let's be honest, I don't make it into the office all that often. Fordham is part of the problem in Washington DC with the downtown corridor
Robert Pondiscio (01:32):
And all hitting aside, I don't think I know more than a handful of people who go into an office every day, nine to five, Monday through Friday. That is a profound cultural shift. Thanks.
Michael Petrilli (01:44):
Yeah, that's right. Although of course the sector that we cover, education is quite different. Right.
Robert Pondiscio (01:50):
Teachers got to stop.
Michael Petrilli (01:52):
Hey, before we keep on chatting, let's bring in my co-host, David Griffith.
David Griffith (01:57):
Hey Mike. Good morning. Good morning, Robert.
Robert Pondiscio (01:59):
Good morning, David.
Michael Petrilli (02:01):
Alright, great to have you with us. Alright, well, everybody should know that Robert is a senior fellow at the American Enterprise Institute as well as here at Fordham, and we were very excited when we saw Robert hitting the big time, had an article in the Free Press. Barry Weiss's Big, what do you, it's not even substack, it's beyond that now, right? Well,
Robert Pondiscio (02:23):
It's still published on Substack. Oh,
Michael Petrilli (02:25):
Is it? Okay. We'll call it
Robert Pondiscio (02:26):
Substack. Technically a substack, but it is also, I just saw figures that showed if it were actually a, when you combine its print and online audience's print being zero. It's one of the largest newspapers in the country out of nowhere. And I like to say I've known Barry since before she was Barry Weiss, so I have almost a fatherly like affection for what she's building Good for her
Michael Petrilli (02:48):
Former New York Times reporter who an editorial writer broke away. And now let's talk about this great article you have. Let's do that on ed reform update. Okay. The article is called How Public Schools Became Ideological Bootcamps. Not Surprising for the Free Press, which kind of leans into its anti woke arguments, but again, is very iconoclast. It's, you can't quite pin it down in left, right? But certainly anti woke. So tell us about this article. It sounds like there's the ideological bootcamps, but really your argument is that if people think that someone's in charge of curriculum in the American classroom, they are sadly mistaken.
Robert Pondiscio (03:35):
Yeah, I mean, I'm not going to complain about the headline, but it's a bro mind to say that writers don't write the headlines. And I wish I could remember the headline with which I submitted the piece, but it really was about in its original incarnation and still is about how the sausage is made. And Mike, you're familiar with this. You've had to listen to me nater on about this for the better part of the last decade or so that we make this assumption that what happens in schools is a top down thing and we just fail to reckon with the degree to which, and I think I said this verbatim in the piece, almost no other category of public employee has the latitude for good or for ill that ordinary public school teachers have. Actually, I can tell you how this piece started its life.
(04:26):
It started off as a rere reporting of a terrific piece that the Free Pressers Francesca Block did about a school that I used to live across the street from back in the day, PS 3 21 in Brooklyn in Park Slope, Brooklyn. Over the winter there was this big controversy that Franny block uncovered where a Black Lives Matter activity book got sent home with a bunch of elementary school kids during not a school holiday, but a snow day. And I recognized the thumbprints right away. There's no way in heck that this was something that the New York City Department of Education or even District 15 authorized, but that's not the point. They didn't forbid it either. So I thought this was a nice opportunity to tell a story that frankly I've told before about how you think what's going on in your kid's school because you know what the official curriculum is, you don't because teachers and administrators have this extraordinary latitude. So I confirmed my suspicions, I called sources in the DOE in a district 15 to say, Hey, is this official curriculum? And of course the answer came back as no. Interestingly, I also reached out to the principal of PS 3 21 who did not return my calls or emails. So
Michael Petrilli (05:41):
Help us understand what we should think then about the big picture here, because I feel like one thing we've debated, we as in you and I, but also in the education reform space is when a lot of conservatives especially started complaining about some of this crazy stuff showing up in America schools. The question is always, look, is this the rare rogue teacher or is this really widespread? I send my kids to school, public school, Montgomery County, Maryland, very, very blue district. They do some woke stuff, especially in assemblies and things, and the kids rolled their eyes. But I don't see a lot that I'd call indoctrination. But again, I don't know, how do we possibly know if nobody's in charge of this stuff?
Robert Pondiscio (06:27):
I think it's unknown and unknowable, Mike, and I'm trying to be as fair as possible about this because it's not as if this is necessarily a bad thing. In other words, teachers have to have a certain amount of latitude for good reasons, differentiated instruction to engage kids. But the same thing that allows them to be effective also creates a permission structure for teachers, whether because they are ideologically committed or frankly they're just naive to take material that is either inappropriate or has not been vetted. Fordham knows this better than anybody. Remember the study that you did a few years ago? I can't remember the title of it.
Michael Petrilli (07:09):
The curriculum Bizarre Study with Morgan Poff? Yeah,
Robert Pondiscio (07:13):
Yeah, that's right. And it showed, not only is it kind of a crazy quilt and from my perspective, hey, this is incoherent inappropriate curriculum, but as I recall, that study also showed, hey, when teachers choose, they set the bar too low. So you also end up with stuff from sites like Share my lesson in Teachers Pay teachers. That is not only just potentially inappropriate, but just really unchallenging and below grade level.
Michael Petrilli (07:38):
So should we try to fix this? Should there be rules that say you may not use anything that hasn't been vetted by the district central office? I mean, is that where this is going hard
Robert Pondiscio (07:50):
To do and I'm not going to sit here and be the guy because I don't want the responsibility. And this is kind of the point we're asking teachers to do things that maybe they shouldn't have the latitude to do, but I think there's a value in just surfacing the problem. And the problem is, I think I said in the piece was you think what's going on in your child's school? Because the quote adopted curriculum, but then there's that RAM study that shows that almost literally every teacher in America uses some, or at least some material that they either create customized cure rate. So in order to answer your question, Mike, you first have to see the issue clearly. And my favorite example of this, I also cited this in the piece, is the example from a few years ago of the 1619 project, which as far as I could tell was authorized by exactly three school districts. But the poets are center can tell you 4,500 in which it was used. That's a big difference between in 4,500. I don't think either one of them are lying. It's just teachers doing what teachers, oh, I heard about the 1619 project. That sounds interesting. Maybe my kids will be engaged by that. I'm going to give them a reading from that, et cetera.
Michael Petrilli (09:02):
And by the way, some of the lessons, some of the lessons in that project were pretty good, right? I mean, it was not necessarily giving the lead article by Nicole Hannah Jones. Some of the other stuff was okay. It was kind of just regular black history. I looked at this stuff Robert way back.
Robert Pondiscio (09:19):
Okay, you're going to take your word for it,
Michael Petrilli (09:21):
But I will disagree with you about one other thing about this being a noble, once we put video cameras in every classroom and turn 'em on, we'll know this stuff. Okay.
Robert Pondiscio (09:29):
Do you want cameras in the classroom? Yes. I'm looking at you. You're a former teacher. You want to be on camera 24 7 or from
Michael Petrilli (09:36):
Some, oh, David loves this idea of mine. It's my hobby horse. David, what's your thought about all of this?
David Griffith (09:41):
Well, I think I have several thoughts. First, I agree that there is a danger that teaching can become politicized if teachers misunderstand their mission. And I tend to agree that this has become more common, although like we've been saying, it's basically hearsay and impressionistic, right? It's hard to know for sure. I think what makes it challenging is that one of the reasons we can't decide, say I'm just going to use the term CRT, how much CRT there is in the classroom. It's not just about observation. It's about what it is. It is not a thing. It is extremely difficult to define. Politicization is sometimes obvious and to a certain extent in the eye of the beholder. So let me just give an example. You want to talk about climate change. Should that be taught as fact or it should be taught as sort of opinion?
(10:45):
What is the right way to approach something where the preponderance of evidence is very clearly on one side, but there are skeptics, and I suspect that some of our listeners will give different responses to that. In other words, I think it's good for teachers to aspire to not be political, but I think to a certain extent, politicization depends on what you consider facts and what you consider contested. And it's kind of tough. I don't know. It is not that informative and in practice it's sort of something that we adjudicate politically. It's not written in the sky somewhere that you should take certain issues or factual claims as facts and other things as opinion. Robert has all parts say,
Robert Pondiscio (11:37):
I can tell David, I think you said, I don't want to put words in your mouth, but you said, suppose I want to talk about climate change. So my first response is, well, hold on. Is climate change in your scope and sequence? Is it part of, let's at least
David Griffith (11:48):
Start, suppose I want to talk about climate change in science class, right? I mean, I tend to agree. We saw social studies curriculum that were kind of wandering into this more advocacy space and we didn't penalize them. But it isn't really what I consider central to civics, for example. So I agree with you, but I think there as well mean you can imagine historical circumstances that have more than one interpretation, right? And should it be taught as facts or should it be taught as the liberal interpretation of events? Is this? And the conservative interpretation is this, I'll give you just one example. When I was taught about World War ii, I was told that basically that we dropped the bomb on Japan to scare the Russians. That it wasn't really even about winning the war, that it was about broader geopolitical posturing. And I would consider that a pretty left-wing interpretation of events. But someone to my left might not. And so I think it's challenging to draw a clear line.
Michael Petrilli (13:07):
And the key question we keep coming back to is who should do that? Who should do the line drawing? In a public school system, you'd like to have some process where the public gets to participate. And some you get people with content expertise and you don't let every one of the 4 million teachers freelance, but you can never button it down entirely.
Robert Pondiscio (13:32):
I mean, I've made this observation elsewhere that's somewhat related. Whenever the subject of teachers abusing their roles, performatively, the type of thing you see on Libs of TikTok every day, some teacher behaving badly. And I always say this, I say, look, do I think that libs of TikTok, what you see there is indicative of the profession as a whole? No, of course not. On the other hand, should it be quite so easy for them to find examples of it every day? No, of course not. So in other words, what that suggests to me is that there's a culture, a permission structure in place, which if you were concerned about what people thought, if you didn't view your role as somewhat performative, you wouldn't post things like that on lives at TikTok. So the existence of those kinds of videos, even if they're not the norm, suggests a teaching culture in which teachers are inclined to view themselves as the first and last word on what they put in front of children.
(14:34):
And if nothing else, if this piece accomplishes nothing else, Mike, I would like to see us discuss the efficacy of that. How much leeway about not just individual lessons, but subject matter. Because look, I'm going to get on my Ed Hirsch core knowledge platform again, when teachers make their decisions based on what will engage kids, et cetera, that's where you get this incoherent curriculum, as I like to say, where you get the rainforest four times and the bill of rights never. So it's not just politics, it's the breadth of knowledge that from one end of an education to the other that we want kids to learn and know. Alright,
Michael Petrilli (15:11):
Well we will give you the first and the last word. That is Robert Pio of the American Enterprise Institute. And Fordham, thanks so much for joining us on the show.
Robert Pondiscio (15:20):
Always a pleasure, boys.
Michael Petrilli (15:21):
Alright, now it's time for everyone's favorite Amber's research minute. Amber, welcome back to the show.
Amber Northern (15:35):
Thank you Mike.
Michael Petrilli (15:36):
So had a very special Father's Day weekend this weekend in the Petri family. Yes. My father got remarried.
Amber Northern (15:45):
Wow, nice. What was that like?
Michael Petrilli (15:48):
82 years old? This is about five years after my mom passed away and we're so happy. He and Mitzi, we had a wonderful small family wedding down there in Hilton Head. And now we'll see now they're going to start living together. And
Amber Northern (16:05):
How did he meet Mitzi? Mind you?
Michael Petrilli (16:07):
At church? At the
Amber Northern (16:09):
Church. Wow. Nice. Really sweet.
Michael Petrilli (16:13):
I don't think the Catholic church would recognize this, but I told her that he comes with a 30 day money back guarantee. So we'll see how this goes. It was, it was a wonderful occasion and it's a courageous and hopeful thing to do at any age.
Amber Northern (16:30):
Yes, it is indeed. But I have a feeling they're going in with their eyes open and we'll appreciate one another.
Michael Petrilli (16:37):
Yeah. Alright, well, hey, what you got for us on the research front,
Amber Northern (16:41):
We have a new study out by our friend Dan Gold Haber and colleagues at the University of Washington. It came out in the A EFP journal. Well, it was about to, this is sort of a sneak peek. It examines whether ratings on professional references on teacher applications are predictive of subsequent teacher performance for those hired based on teacher value added and classroom observation ratings. So a new, you don't see a lot on professional references on teaching applications. So I love that. This is a new area for research. They use data from around 3,500 teacher applicants from Spokane Public Schools in June, 2015 through September, 2018. They use a variety of control variables as well as various fixed effects, including rater fixed effects. That turns out to be pretty important. They're trying to isolate the impact of raters who rate multiple candidates and the types of raters since they can't track outcomes for applicants not hired, there's a big risk selection bias.
(17:50):
So they run a bunch of models to try to suss out this impact of selection bias, including examining variables that they consider related to hiring outcomes, but not performance outcomes like how competitive a job is. So suffice it to say their robustness checks run several pages and most of their results to the best that they can measure it, were not impacted by selection bias, at least in these various ways that they tried to get at it. Key finding, they find an overall significant relationship between reference ratings and teacher performance as measured by both observational evaluation ratings and teacher value added in math only. But these relationships are constricted to certain subsamples of their sample. So for instance, the reference ratings of applicants with prior teaching experience are predictive of performance, but the ratings of novice applicants are not. Also the predictive validity of reference ratings varies according to the rater type.
(19:00):
So if you are a quote, if you check the box that you're a rater as a principal, a supervisor, an instructional coach, or a department chair, even colleague, those are significantly predictive of performance. Whereas these other rater types that are a little bit more far removed from knowledge of that applicant's attributes are not, and even colleague is not as predictive as principal and supervisor. Specifically a one standard deviation change in the summatives rating measure corresponds to roughly 10% of a standard deviation change in teacher performance. So not huge but significant. Again, rater fixed effects make a difference because those are a big source of variation. So they say there's plenty of room for improvement because we think there's a lot of promise in these reference ratings. But maybe if districts did a lot better job of giving references guidance as to how they should think about rating applicants, that might make it a little bit less a variation and more consistency. And they also say that this is predictive, but there are limitations relative to who's doing the rating and who's being rated in terms of whether it's a novice or an experienced teacher. That's what I got.
Michael Petrilli (20:27):
Well, very cool. Professor Gold Haber, another great one. It does make sense that it works for people that are teachers. And then you can ask people who've actually seen them teach. I mean, this has forever been, the challenge of teaching is it's hard to predict who's going to be a good teacher. But once somebody's in the classroom and you can see how they perform, then you get a much better idea of this. And so good to know that people, it sounds like being honest in their assessments and that these assessments, these referrals track with something. I mean, I always have been curious whether referrals in any line of work do a whole lot of good, especially in some fields where people, because of legal reasons are very nervous to say hardly anything other than, yes, this person worked here for this time and this being, and maybe unless the person really was a low performer, and maybe you can read the tea leaves and get that out of someone, but I don't know, it sounds like here this is something worthwhile.
David Griffith (21:28):
Amber, I have a question. So it's about the rater fixed effects. So as I understand it, that means that essentially a particular rater when looking at the future performance of the teacher, teachers that they rated was able to distinguish between better and worse teachers, or at least their ratings were predictive, right?
Amber Northern (21:50):
If they were rating a experienced teacher, they weren't for novice teachers.
David Griffith (21:56):
Right. Okay. So correct me if I'm wrong, but I think that does not necessarily imply that that information is acquired by whoever is hiring the teacher. In other words, if someone is hiring a teacher and they have teachers who are rated by different individuals, the fact that raters were able to distinguish between the teachers that they were rating does not mean that the person who is hiring a teacher can successfully use that information when the raters have very different standards to distinguish between different teachers who were rated by different raters. In other words, the raters are picking up on something, but that may not be apparent to the person hiring. Is that fair?
Amber Northern (22:45):
That is fair. And they also, by the way, which I didn't mention, looked at raters who had rated multiple applicants to see how that may have affected the ratings. Yeah, I mean, I think that's right. I think that the big story here is that these ratings raters matter and that to the extent that we can help them out in terms of how their rating would be better.
David Griffith (23:13):
Yeah, I mean, I guess, and my other comment is that even in the best case scenario here, it seems like we haven't really solved the fundamental problem, which is knowing who's going to be a good teacher before they get into the classroom.
Amber Northern (23:24):
Right.
Michael Petrilli (23:25):
Well, I have an idea about that one that maybe Dan could research next, which is that we have now these quasi novice teachers that are called tutors. We've done this big experiment all over the country with high dosage tutoring. And so there should be thousands and thousands of people out there who have now taken a crack at tutoring kids, some of whom may decide they want to become teachers, and we have a little more information about them than about other brand new novices. I wonder if there's a way that we could figure out which of those really have potential,
David Griffith (23:56):
Mike. That's the sort of idea you should put in an email instead of shouting out to the entire universe. But I like it. I actually think it's a really good idea and there may be research on that. I'm not sure.
Michael Petrilli (24:07):
There we go. Okay, good. Alright. Hey look, these ideas, they pop into my head. I can't hoard them.
David Griffith (24:13):
Notice I've noticed
Michael Petrilli (24:15):
They long to be free. They long to be free. Alright. Hey, thank you Amber. Good stuff. Thank you to Dan Gold Haber and his colleagues. That is all the time we've got for this week. And so until next week,
David Griffith (24:28):
I'm David Griffith.
Michael Petrilli (24:29):
And I'm Mike Petri of the Thomas b Fordham Institute. Signing off
#925: We need more curriculum oversight, with Robert Pondiscio