‘It’s Not About the Evidence Anymore’

Scott Freeman has assembled the evidence: When STEM courses are taught using active-learning techniques rather than a standard lecture, students perform better, according to a widely cited meta-analysis he published in the Proceedings of the National Academy of Sciences journal. And not only that, Freeman’s work shows, when much of class time is devoted to active learning, it reduces gaps in student performance. So why aren’t more courses being taught this way? And how might that finally change? To find out, The Chronicle caught up with Freeman, who has retired from his position as a lecturer in biology at the University of Washington but continues to conduct research. This conversation has been edited for length and clarity.

You’re probably best known for your 2014 meta-analysis on the effectiveness of active learning, published in PNAS. Can you quickly walk through those findings and the paper’s impact?

That paper was inspired by earlier work on my own courses at the University of Washington, showing that, when I quit lecturing and started doing active learning, we were getting really important reductions in failure rates. It worked here, but that’s one place, one professor, one course. I just asked the question: What’s really happening? That is the core of what a meta-analysis is supposed to do. So what that work showed was across science, technology, engineering, and math courses at the undergraduate level, on average, when people switch from lecturing to any kind of active learning — we were extremely generous with the definition — that we were seeing important increases in average exam scores on equally rigorous exams, and significant drops in failure rates.

That paper struck a chord in the community. I think it’s been cited over 7,500 times. The analogy I used when I was giving talks was that the impact seemed to be similar to when the surgeon general of the United States published essentially a meta-analysis in 1964 on smoking’s being a danger to public health. It’s not about the evidence anymore. We know that this works better.

As I would remind people: People still smoke.

Do you have a ballpark of how frequently STEM courses — let’s just focus on STEM — are taught using active learning?

There was a really good analysis — a nationwide, mostly U.S.-based, but very large data set — that was done by a team led by Marilyne Stains that was published in Science a couple years ago. They attacked exactly that question and found that lecturing was still predominating.

So what’s keeping more instructors from using active learning?

Several things are going on. First, although faculty are evidence-based people in their fields, we’re still human beings. And often, people demanding levels of rigor and evidence in their field switch to teaching, and all of a sudden they go back to intuition. So commonly you’ll hear from people that, Well, it worked for me. Which means that something’s wrong with my students, because it was fine for me and look where I am now. And that’s just a human thing: personal empiricism.

But the nut of the problem is that the system doesn’t reward change, especially in teaching. If you’re an average faculty person and you read these papers and talk to colleagues who are getting really excited about teaching and innovating and seeing their students do better, you could say, Well, that would be great, but that’s going to cost a lot of time and energy.

Another thing is, nothing works the first time. When I started doing active learning I got no change. I always fell on my face, I just expected it. But that’s not fun. And people will say: Oh, my promotion, my merit pay is dependent on student evaluations, and I’m going to get creamed. And if I’m eventually successful, and learn all these new skills and really extend myself and redo my courses and sacrifice other parts of my job, I don’t get paid any extra. The university doesn’t get any more money. Nothing changes.

So how could institutions encourage excellent teaching?

Far and away the most important thing is the reward system. My pat phrase is: You don’t get systemic change unless you change systems. But again, we’re human beings. Systems are really hard to change.

So I’m talking about evaluating teaching. We’ve known since the 1980s that there’s a very strong active bias against women. So we’ve known for two generations that these are terrible data. And if you use bad data, you make bad decisions. So that’s an obvious problem. Several universities around the country have recognized this and are working on new systems for evaluating the quality of teaching. We have new tools; they’re really strong, research-based — they’re called classroom-observation protocols — that are published and have good validity evidence. We now can evaluate teaching in an objective, reliable way and use it to make good decisions.

And also just getting real about it. Getting beyond lip service. Because everyone’s going to say, Oh, yeah, undergraduate education is important to us, and teaching is part of our institutional mission. But when the rubber meets the road, who actually gets hired? Who actually gets promoted? Who actually gets the corner office, the awards on campus? Where does the prestige come from? It’s still really about research. Institutions have to take this evidence about teaching and say: People who are getting better student outcomes, especially for students who are underprepared coming into college, are going to get paid more, promoted faster, and given the kind of perks that top-flight researchers get.

Where has your research gone since the 2014 paper? What are you working on now?

The biggest project I worked on since that 2014 paper was extending that result, again inspired by something that happened in my own classroom. When I got better at teaching with active learning, we were seeing differences in exam scores, and failure rates between low-income and higher-income students, shrink. I’d read enough of the literature in higher education to know that people have been talking about this as a crisis for almost 100 years — that lower-income individuals in higher education had really just been struggling under traditional teaching. And I looked at the data in my own classroom and said, Oh my god, we could do something about this.

And again it was a process of, that was one instructor, one student population, one course, but what about everybody else? We did a follow-on meta-analysis. It was published in a very widely read journal but, unfortunately, it was also the Monday that Italy went into its first total Covid lockdown. That was the day everybody kind of realized, Oh my god, Covid is real, and it’s coming. And so our paper came out then. It’s getting widely read and widely cited, but it hasn’t had the sort of phenomenal impact that the 2014 paper had. But I consider it far and away the most important work I’ve ever participated in.

The data are showing that, in STEM at least, if you do a little bit of active learning, you don’t see any of these performance differences change. But if you do a lot, they start to shrink, often dramatically. And now the follow-on work is to figure out why.

What do you think the answer is?

We have a hypothesis. This work, by the way, is being led by Elli Theobald at the University of Washington and Sarah Eddy at Florida International University; I’m part of a really fantastic team. Elli calls it the heads-and-hearts hypothesis. So we think it’s a combination of deliberate practice — that you have to put in 10,000 hours of really hard, uncomfortable practice — if it’s athletics, it’s sweaty, if it’s music, it’s playing scales over and over. And we think that’s what the active-learning exercises do: When you’re asking and not telling, the students are practicing.