References:

He, Y., Lu, J., Huang, H., He, S., Ma, N., Sha, Z., … & Li, X. (2019). The effects of flipped classrooms on undergraduate pharmaceutical marketing learning: A clustered randomized controlled study. PloS One, 14(4), e0214624.

Setren, E., Greenberg, K., Moore, O., & Yankovich, M. (2019). Effects of the flipped classroom: Evidence from a randomized trial.

---

Although the notion of “flipping” classrooms has been around since the 1990s, the increasing ubiquity of Internet access has resulted in increased adoption and research concerning this pedagogical tool. In flipped classrooms, the common paradigm of in-class lecture, out-of-class practice is reversed; students are primarily exposed to the content of the course through out-of-class video lectures. During class, instructors act less as “sages on the stage” and more as “guides on the side,” facilitating learning through in-class activities. In other words, flipped classrooms mean in-class practice, out-of-class lecture. 

Flipped classrooms are a clear departure from how many of us learned content in school, and the effectiveness of this instruction style remains in contention. 

In this post, I describe the results of two recent, and somewhat conflicting, randomized controlled trials implementing flipped classrooms. Despite the incongruous results of these experiments, I believe that comparison and contrast will illuminate possible causes underlying the success and failure of flipped classrooms.

Study 1: He et al. (2019) — “The effects of flipped classrooms on undergraduate pharmaceutical marketing learning: A clustered randomized controlled study”

In He et al’s (2019) study of flipped classrooms, a single professor taught five classes (3 flipped and 2 traditional) to a total of 137 undergraduates majoring in pharmacy at a Chinese university. Students were randomly assigned to either the flipped or traditional classrooms. 

Students in the flipped classroom watched video lecture before class with interspersed quizzes (see Figure 2). Class time centered around case discussions, student presentations, and other group-based activities. The traditional classes revolved around in-class lecture and out-of-class supplementary activities, such as quizzes and case reports. 

Overall, He et al. report positive findings from their flipped classroom intervention, noting higher final exam grades (88% in flipped versus 80% in the traditional), better student evaluations, and more frequent student-instructor interaction. However, the positive results of this field experiment could also be partially explained by the fact that flipped-classroom students spent significantly more time on their classwork than their traditional counterparts.

Nevertheless, He et al.’s study analyzed data from class sections taught by a single professor. This element of their experimental design limits our ability to make strong generalizations to other professors. It could be, for example, that the professor in this study was a very experienced flipped classroom facilitator. Thus, their success may not translate to other classrooms.   

Study 2: Setren et al. (2019) — “Effects of the flipped classroom: Evidence from a randomized trial”

In comparison to He et al.’s relatively rosy outlook on flipped classrooms, Setren et al. (2019) report negative findings in a similar randomized controlled intervention. In Setren et al’s field experiment, units of instruction dispersed throughout 80 sections of economics and calculus at West Point were randomly assigned to either a flipped or traditional class implementation (with 1,328 participating students and 29 participating faculty members).

West Point was chosen due to its standardized curriculum across course sections. Additionally, calculus and economics both require solving practice problems with objectively correct/incorrect answers, allowing for the direct evaluation of the effects of flipped classrooms on exam performance across different instructors.

Students in the flipped classrooms received initial instruction from pre-class video lectures that were supplemented with in-class question-and-answer sessions and problem sets.

Students in the traditional classrooms received initial instruction via in-class lecture and were given the same problem sets as those in the flipped classrooms. However, they were not required to complete this extra practice, nor was it graded.

Learning outcomes were measured using a quiz at the end of the unit and unit-relevant questions on the final exam. Additionally, Setren and colleagues analyzed the video-watching habits of the students in the flipped classrooms.

Interestingly, the average student only watched two out of three videos. Despite this relatively low use of the out-of-class videos, students in the flipped classrooms outperformed their traditional classroom counterparts on the quizzes occurring directly after each unit.

However, this effect on quiz scores was small and varied between math and economics. Math courses showed positive effects and economics courses did not show any effects. This difference in scores exhibited on the unit quizzes in math classes disappeared on the relevant final exam questions. In the words of Setren and colleagues, students in the traditional class “caught up to the flipped classroom students” on the final exam (p. 15). 

Overall, Setren et al. interpreted these results as showing little positive effects of their flipped classroom implementation, with no long-term effects on student learning and an exacerbation of existing educational inequalities. However, due to the high rate of noncompliance (failure to watch the out-of-class lectures) and the fact that only a single unit of each class was flipped, drawing strong conclusions concerning the overall benefit of flipped classrooms is inappropriate.

The positive effects of flipped classrooms may have been found had the entire class been flipped, rather than individual units, as both students and teachers need to learn to accommodate this relatively less-common classroom structure. As Setren et al. note, the details of flipped classrooms implementation are most likely key to the success or failure of these types of interventions. 

Conclusion

At first glance, the results of these two studies may seem conflictory. But this is not necessarily the case. Several details of the flipped classroom implementation varied between the two studies. Furthermore, these studies were conducted in different countries using different subject matter (pharmacy versus mathematics/economics) and each study focused on different variables of interest. 

In my eyes, the biggest difference between the two studies is the level of student accountability ensured by the respective professors. In Setren et al.’s (2019) intervention, students did not seem particularly enthusiastic about watching video lectures — the average student only watched half of the available lecture content! He et al.’s (2019) study implemented frequent quizzing in the out-of-class activities, which resulted in flipped-classroom students spending significantly more time on their classwork than their control condition counterparts.

The importance of these differences in the amount of practice testing and time-on-task cannot be understated, given the robust evidence from cognitive psychology showing the benefits of these very practices on long-term retention. 

Additionally, the Setren et al. (2019) only flipped one unit (three class sessions) within each course. As a result, professors and students in the flipped classroom did not have much experience with flipped classrooms. Conversely, the He et al., (2019) study flipped an entire course, which may have allowed the participants to become acclimated to the flipped class pedagogy.

Based on these two studies, it appears that flipped classrooms are more likely to be effective if professors pay detailed attention to the instructional design of their flipped classrooms. Perhaps more importantly, students must be appropriately incentivized to complete out-of-class activities in preparation for the in-class learning activities. The inconsistent results of these two studies highlight the need for further rigorous investigation of the effects of flipped classroom to disentangle the key contributors to the success and failure thereof.