Written by Dr. Autar Kaw, from the University of South Florida. Find him on Twitter @numericalguy.
Introduction
In a three-year study conducted at three universities, University of South Florida, Arizona State University and Alabama A&M University, for the same course of Numerical Methods, no statistical differences were found in the cognitive and affective learning outcomes between teaching the class in a flipped mode vs. the blended mode. The effect sizes were also negligible to small when comparing the final examination scores and results from a classroom environment survey.
However, in focus groups and qualitative data collected through surveys, many students liked the interactive nature of the flipped classroom, while instructors opined that this mode improved self-learning regulation as well as skills for life-long learning.
Note that the flipped mode was not compared to the traditional lecture mode as it has already been challenged consistently (such as in the famous meta-study on active learning by Freeman et al) that control groups for current pedagogical interventions should not continue to be the traditional setting.
Challenge
The most challenging part of the flipped classroom has been established to be the pre-class learning. This could include pre-requisite knowledge, which, if covered in class, can be boring to some and a revelation to others.
The other part is the preparation of the new material, which in the flipped class is generally covered through a combination of video lectures, assigned pages from the textbook, and an online quiz taken through a LMS or in-class. However, this method also does not differentiate between the needs of the various students.
Most of the student resistance for flipped classrooms stems from the pre-class learning assignments. The adaptive lessons developed using Smart Sparrow improved the pre-class learning experience of my students – both, cognitively and affectively.
Solution
The solution to differentiating among various students with different levels of pre-requisite knowledge, as well as the preparation of the new content for the flipped classroom, was to develop and present this content through the adaptive learning platform Smart Sparrow.
Teaming up with an undergraduate student at University of South Florida, Eleonora Delgado, we developed the adaptive lessons. We faced several challenges around using our own scoring algorithm and the inclusion of algorithmic questions. Many of these were solved through the Smart Sparrow Knowledge Base, support team assistance, as well as some paid training from Smart Sparrow. By summer 2017, we had the adaptive lessons available for half of the course, as proposed in a NSF grant that was led by me (Autar Kaw) for transforming the flipped classroom via adaptive learning.
Each lesson included short video lectures, textbook readings, and assessments on new and pre-requisite content. Depending on how a student progressed through an assessment, self-paced and unique paths were selected for the student. Students could repeat the lesson as many times as they wished. The adaptive lessons for the upcoming week were assigned after the last class of the week on Thursday, and were due on Tuesday morning a few hours before class met for the week.
This structure also created a one-stop learning resource for the students, as all the content was in one place and accessible across multiple devices.
Results
Assessment was done through a final examination, surveys, and focus groups. Dr. Renee Clark of University of Pittsburgh directed the assessment activities of the project as an external evaluator.
The final examination consisted of two parts: multiple-choice questions for assessing lower order thinking, and free-response questions for checking higher-order thinking.
The adaptive method was associated with higher achievement for community college transfers, URM students, and Pell grant recipients. The effect sizes associated with the flipped-with-adaptive mode vs. flipped-without-adaptive comparisons approached a medium level, with Cohen’s d approximately equal to 0.40 for the free-response questions.
The classroom environment was measured via College and University Classroom Environment Inventory (CUCEI). The survey measures seven dimensions: cohesiveness (students know and help one another), individualization (students treated individually and differentially), innovation (novel class activities or teaching techniques), involvement (active student participation in class activities), personalization (interaction with instructor), task orientation (well-organized class activities), and satisfaction (enjoyment of classes). The classroom environment measure was statistically significantly higher with medium effect sizes for the adaptive learning classroom on four of these dimensions: cohesiveness, individualization, personalization, and task orientation.
Student Feedback
“Smart Sparrow did impact learning more than the typical approach. I had to pay attention and take notes when watching the video, and it helped me to understand better. I liked watching a video and then answering a set of questions right away.”
“I went over the material earlier and so would have to re-learn it later, which helped me. So, I had several times of being exposed to the same material…”
“I was more engaged during class, because I knew what was going on.”
“We recently went over ordinary differential equations (ODEs) — first, second, and higher order. I feel less prepared for the upcoming exam versus earlier exams that covered the Smart Sparrow topics. I will have to spend time outside of class studying. I would have more confidence if ODEs had been covered in Smart Sparrow. With Smart Sparrow, I was ‘walked through’ the material, could see my grasp of the material, and received a reward, which was nice, but I didn’t have this with ODEs.”
This material is based upon work supported partially by the National Science Foundation under Grant Number 1609637. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
