Blended Learning with Khan Put to the Test #Edtech #KhanAcademy #k12 @Edsurge
In this post, that will likely be a long one, I examine the latter exploration because the team that is carrying out this study is calling it an experiment. Experimentation in any social science is difficult, and it is particularly difficult in education. There are few well controlled experiments in education, and those that are well controlled are usually limited in scope. I recently dissected a pilot study of a blended learning environment with School of One technology in the New York City public schools. Below, my scalpel is even sharper. I do this not because I am a curmudgeonly union stiff. Quite to the contrary, I really want this to work. I am critical because I want it to really work.
I am excited about the possibilities offered by well-used, appropriately designed EdTech. However, far too often in education, the real experts, classroom teachers, become the subject of someone else’s untested idea. There are many reasons for this, but I will leave a thorough discussion of such issues to those in education politics. Let’s talk tech!
I am entering this conversation two weeks into the Envision academy “experiment,” and all of my information is coming from the blog where they are documenting their progress.
The Blend My Learning Team that is doing this important work in Oakland recognizes that what they are doing would not stand up to academic standards because of their small sample size; two classes of 25 students each. Nonetheless, the study appears to be controlled in three significant, albeit insufficient, ways. Kudos to the team for randomly assigning students to the experimental and control groups. More praise to the team for recognizing that the study would have to be done with the same teacher in both the control and experimental groups to have any meaning. And lastly, the team selected for a pre and post test a standard algebra concepts understanding metric, the MDRT, to measure the effectiveness of their treatment.
Unfortunately, this is where things fall apart. Based on the following paragraph lifted from the BlendMyLearning blog it is apparent that there is a team of researchers/educators in the experimental, blended Khan class.
“One of the questions we are considering with Khan is whether we truly let students work on whatever content they need. Khan suggests that all students start at the beginning of their “star chart” with single digit addition and then build up steam as they get a lot of problems right, earn “energy points” and badges, and get comfortable on the platform. That said, what about the students who have big numeracy gaps in fractions, decimals, percentages, etc.? If they only have five weeks of summer school to remeidate algebra, we are pondering whether it is “okay” for them to focus on their numeracy gaps but not get as much of the algebra content. Our plan is to give them a week or so to focus on where they need it most and then monitor the data. But it may be that these students get more of a foundational shoring up rather than true algebra intervention.”
More than one adult in a classroom gives a class an advantage. Even if that adult is an impartial observer, and it does seem that the BlendMyLearning team is involved in a not insignificant way in the classroom, students behave differently and are likely to be more task oriented with more supervision. Is the team equally present in the traditionally taught class?
The research team decided to control the experiment by excluding all technology from the traditional class. This is unrealistic. Most teachers employ technology in some fashion. Thousands of math classrooms across the country use interactive white boards. Many teachers do some kind of online lab work with students, and there are a growing number of classes employing remote response pad technology. Preventing the teacher from using any technology in the traditional class, the control, is akin to attempting to control an experiment on the effectiveness of using GPS in navigation by preventing the control from using a map.
Experimental effect. The experimental group is being treated as special in several additional ways. First, every student has a brand new Google Chromebook. Did they have to be new? Students are asked to give feedback on the learning process on a regular basis. Are students in the traditional class also asked how the class might be improved to better meet their needs? The experimental group students have been interviewed on video and asked to reflect on their learning process. Is this valuable reflection also happening in the traditional class? Students in the experimental group were told that they are part of a “revolutionary” new technology program. No doubt, buy-in to the process was necessary. I know. I am a high school teacher. But the experimenters have already biased the students toward success by telling them that they are a part of something that will change them.
In short, while the Blend My Learning study at Envision Academy may provide the Khan Academy team (who, by the way, further tainted the experiment by visiting the school during the process) with valuable insight into what works for kids in a blended environment, any results the team attempts to publish should be consumed with scrutiny. For example, nothing but a significant gain on the MDRT metric should be considered, well, significant. Any qualitative analysis about student buy-in will have to be disregarded entirely because of the adult attention the experimental group has received – “Are we going to be famous?”
Nonetheless, I was impressed with the reports of the on-task behavior, and the comment about zero classroom management issues in the experimental class. Blended learning, so far, has shown great promise in high school science classes when used as an inspiration to inquiry, and when the video is not (paradoxically) a straightforward explanation of a concept. Blended learning has also been used successfully for years as a credit recovery device for students, like these Envision Academy students, who are repeating the class having failed it at least once already. I am skeptical but hopeful about the possibilities for differentiation that blended learning offers in high school. If Envision Academy students show significant progress on the MDRT there may be reason to get curious, but please contain your excitement until the teacher reviews start coming in.