Every semester, college instructors face a choice: whether to restrict the use of laptops and other devices in their classrooms or to, instead, let students decide for themselves.
And for classrooms that do allow devices, students face an ongoing set of choices: to take notes electronically or by hand, to check the textbook or the text message, to check Instagram or Twitter.
This bounty of choices, and the multitasking that often ensues, may be the very problem that drives some instructors to ban devices altogether. In fact, evidence suggests that computer-based multitasking can reduce student learning, not only for those students using devices but also for their distracted neighbors. Even when computers are used for the praiseworthy purpose of taking class notes, computer-using students tend to do more poorly on later tests than their peers who took notes by hand.
For device aficionados, these are dispiriting results. But they're also quite limited.
Studies investigating the effects of computer use on student learning tend to follow one of two approaches: They either look for correlations between device use and student learning, or they experimentally assign people to use computers or not. The problem with the first approach is that a correlation does not entail causation. It could be that students who tend to use laptops do poorly because they're the ones who are bored or prone to distraction, not due to computer use per se. The problem with the second approach is that experimental manipulations are more likely to last an hour than a full semester. Perhaps over the course of several weeks, students begin to benefit from computer-based notes, or from the opportunity to reference earlier material as a lecture unfolds.
The strongest form of evidence for the effects of computer use on classroom learning would come from a large-scale randomized control trial (RCT): an experiment in which students are randomly assigned to otherwise-identical courses that vary only in whether they allow students to use devices during class. For obvious reasons, this kind of experiment is difficult to conduct. In most educational contexts, students choose their courses — they aren't randomly assigned. Different courses involve different instructors, different material and different tests. And if an instructor were to teach two versions of the very same class, she'd be pretty unlikely to adopt different policies about computers in each one.
Given these challenges, it's pretty remarkable that researchers Susan Payne Carter, Kyle Greenberg and Michael Walker may have pulled it off: an RCT investigating the effects of classroom computer use on academic performance.
In a working paper published last month, they report the results from a study of 726 sophomores distributed across 50 offerings of a highly standardized course in introductory economics at West Point. The students were effectively assigned at random to versions of the course that adopted one of three policies: a ban on all devices, unrestricted use, or permission to have a tablet, face-up, on one's desk.
At the end of the semester, all students completed the same final exam, which included multiple choice and short answer questions. The exam grading was automated, resulting in the same objective measure of learning for students across all versions of the course. The researchers could thus assess whether differences in computer use policies across the otherwise similar versions of the course resulted in differences for student learning.
And the results were dispiriting — at least for those who love their devices.
Students in either of the conditions that allowed devices performed significantly worse, on average, than their peers in the condition that banned devices. This effect held up even when the researchers controlled for other factors that could potentially affect student performance, such as composite ACT score and baseline GPA. The two conditions that did allow computers — unrestricted or face-up tablet — did not differ from each other.
While the effect of computer usage wasn't huge, it wasn't tiny, either. Being in a condition that allowed computers was associated with a grade drop of about one-fifth of a standard deviation. To make that concrete, imagine a 100-point test with a 9.2-point standard deviation. The average student in a computer-using classroom would score 1.7 points lower than her peers in a computer-free room.
Another way to get a handle on the magnitude of this effect is to compare it to other ways one could improve student learning. Based on prior studies assessing the impact of teacher quality at the high-school level, the study authors suggest that "removing laptops and tablets from a classroom is equivalent to improving the quality of the teacher by more than a standard deviation."
While the paper delivers a clear verdict on which classroom policy generated the best average test scores, a variety of important questions remain open.
First, what is it about computer use in classrooms that interferes with student learning?
The authors consider several possibilities. The first — and most obvious — is that students aren't necessarily using devices for course-related tasks. It's no surprise that spending class time surfing the Internet could be associated with inferior learning.
A second possibility is that even legitimate course uses — such as taking notes — could be less effective on a computer. A study by Pam Mueller and Daniel Oppenheimer, published in 2014, found that students who took notes on a computer tended to copy down what they heard verbatim, without engaging in the deeper processing required for conceptual learning. On subsequent tests, students who had taken notes by hand outperformed those who used computers.
Finally, the authors speculate that teachers could be teaching differently, or interacting differently with students, when devices are present. If this is the case, it's possible the teachers themselves don't notice that they're doing so.
Identifying the mechanism behind detrimental effects could help address another important question: whether there are contexts in which computer use can instead be beneficial.
In the current study, computers weren't integrated with the instruction. Students who used computers did so for their own purposes, be they legitimate (checking a textbook) or not (checking a text message). When technology is instead integrated with class activities, perhaps the advantages outweigh the costs. It's also possible that the effects of computer use depend upon the type of material being learned, or that having a computer during class is genuinely beneficial for certain kinds of students, even if the average effect is decidedly detrimental.
These caveats aside, the findings to date suggest that banning computers from classrooms may be the most sensible policy to adopt, and in many cases it likely is. But there's a more general policy that's also worth keeping in mind: the policy of constant experimentation and improvement. We can't always approximate an ideal RCT, but we can stifle opportunities for progress by applying a policy uniformly, without variation. We can only inch our way beyond the status quo by trying out different options, subjecting them to rigorous comparison, and repeating as needed. That means entertaining a new suite of potential policies, not just discarding the superseded.
Tania Lombrozo is a psychology professor at the University of California, Berkeley. She writes about psychology, cognitive science and philosophy, with occasional forays into parenting and veganism. You can keep up with more of what she is thinking on Twitter: @TaniaLombrozo