Evidence-based Tips For Choosing Online Learning (Part 2)

(originally posted on LinkedIn) In part 1 of this series, I went over the importance of defining your goals and understanding motivational elements in successfully choosing online courses. In part 2 (this article) and part 3, I will go into elements that help take your learning to the next level in online courses. In part 2, I’ll cover assessment and active learning, and in part 3, I’ll cover visuals and multimedia as well as use of real-world or authentic examples.

Experts in cognitive science and related areas have made enormous strides in understanding what makes for more effective and efficient learning, and most of those techniques are very well-suited to the online setting. If one of your goals is to come away with learning that stays with you, look for courses that bring some or all of the following features or practices to the table.

Learning Experience vs. “Resource Bank”

Many online learning sites position themselves as collections of resources, or “resource banks,” but there are distinct benefits to a guided learning experience. Individual videos that have little relation to each other can work well for focal, bite-sized topics, but fall short when trying to learn a broader topic. A well-designed learning experience can use the overall course structure to create an arc, where later material builds on earlier material, allowing you to progress to higher levels of Bloom’s taxonomy [1] rather than getting stuck at the base of the pyramid (figure 1). Making connections between different parts of an overall subject helps establish and strengthen neural pathways that ultimately give you more ways to access the material from memory later when you need it.

Figure 1 - Bloom’s Taxonomy (adapted by Katie Yost, HMX Online Learning team). Bloom's taxonomy pyramid, adapted. From bottom to top: Remember, Comprehend/Understand, Apply, Analyze, Evaluate, Create.

One of my branch points early on in starting Harvard Medical School’s HMX online learning team was whether to approach medical science topics from the standpoint of creating full learning experiences, packaged as online courses, or collections of individual resources. Most of my earlier professional life had been spent creating hundreds of unique interactive diagrams and simulations to illustrate difficult medical concepts, so I am no stranger to the idea of creating collections of resources. But one of the lessons I took away from that earlier experience is that, as powerful as individual resources can be, the greatest learning and the highest impact on people’s lives could be had from creating coherent learning paths that take advantage of the learning arc mentioned above, including a longitudinal assessment strategy and active learning.

Assessment and Active Learning

Many of us remember staying up late to cram for a test. Unsurprisingly, knowledge gained in this way fades quickly. Fortunately, there are proven techniques that can lead to lasting learning with the same overall time spent. These techniques are best when embedded in the arc of a learning experience and include the principles of spacing (or spaced repetition) and interleaving (both described in a highly relatable way in the book Make It Stick [2]), and the testing effect more broadly. Spaced repetition refers to re-assessing knowledge consistently over time, in contrast to massed practice (otherwise known as cramming). Interleaving refers to interspersing different types of problems in your practice, as opposed to bunching up practice of the same type of problem. The “testing effect” (also known as retrieval practice [3], [4]) refers to the ability of testing to enhance learning; rather than simply evaluating the state of your knowledge, the act of taking a test or quiz actually has been shown to help you learn the material better.

In fact, Bjork et al [5] coined the phrase “desirable difficulties” to describe the phenomenon that certain learning activities that feel challenging—and potentially even unintuitive in terms of efficiency—such as those outlined previously (spacing, interleaving, retrieval practice) lead to more durable learning.

While there is overwhelming empirical evidence that learning works this way, mechanistic explanations have been more difficult to find, given the complexity of studying the brain. Excitingly, recent research in animals provides evidence that initial memory formation may act as a “priming” event, facilitating enhanced gene expression that leads to strengthening of memories in response to later recall events [6].

Given that online learning is ideally suited to helping you take advantage of these types of approaches, it is surprising to still see some online courses almost entirely composed of videos. Although this type of passive watching experience can feel good and seem like effective learning, particularly if the presenter is clear and engaging, this approach in isolation falls short unless it is combined with “active learning” such as assessment [7]. Recognize that good learning requires activity and effort; in online courses that you are considering, look for this in the form of frequent quizzing or projects and don’t shy away from it even if it feels less comfortable than “watching TV.”

I can attest from my own learning that assessment and spaced repetition have made a world of difference in retaining material; I still enjoy hearing from talented teachers as part of learning but have internalized the need for practice to solidify the knowledge and move it to a place where I can access it in the future.

An added benefit of being able to check your knowledge frequently during an online course is greater insight into how you learn effectively. I’ve heard from students that the experience of watching a video, only to find themselves unable to answer relatively straightforward questions immediately afterwards, helped them directly understand the necessity of retrieving and putting material into practice to drive it home.

Summary

In summary, look for evidence of the following features or practices in courses that you are evaluating:

Learning experience vs. resource bank - Is there a learning arc? Is this a course or a collection of videos? See if the overall outline (if there is one) makes coherent sense to you.
Active learning, with retrieval practice including spacing and interleaving - Are assessment questions a key component of the course? Does the course involve activities that require you to engage with your nascent knowledge rather than just sit back and watch?

In part 3, we’ll cover visuals and multimedia—what you should look for in online courses to get the most out of them for your learning.

References

[1] - Bloom, B. (1984). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive Domain. Boston: Allyn and Bacon.

[2] - Brown, P. C., Roediger (III), H. L. and McDaniel, M. A. (2014) Make It Stick. Harvard University Press.

[3] - Roediger, H. L. and Karpicke, J. D. (2006). Test-enhanced learning: taking memory tests improves long-term retention. Psychological science, 17(3), pp. 249–255.

[4] - Karpicke, J. D. and Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), pp. 772–775.

[5] - Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing about knowing, pp. 185-205. Cambridge, MA: MIT Press.

[6] - Marco, A., Meharena, H.S., Dileep, V. et al. (2020). Mapping the epigenomic and transcriptomic interplay during memory formation and recall in the hippocampal engram ensemble. Nat Neurosci 23, 1606–1617.

[7] - Freeman, S. et al. (2014). Active learning boosts performance in STEM courses. Proceedings of the National Academy of Sciences, 111(23), pp. 8410–8415.