Is Anybody Watching: A Large-Scale Study of Educational Video Engagement

Overview

This study, leveraging 3.1 million video views from the HMX platform—the largest dataset to date for high-stakes (credit-bearing or certificate-bearing) online learning—revealed a critical gap in understanding educational video engagement. We showed that students regularly engage with individual educational videos over multiple viewing sessions, and that single-viewing session metrics alone significantly underestimate true engagement. This finding directly contradicts the established belief that people do not watch longer videos and has practical significance for how educators develop learning materials. We also showed that factors other than video length, such as the number of associated quiz questions, have significant effects on engagement, and use these factors to develop an accurate predictive model.

Technical Notes

The modeling was intentionally kept simple for this work, given that interpretability was a priority to make the work accessible/relatable to a wide range of educators. We created multiple linear regression models with ridge (L2) regularization that are able to predict whether a video will garner high or low engagement across single and multiple viewing sessions. These models are significantly more accurate than models that rely on video length alone, further substantiating the importance of course context.