Computational notebooks enable data scientists to document their exploration process through a combination of code, narrative text, visualizations, and other rich media. The rise of big data has increased the demand for data scientists degree programs in colleges, as well as data science MOOCs. In data science classrooms, instructors use computational notebooks to demonstrate code and its output. While learners navigate example notebooks to enhance their perceived knowledge from watching video lectures, or create their own notebooks for assignments or capstone projects. In the Educational Technology Collective, we reflect on the current practice of computational notebooks in data science courses and explore the opportunities and challenges of better adapting computational notebooks for data science education. In particular, we have several projects on redesigning computational notebooks for supporting real-time group collaboration, facilitating joint discourse over shared context, and encouraging active learning.

Project leads: April Wang, Christopher Brooks, and Steve Oney
Collaborators: Anant Mittal and Zihan Wu

Advances in artificial intelligence, coupled with extensive data collection provide unparalleled insights into determinants of every-day activities. In education this has spawned research on predictive modeling of learning success, which are used to power early warning systems. They can be used to identify students who are at-risk of failing or dropping out of a course, and to intervene if necessary.  However, because the effectiveness of these systems rests upon the collection of student data, including sensitive information and confidential records, this creates a tension between developing effective predictive models while supporting learners’ agency and privacy.  

In our research, we are interested in finding out students’ views on the collection and use of their educational records, as well as their overall privacy perceptions and how this, combined with personal background and academic information, may be tied to one’s propensity to opt-in or opt-out of having their data collected for the purposes of predictive modeling. These findings may impact institutions’ abilities to help identify “at-risk” students, suggest possible intervention strategies, and to contextualize opt-in or opt-out, thereby impacting how they choose to interact with students. We also gain a better understanding of whether students are aware about how their personal data is used, how the use of this data should be managed, and how these decisions impact predictive models’ performance. 

Project Leads: Warren Li, Kaiwen Sun, Florian Schaub, and Christopher Brooks

Self-regulated learning (SRL) is the process where a learner monitors and controls metacognition, cognition, motivation, affect, and contextual factors to achieve goals ​(Boekaerts & Niemivirta, 2000; Greene & Schunk, 2017; Pintrich, 2000; Winne, 2001; Zimmerman, 2000)​. SRL process includes numerous goal-oriented actions from building strategies to tackle a given task to evaluate learners’ performance. Researchers who study SRL should carefully align their study design with these common features, regardless of which models they use as a fundamental theoretical base for the study. In particular, alignment between measures and theory is important, because SRL measures indicate how researchers understand and model SRL processes ​(Greene & Azevedo, 2010)​. Because of the importance of alignment between measurements and SRL models, there have been discussions on how to verify and improve the validity of measurements detecting SRL components. Through the study, I will investigate the validity issues of SRL measures used in previous studies and discuss the current approaches to resolving the issues. Furthermore, I will focus on systems for investigating the validity of trace data and self-report data on learners’ help-seeking strategy usages.

Project Leads: Heeryung Choi and Christopher Brooks

Network analysis in educational research has primarily relied on self-report and/or data generated from online learning environments (e.g. discussion forums). However, a large part of social connections amongst students occurs through day-to-day interactions on campus. This research project explores the application of spatial-temporal network data to model social network structure amongst students on campus at scale. Links between individuals were inferred based on their spatial co-occurrences along a similar temporal dimension (i.e. two individuals connected to the same access point at the same time). We propose a potential approach to test the statistical significance of these connections against a null model, in which two individuals might randomly be at the same place at the same time. 

Project Leads: Quan Nguyen, Warren Li and Christopher Brooks

Whether machine-learned, human generated, or a hybrid, models of student success in education need to be replicated in new contexts and datasets in order to ensure generalizability . We’re building the software to do this, and hooking it up to large datasets of hundreds of classes with millions of learners through collaboration between the University of Michigan and the University of Pennsylvania.

Links for more information

• replicate.education, where we list upcoming events we have for replication efforts
• the MOOC replication framework (MORF), an open-source docker-based enclave system for replication of educational models

Our work focuses on nurturing the development of self-regulated learning (SRL) skills in students.  Self­regulation is an important feature of engaged learners: those who have the ability, willingness, and experience to reflect on how their behaviors relate to learning outcomes. By predicting the academic success of students (PASS) and revealing how that success correlates with their activities, we aim to help learners engage in conversations with themselves, their peers, and their instructors to improve learning practices and outcomes.

In collaboration with the Information Quest team, we’re building supervised machine learning models and additional infrastructure to achieve these goals and bridge the gap between research and production. We aim to support the work of researchers and tool developers who want to make use of predictive models, and to impact the broader landscape of higher education through partnerships such as the Unizin Consortium.

Collaborators: Information Quest (IQ), Josh Gardner