The growing popularity of Data Science courses demands more research on their pedagogy and assessment practices. Based on insights gained from the analysis of students’ programming assignment submissions in a Data Manipulation course (offered in an online Master of Applied Data Science program), this project explores the common errors and misconceptions that learners encounter while engaging in notebook-based introductory data science programming exercises.

We are now collaborating with Microsoft to build a human-AI collaborative tool that generates feedback on students’ data science programming mistakes. A pilot of this tool was deployed in the Fall’22 and Winter’23 offerings of the Data Manipulation course.

Project Leads: Anjali Singh and Christopher Brooks
Collaborators: PROSE team at Microsoft

Learnersourcing, a form of crowdsourcing where learners engage in learning activities while contributing useful inputs for other learners, is emerging as an effective technique for the generation of teaching and learning resources at scale. In this project, we take a student-centric approach to understand how we can design learnersourcing systems where students value the learnersourcing task, learn deeply, and generate high-quality output. 

In Fall’20, we conducted a large-scale field experiment in the MOOC Introduction to Data Science in Python to study the effects of learnersourcing on students who create Multiple Choice Questions (MCQs) and their motivations for engaging in question generation.

We are now exploring the use of learnersourcing for generating feedback messages by prompting students to reflect on their mistakes in data science programming assignments.

Project Leads: Anjali Singh and Christopher Brooks

Smart home technologies are transforming childhood profoundly by shifting children’s play, learning, and social interaction to the digital realm. This development introduces privacy and safety risks for children. Smart speakers, wearables, even smart vacuums, capture data about children and their families for powering services and experiences, profiling, analytics, and commercial purposes – often without transparent communication of data practices. Children’s healthy development and self-identity rely on a safe and privacy-friendly (smart) home environment. 

This line of research aims to understand children’s understanding of smart home related privacy and safety risks and explore age-appropriate ways to SUPPORT children and parents’ understanding and management of such digital privacy and safety risks.

Project leads: Kaiwen Sun, Chris Brooks, Florian Schaub

Computational notebooks enable data scientists to document their exploration process through a combination of code, narrative text, visualizations, and other rich media. The rise of big data has increased the demand for data scientists degree programs in colleges, as well as data science MOOCs. In data science classrooms, instructors use computational notebooks to demonstrate code and its output. While learners navigate example notebooks to enhance their perceived knowledge from watching video lectures, or create their own notebooks for assignments or capstone projects. In the Educational Technology Collective, we reflect on the current practice of computational notebooks in data science courses and explore the opportunities and challenges of better adapting computational notebooks for data science education. In particular, we have several projects on redesigning computational notebooks for supporting real-time group collaboration, facilitating joint discourse over shared context, and encouraging active learning.

Project leads: April Wang, Christopher Brooks, and Steve Oney
Collaborators: Anant Mittal and Zihan Wu

Self-regulated learning (SRL) is the process where a learner monitors and controls metacognition, cognition, motivation, affect, and contextual factors to achieve goals ​(Boekaerts & Niemivirta, 2000; Greene & Schunk, 2017; Pintrich, 2000; Winne, 2001; Zimmerman, 2000)​. SRL process includes numerous goal-oriented actions from building strategies to tackle a given task to evaluate learners’ performance. Researchers who study SRL should carefully align their study design with these common features, regardless of which models they use as a fundamental theoretical base for the study. In particular, alignment between measures and theory is important, because SRL measures indicate how researchers understand and model SRL processes ​(Greene & Azevedo, 2010)​. Because of the importance of alignment between measurements and SRL models, there have been discussions on how to verify and improve the validity of measurements detecting SRL components. Through the study, I will investigate the validity issues of SRL measures used in previous studies and discuss the current approaches to resolving the issues. Furthermore, I will focus on systems for investigating the validity of trace data and self-report data on learners’ help-seeking strategy usages.

Project Leads: Heeryung Choi and Christopher Brooks

Advances in artificial intelligence, coupled with extensive data collection provide unparalleled insights into determinants of every-day activities. In education this has spawned research on predictive modeling of learning success, which are used to power early warning systems. They can be used to identify students who are at-risk of failing or dropping out of a course, and to intervene if necessary.  However, because the effectiveness of these systems rests upon the collection of student data, including sensitive information and confidential records, this creates a tension between developing effective predictive models while supporting learners’ agency and privacy.  

In our research, we are interested in finding out students’ views on the collection and use of their educational records, as well as their overall privacy perceptions and how this, combined with personal background and academic information, may be tied to one’s propensity to opt-in or opt-out of having their data collected for the purposes of predictive modeling. These findings may impact institutions’ abilities to help identify “at-risk” students, suggest possible intervention strategies, and to contextualize opt-in or opt-out, thereby impacting how they choose to interact with students. We also gain a better understanding of whether students are aware about how their personal data is used, how the use of this data should be managed, and how these decisions impact predictive models’ performance. 

Project Leads: Warren Li, Kaiwen Sun, Florian Schaub, and Christopher Brooks

Whether machine-learned, human generated, or a hybrid, models of student success in education need to be replicated in new contexts and datasets in order to ensure generalizability . We’re building the software to do this, and hooking it up to large datasets of hundreds of classes with millions of learners through collaboration between the University of Michigan and the University of Pennsylvania.

Links for more information

• replicate.education, where we list upcoming events we have for replication efforts
• the MOOC replication framework (MORF), an open-source docker-based enclave system for replication of educational models