Action Lab Reveals New Research On Student Success, Course Performance and Social Psychological Interventions

The Action Lab, a dedicated digital teaching and learning laboratory within EdPlus engages in deep learning analytics, providing continuous program improvement ultimately resulting in student success. The Action Lab translates data into practical classroom applications to improve student outcomes at scale. It’s mission is to make technology-enabled education research useful for systemic, scalable and radical advancement in digital teaching and learning. Some of the fascinating work going on right now in the Action Lab includes the following:

Exploring Open-Scale Access for Less-Educated Learners
Hundreds of thousands of learners have enrolled in ASU’s Global Freshman Academy (GFA), developed by the ASU enterprise and EdPlus creating an opportunity for historically under-represented groups to try college for free and allows those who pass to earn a low-cost-transferable college credit towards a bachelor’s degree. The Action Lab has now examined how learners, who have not completed a bachelor’s prior to enrolling in a GFA course, perform in terms of course completion and whether course completers from this group are more likely to ID verify, pass the course and earn college credit.

In a recently-produced working paper, the Action Lab finds that while those without a bachelor's or higher degree complete at the same rate as those with degrees, less-educated GFA learners are more likely to pass the course and earn college credit than those holding a bachelor’s or above. Factors including education level and country of residence were considered and “committed learners” - those who submitted any graded item after the ID verification date - were isolated for purposes of this study. 

In ongoing work, the Action Lab plans to collect learner socioeconomic status variables and examine how lower socioeconomic students learn in the GFA environment. If you would like to learn more, speak with Elle Wang or Tom Fikes in the Action Lab.

Rethinking Online Efficacy for Underrepresented Minorities
In another recent working paper, the Action Lab addressed the contention that online programs disadvantage historically underrepresented minorities such as black and Hispanic students. Findings from the Action Lab reveal that unadjusted comparisons can lead to incorrect conclusions and that once an appropriate comparison between modalities is constructed, evidence shows that minority students earn higher grades online than campus immersion students on average. 

Specifically, in a population-adjusted comparison of ASU online and campus immersion students, the Action Lab found that both majority and minority male students tend to earn about a fifth of a grade point higher cumulative GPA online than their campus immersion counterparts. The story is similar for female students, though the positive difference in GPA is slightly smaller. Female students from most minority groups also earn higher grades online than their campus immersion counterparts. If you would like to learn more, speak with René Kizilcec or Tom Fikes in the Action Lab. 

Continuous Improvement Analysis Determines Causes for Underperforming Courses
In the case of underperforming ASU Online courses, an unusually high percentage of students who earned a D, F, or W in a given course indicates that something may be wrong with the course. However, this metric does not provide stakeholders with the reasons why the courses are not doing well.

To help address this challenge, the Action Lab is now examining Learning Management System student performance data to disentangle DFW rates based on (a) performance on course assignments and exams and (b) level of engagement in the course. This finer-grained information seeks to better inform stakeholders about potential reasons why students are not doing well in a course so they can in turn apply the most appropriate solution. For example, in one course, the majority of students who earned a D, F or W never submitted a single assignment. This suggests that the low performance is not the result of a course design or pedagogical issue. The more closely we can identify the cause, the more effectively we can apply the most appropriate solution in order to continuously improve student success.  For more information speak with Phil Arcuria in the Action Lab.

Designing Social-Psychological Interventions to Increase Enrollment and Student Success
The Action Lab has just begun a collaboration with Stanford University and Carnegie Mellon University to develop and test scalable social-psychological interventions that steer ASU Online learners to use more effective learning techniques in their online courses. Brief social-psychological interventions have proven effective in shaping student behavior and increasing student achievement by reframing how students perceive social norms, their own potential to grow and goal-related processes. These interventions work by understanding and then changing psychological processes that steer behavior in different contexts. The Action Lab is also collaborating with the EdPlus IT Design and Development Group where similar interventions and A/B testing are ongoing in the GFA courses, with positive results.

For example, rather than merely providing the opportunity for a learner to participate in a learning activity, an accompanying intervention might frame the behavior for them by informing the student of the fact that 85 percent of students who passed the final exam completed this optional exercise. 

In the first year of the ASU Online project, researchers will identify the most appropriate courses in which interventions can be implemented, identify target groups of students, select the most appropriate learning science activities and pilot specific forms of interventions. In year two, they will A/B test the interventions developed within select ASU Online courses to determine their ability to increase the use of learning science-based activities and their effect on student enrollment and grades. For more information, see René Kizilcec or Tom Fikes in the Action Lab.