The team’s project focuses on finding whether personalized learning tools and resources have a positive impact on student outcomes in undergraduate mathematics courses.
BY DANIELLE HENDRIX '15 | JANUARY 10, 2024
(Photo via Charlotte May, Pexels)
Everyone is unique in how they learn, but not all learning formats are suited to be conducive to the academic needs of each student.
That’s where personalized adaptive learning comes in, taking a software platform approach that provides each student with an individualized learning experience and unique pathway through course content based on their knowledge, skills and learning needs.
It’s also the subject of Debbie Hahs-Vaughn’s latest research endeavor, a meta-analysis (or study of previous research) of personalized adaptive learning in undergraduate mathematics. Hahs-Vaughn — a professor in the Department of Learning Sciences and Educational Research’s methodology, measurement and analysis program — is the principal investigator on the project, which is funded by a $300,000 grant from the National Science Foundation. She and a team of fellow UCF researchers are analyzing prior research on personalized adaptive learning with the goal of determining whether it is indeed effective in improving undergraduate students’ mathematics performance.
Personalized adaptive learning can be implemented in various ways, from using publisher-created tools to having faculty use content agnostic platforms to develop the personalized learning course content and assessment metrics. Courses can be designed in such a way that there are various learning pathways available to cater to different skill levels. For some personalize adaptive learning, students’ knowledge is assessed at the beginning of a module and from there they are placed into their designated learning pathways. As the student interacts with the content, the learning system adapts and updates.
“The idea is to make it customized to each individual student, so it really meets them where they are, rather than a more traditional method of teaching in which the course is designed toward the middle of the class,” says Patsy Moskal, co-principal investigator and director of digital learning impact evaluation at UCF. “We think personalized adaptive learning makes a difference in the quality of education and believe it has helped us at UCF, but we're just one piece of the puzzle. This grant helps us look at many more pieces.”
The team began its meta-analysis in Spring 2023 with a search of 13 different databases, during which it worked with UCF librarian Corinne Bishop to determine and execute comprehensive search strategies. The search resulted in nearly 13,000 studies that met their preferred criteria. From there, the team began screening abstracts and titles to filter results based on eligibility criteria, such as searching for studies conducted in the context of education rather than machine learning.
Now, the team is moving on to the next phase of the project, which consists of screening, data extraction, and analysis of the full texts of the roughly 260 studies that made the cut to determine the efficacy of personalized adaptive learning.
“The way that I look at personalized adaptive learning is that it allows students to practice and build their skills so they can get to where they need to be,” Hahs-Vaughn says. “There is some mixed research on personalized learning, but generally I think the consensus is that it seems to be effective. However, nobody has specifically done a meta-analysis like we're doing at the intersection of undergraduate math and personalized learning. We’re asking questions about whether it is effective, as well as to what extent or in which conditions it is effective.”
Through their analysis, Hahs-Vaughn and her team hope to find that personalized adaptive learning has a positive impact on student outcomes in undergraduate mathematics. The results could help increase undergraduate student success in mathematics, undergraduate student retention and degree completion, and access to career opportunities and economic stability.
“Undergraduate math courses often have high DFW rates, which is the percentage of students who receive a D or F, or those who withdraw,” Hahs-Vaughn says. “It’s especially important for students in STEM majors that they understand the material and can be successful. If they can't get past that first class, it’s going to be hard to progress in their anticipated major.”
“Especially as universities try to move the needle on student success, anything we can do to eliminate roadblocks and challenges for students is critical,” Moskal says. “Ensuring student success within individual courses keeps them on track toward their desired degree and also improves university retention rates.”
By the end of the grant period, Hahs-Vaughn and Moskal will have a complete meta-analysis that they hope will be helpful in advocating for personalized adaptive learning components. In addition to the meta-analysis, they also are writing a paper to assist researchers in understanding resources for best practices in conducting a meta-analysis.
“We’re going to have a final report, but we also hope to publish and present our results to others in higher education, whether they want to expand those efforts or get involved with personalized adaptive learning,” Moskal says. “We want to help inform the future of personalized adaptive learning and where we go next through this research.”
Other UCF contributors include co-principal investigators Tammy Muhs, senior lecturer of mathematics in the College of Sciences; and Katiuscia Teixeira, assistant professor of mathematics in the College of Sciences. Additional team members include Oluwaseun Farotimi — a PhD candidate in the methodology, measurement and analysis program — and Christina Carassas, a master’s student in the industrial and organizational psychology program.
Hahs-Vaughn is a professor and academic coordinator in the College of Community Innovation and Education’s Department of Learning Sciences and Educational Research. She received her doctorate in educational research from the University of Alabama. Her research focuses on methodological issues associated with applying quantitative statistical methods to survey data, using it to answer substantive research questions, and program evaluation.
Moskal is the director of digital learning impact evaluation in UCF’s Division of Digital Learning. She received her doctorate in curriculum and instruction from UCF. She specializes in statistics, graphics, program evaluation and applied data analysis. Her research focuses on the impact of digital technologies toward improving student success.