A new study by a team of University of Notre Dame researchers reveals that electronic portfolios can be quantitatively and qualitatively analyzed to generate learning engagement analytics that indicate the level to which students are surviving or thriving during their first-semester Intro to Engineering course. This information can provide instructors, teaching assistants and advisers critical early warning signs that can help with STEM retention.
Electronic portfolios, or ePortfolios, are digitized collections of material including demonstrations, resources and accomplishments that represent an individual, group or institution.
Nitesh Chawla , Frank Freimann Collegiate Associate Professor of Computer Science and Engineering and director of the Interdisciplinary Center for Network Science and Applications (iCeNSA), pointed out that ePortfolios have been used in higher education for more than a decade for classroom and program assessments and to support the accreditation and professional licensing of degrees in fields such as education and nursing. More recently, higher education institutions have been using ePortfolios with career services to prepare students for professional transitions.
“Notre Dame has become a leader in the last three years in integrating ePortfolios into the advising process to improve the mentoring process and relationship and help students integrate all their educational experiences in, across and outside of classes throughout multiple years,” G. Alex Ambrose , coordinator of the University ePortfolio Initiative and assistant professional specialist in the Education, Schooling and Society department, said.
In a best paper runner-up presented at the recent Conference on Learning Analytics and Knowledge, Chawla and his fellow researchers showed that, given the very nature of how ePortfolios are used at Notre Dame, they can be analyzed to measure student engagement levels providing a new digital learning environment opposed to traditional learning management systems.
“Given a series of quantitative features extracted from each students’ ePortfolio describing their interactions with that tool, we are able to provide an estimate of how engaged each student is to their academic program and with respect to his or her peers,” Everaldo Aguiar , a doctoral student of Chawla and Jay Brockman , associate professor of computer science and engineering and associate dean of engineering, said. “Furthermore, we also showed that there is a significant correlation between these metrics and retention outcomes. That is, students who are described as being highly engaged, based on these ePortfolio-based features, are more likely to be retained in the engineering program than those who display lower levels of engagement.”
The information is especially helpful when incorporated into the design of early-warning systems.
“We showed that the predictions made by models that had been augmented with ePortfolio-based engagement features were significantly more accurate than those made by models that did not consider that information,” Chawla said.
Although their paper focused on engineering education, the researchers believe there are many areas where the use of ePortfolios would prove beneficial.
“Our work is ground-breaking in this topic from the integration of various types of data,” Chawla said. “We think there are several areas that can be explored and potentially useful for the next generation of learning analytics by both qualitative and quantitative analysis of data. We can then begin to answer questions about retention, engagement and even success.”
The paper was prepared in collaboration with Notre Dame’s First Year of Studies program. The authors include Aguiar, Chawla, Ambrose, Brockman and Victoria Goodrich , an assistant professional specialist in the College of Engineering . The research was supported in part by the NSF STEM Enhancement Program ( STEP ) grant, titled “ "Finding Your Vocation in STEM ":http://stepcentral.net/projects/251 .”
_ Contact : Nitesh Chawla, 574-631-8716, nchawla@nd.edu _