UC Berkeley sent almost 25,000 academic-performance alerts last fall

After UC Berkeley’s automated advising system sent 24,980 alerts to advisers last fall, warning of students with poor grades and inactivity in their classes, university leaders said Thursday they want to use this data to better predict students who are at risk.

Berkeley Online Advising is a data dashboard for the school’s 1,000-plus advising staff, which pulls data from the institution’s student information and learning management systems. By attaching alerts to student activity and allowing students to be grouped into cohorts — like engineering majors or first-generation students — advisers can proactively address problems.

Alerts can be triggered by students dropping below a certain grade or failing to participate in a course. By analyzing these alerts and students’ end-of-course performance data, advisers can start digging into what factors can predict student success, Steven Williams, project manager for Berkeley Online Advising, said during Educause’s annual meeting Thursday.

About 42% of UC Berkeley’s nearly 31,000 undergraduates generated alerts during the Fall 2020 semester, he said, with 19% creating more than one alert.

“We need to think of a way to present the alerts on the timeline that isn’t just ‘an alert is an alert is an alert.’ We need to sort of think there are alerts that maybe mean more than others,” Williams said. “I think we know enough today to begin at least starting that process.”

The research examined relationships between student inactivity alerts and midpoint grade alerts to academic standing at the end of the semester.

“The presence of any individual alert isn’t as meaningful as the pattern of alerts,” Williams said. “Once we see that pattern emerge, there’s real risks that that seems to exist for students.”

Williams said the strongest correlation in the analysis was between how many alerts a student received and academic standing at the end of the semester. In the pool of 55 students that received at least 10 alerts, 41% were on academic probation at the end of the semester, he said, compared to 24% of students who received between six and nine alerts.

Researchers will need to continue looking at when beginning to send these alerts is useful, Williams said. In September, researchers found a weaker correlation between student standing at the end of the semester and receiving an alert. Only about 5% of the students who received alerts in September did not pass the course, compared to about 10.5% of students who received alerts in October. That could show that the September alerts might be too early in the semester to indicate outcomes.

Of more than 20,000 students who received an infrequent activity alert, 8.5% did not pass. The analysis broke down the alerts by the length of the inactivity in the course, and the outcomes were similar between as few as 11 days and more than 30 days. The data did not show the courses that produced these alerts, though, and some classes used learning management systems more than others, Williams said.

To get a better idea of how to use these alerts proactively, the university will need to analyze the spring’s data and then compare last fall’s data to the Fall 2021 semester, when more students are on campus, Williams said. The advising system will also be built out to include more data sources, he added.

“This data is really meaningful today — students failing their courses, students on probation, that’s really important for advisers, and that really supports advisers in their work, but it’s also a deficit model,” he said. “Students are more than their grades and student progress is more than their outcome.”