Predictive analytics are only as useful as the university culture around them

The ability to forecast how successful students will be in their chosen field, or to proactively offer support to reduce the likelihood of dropping out, is the promise that predictive analytics can offer a higher education industry that’s currently facing a historic decline in student enrollment. But despite recent improvements in machine learning that enable administrators to make data-driven decisions, experts maintain that predictive analytics are only as useful as the academic cultures around them.

The ability to quantitatively predict future success isn’t new or emerging, even within the higher education space, said Randy Weinstein, the vice provost for teaching and learning at Villanova University. Weinstein, an engineering professor, was running statistical analyses 20 years ago with his colleagues to determine how students’ scores on certain SAT sections correlated with their college calculus grades. They used the findings to offer tailored academic support. Similar analyses have for years been conducted outside the classroom to help admissions departments understand how many students they’re looking to enroll or how to market their school to prospective students.

And thanks to improvements in both software and computer hardware, Weinstein said, it’s no longer a challenge for most colleges to hire either in-house data scientists or contract vendors to deliver “actionable insights” for whatever performance metrics an administrator is looking for. Institutions can seek out vendors that specialize in tracking and improving student retention rates, persistence rates (the likelihood that a student returns semester-after-semester), financial stability or, inside the classroom, how likely it is that students will enjoy using one piece of learning software over another.

Over the next several years, universities will have to dedicate even more predictive data analysis toward encouraging students to enroll and keeping them in school once they’re on campus, said Bob Carpenter, deputy chief information officer and associate provost for analytics at the University of Maryland, Baltimore County. Because college enrollment continues to drop nationwide — along with birth rates, possibly foreshadowing even lower enrollment rates in the near future, he said — it will become even more important for vendors to enable universities to track how well certain groups of students are doing. Many vendors, in response, are creating what amounts to a customer relationship management system for the university, Carpenter said.

And the field is maturing elsewhere as well; Blackboard, for example, is using predictive analytics to help universities “nudge” students and instructors toward improving existing academic processes, rather than asking a school to create new programming, according to Jim Chalex, the company’s vice president of product management.

At least a few universities have seen substantial success from embracing institution-wide predictive analytics. Georgia State University, which has 32,000 students, began tracking about 800 factors that it deemed indicative of student success in 2012. The institution claims that it has since watched its six-year graduation rate rise by 23 percentage points to 56%.

GSU and many other large institutions, including Arizona State University, embraced predictive analytics out of necessity — GSU because of a low graduation rate, and ASU because it needed to scale support for its 70,000-person student body without hiring thousands of new additional counselors or academic advisers. But for most schools, Carpenter said, people are the most important element of any type of predictive analytics platform, and cloning the success of institutions like ASU and GSU “is a lot harder than it looks.”

The “personal touch” of predictive analytics, Weinstein said, is critical. Students may be put off by automated messages that say “we noticed your grade was low, you should do X, Y, and Z,” Weinstein said, but they might accept an adviser who uses those same insights to encourage a student during an in-person conversation to take advantage of a university’s support system. Blackboard offers a solution that “augments” university staff to play the role of “retention coach,” Chalex said, an added responsibility that includes reaching out to students and connecting them with existing school support systems proactively.

“The key is how well does it integrate into your campus culture?” Weinstein said. “And how can you use that to leverage everything else you’ve got going on?”

Those support systems only work if university administrators take action in places that predictive analytics suggest might be worthwhile, experts said. What’s difficult, or “where the rubber meets the road,” Carpenter said, is convincing university leaders and staff to change the way they provide academic support in response to any insights gleaned from the data.

“Almost all [higher education institutions] are pretty conservative when it comes to trying new things,” Carpenter said. “And, I mean, there’s some good reasons for that.”

That’s why tinkering with student success has been an incremental journey for UMBC, he said.

Weinstein said predictive analytics are the “low-hanging fruit” of improving student success.

“I think it’s not going to help always, but I think there’s evidence out there that it will help in specific situations,” he said. “It can be a powerful tool to help you be more efficient, solve some more problems that you weren’t able to handle, but I think where the farce is is that it’s an all or nothing, that ‘if we use predictive analytics and it’s going to solve our retention problem.’ No, it’s going to be one of the tools to help you solve the retention problem. And you have to be willing to say that the return of that investment is worth it.”