Balance evidence with innovation, say edtech leaders
Leaders in edtech got candid at a South by Southwest EDU session on Wednesday when speaking about the challenges of leaning into evidence to support product development without standing in the way of innovation.
Varying standards of evidence, the cost-prohibitive nature of research and contradicting perspectives from investors converge to create a difficult balancing act for edtech companies to navigate, said Natalia Kucirkova, an early childhood professor at universities in Norway and England.
“One of the hurdles that many companies have when it comes to research is the cost of it,” said Kucirkova, who also leads WiKIT, a company that helps edtech companies use evidence-based principles from the learning sciences. “High-quality studies, whether it’s a correlational study, an observational study in the classrooms or an experiment costs money and incentives from companies to conduct research are not always aligned with the incentives that they hear from their investors. “
Research is expensive, but there are alternatives companies can consider to analyze the effectiveness of their products, said Olav Schewe, a learning scientist who’s pursuing a doctorate at Oxford University. One alternative, he said, is going beyond general user feedback and diving into randomized samples by interviewing a wide range of users.
Schewe said that it could be “a health push for the entire edtech sector” to lean further into considering evidence in product development.
“If every edtech company, to some extent, thought about evidence, then when you look to competitors or others and what they’re doing you could build on it and it could be innovative, very synergistic for the entire sector,” Schewe said.
The role of evidence is also dependent on the type of market a company is attempting to sell products to, said Ben Wirz, a founding partner at the edtech venture capital firm Brighteye.
“If you’re a parent who’s looking for a literacy tool for your kid, you may be just more interested in whether they like it than whether that particular literacy tool is the most evidence-based,” Wirz said. “When you’re selling to school districts or at a government level, having evidence is really important because the people who make decisions at that level are often looking for evidence.”
Schewe said he’s concerned about requiring early-stage edtech companies to produce high-level evidence that their products are effective.
“If we close the door and say nothing can be tried out in the school unless you have very strong evidence, I’m afraid that could be very damaging for innovation and we need to have a low-barrier for new companies to come into the classroom,” he said.
Adam Sparks, founder and chief executive of Short Answer, a company that makes software to help students learn how to write, shared his experience in attempting to complete a randomized control trial for his product. Despite having the pieces in place to conduct the study and identifying a potential grant, the cost-sharing nature of the grant made it prohibitive for his company, he said, which was still largely relying on grant funding at the time.
“I think we would have been better off having just focused on something more accessible and attainable for us, like a pre-post survey that was fed to our users or building a case study for our website,” Sparks said.
Despite those challenges, Sparks said that edtech companies need to be better at offering transparency.
“Everything works somewhere and nothing works everywhere,” Sparks said. “And I think that just being transparent about that and communicating, being clear with schools we work with about that is just the first important step.”
