Insights interview: why and how to invest in edtech data for your growth

The business-changing opportunities investments in data can give to edtech leaders

Data is the lifeblood of successful edtech companies. They use it to understand their customers and guide their decision making—to drive product improvements, customer training and support, and investment priorities. And, the most progressive use data-driven research into teaching and learning to power breakthrough products and evidence of impact. However, many edtech leaders I talk with quietly confess that they are struggling to successfully use data in these ways. The reasons range from competing priorities and limited budgets pushing data engineering to the back burner, to a lack of suitable tools and on-staff data talent. I’ve blogged about battle-worn tips and published a White Paper on successful analytics techniques with Dr. Rasil Warnakulasooriya, but I’ve not yet provided insights about these operational challenges.

Insights from an expert

Given the tough decisions facing edtech leaders and product owners on investment priorities, I thought it would be insightful to ask one of the most respected analysts in edtech to reflect on his career-long findings on why and how edtech leaders should invest in edtech data for their future growth and success. Bring on Al Essa who is currently Simon Fellow at the Carnegie Mellon University. Al is unusual because he has experience on both sides of the fence. He has developed data-driven products and managed data-science teams in edtech companies. And, he’s managed IT and data within higher-education institutions. This gives him an unusual perspective as both edtech provider and customer.

The interview

Al, thank you so much for making time for this interview. As you know, I’m passionate about helping edtech startups to grow and big education companies to digitally transform. A critical part of this is helping them to use data more effectively for decision making, innovation, and building impactful products. With this audience in mind, I wanted to explore with you three areas: 1. Why and how to invest in edtech data for future growth, 2. Proven areas for using data to power impactful products, and 3. Promising future areas for data-driven insights and product innovation. Does that sound OK?

Absolutely, Adam. I look forward to the conversation!

Why and how to invest in edtech data for growth

Great. So, let’s start at the beginning. A prominent leader in data once advised me to “Invest in people before you invest in engineering.” What advice would you give the CEO of an edtech startup on what to invest in first to enable future data-driven success?

For an edtech startup, the role of the CTO is, of course, paramount. It is also very likely that one or more of the founders will have a strong background in technology. This does not mean, however, that the CTO will have expertise in data and analytics. 

More generally, does the CTO have experience building and evolving a platform which includes data and analytics as a major component? Does the CTO have data architecture expertise? Or, is their expertise narrower? 

Data architecture is destiny. Technology strategy is destiny. Early decisions will set the stage for future success or failure with data. But so will major decisions along the way. A rigid or inexperienced CTO can sink the company. 

Therefore, my first advice to the CEO: don’t delegate responsibility for the data architecture or the platform to the CTO. Of course, the CEO has to delegate. But my point is, the CEO must take the time to make sure that the platform strategy makes sense. The CEO must also be well versed in technology strategy and play an active role in its evolution. Finally, put checks along the way so that some outside people can objectively review progress and obstacles. 

In short: Trust the CTO, but verify.

That’s very helpful. Next, what advice would you give edtech leaders on areas to prioritize for investing in data and why?

A very common pitfall that I see is how organizations navigate build vs buy decisions. This pitfall applies across the board but is particularly relevant for data and analytics. I would avoid both extremes, namely the tendency to buy everything or to build everything. Surprising as it may seem there are startups that believe that they have to build everything for economic reasons. It’s penny wise and pound foolish. It will bite you back in the long run. 

Here’s a heuristic for the data and analytics component of an edtech platform: Identify elements that are not core to your business vs those that part of your competitive differentiation. Anything which is not core, (e.g. HR and finance data and reports), buy it. Don’t waste time, technology, or talent on what’s not core. Anything which is core, you should most likely build, evolve, and integrate. 

Over my career, I’ve found that “not all data are created equal.” That is, some data are more insightful. What advice would you give to product managers on engineering their edtech to capture and leverage valuable data?

My advice here comes from the world of theater and stage. There is a wonderful book called Thinking Shakespeare by Barry Edelstein. The book is a “how-to-guide for student actors, directors and anyone else who wants to feel more comfortable with “the Bard”.”

The book describes how we should think about words and I think the lesson applies also to how we could think about data.

According to Edelstein, every moment of every scene of every play has three elements: Goal, Obstacle, and Action.

1. What is the student’s goal? Keep in mind, we know that an instructor’s goal for the student probably does not match the student’s goal. 
2. What obstacles does the student have and will have in reaching their goal? Similarly, for the instructor in supporting the student’s goal?
3. What actions are the students taking in reaching their goal and overcoming different obstacles? What actions are the instructors taking in supporting the student’s journey?

There is a famous coastline city in India called GOA. It’s an easy formulaic way of remembering the three data elements which is at the center of learning: G(oal), O(bstacle), A(ction)

Just to elaborate a bit. When I think of a goal in learning, I think of learning objectives. If students are spending lots of time on your platform, then the goals should be granular. The goals should also be dynamic—based on the variety of obstacles and actions. Think of Odysseus’ journey as a paradigm. Odysseus had an overall end goal, reaching Ithaca, but his tactical goals kept changing. 

When it comes to obstacles and actions, the key research insight to keep in mind at all times is that students learn by doing, not by viewing. Carnegie Mellon’s Kenneth Koedinger has published some wonderful research on this. Does your edtech create opportunities for students to learn by doing? Is there a feedback loop in your edtech which constantly assesses where a student is in their journey, what obstacles they are encountering, and how to correct their behavior to get back on track. The data required to power an effective feedback loop also has to be granular and dynamic. One final point. Higher-order learning is not a solo journey. Higher-order learning includes students learning together as they collaborate on solving problems. Does your edtech promote collaborative learning and does it capture trace data surrounding collaboration?

Proven areas for using data to power impactful edtech

Great. Let’s move on to the second topic—data-powered edtech. You’ve worked on both the design of a range of impressive higher-ed edtech products and research into their use and impact. Based on the lessons you’ve learned, what are areas where you’ve found that data-driven features move the needle on learning?

Adam, you know that I am bullish on adaptive systems, particularly ones powered by machine learning. I think that’s an area where I believe there are opportunities in education to move the needle in a dramatic and scalable way. 

Are there systems that do this well at scale? Are there systems which also implement the feedback loop that I mentioned earlier, also at scale? Yes, there are. 

I would take a strong look at products such as Duolingo. They are doing many things well. Their back-end adaptivity is based on solid academic research. Always keep in mind exemplars and learn from them.

And how about areas where data-driven features help instructors to evolve and optimize their teaching?

This is a much harder problem. edtech has overpromised and underdelivered. Instructors are rightfully skeptical and suspicious of edtech companies. But many instructors don’t want to face that “the times they are a-changin’”. But I think a meeting of the minds is starting to emerge.

The real opportunity here is smart technology meeting smart teachers, what is sometimes called augmented intelligence.

I came across an example of this recently. Negotiations is a required topic in most business schools and some law schools. But it is a very difficult and complex topic to teach for a variety of reasons. So, there is a company (iDecisionGames) that has taken on the challenge by automating a lot of the tedium involved with teaching negotiations. But the technology allows the instructor to orchestrate the basic and higher-order learning that needs to take place for mastering negotiations. Best of all the technology is unobtrusive, lightweight, and elegant.

Promising future areas for data-driven insights and edtech innovation

Great. Let’s move on to the final topic—the future. What do you see as the most promising areas for learning analytics and educational data mining to provide new insights into learners and learning, and how could innovative edtech companies leverage these findings?

This is a great time to be an edtech entrepreneur.

My advice is two-fold. Develop some connections with academic researchers. There is a lot of interesting research in learning science that has the potential for being directly applied. An easy way to scout the research is to attend a research conference such as Learning Analytics and Knowledge Conference (LAK). Also pay attention to what’s coming out of places such as Carnegie Mellon University. I realize that for startups time is very limited. But it is important to keep at least one eye on the larger emerging landscape.

And, for improving teaching or educational institutions?

I recommend two books. The first is by Rebecca Henderson, John and Natty McArthur University Professor at Harvard University. It‘s called Reimagining Capitalism in a World on Fire. Her basic thesis is that companies that will thrive as new markets emerge will have to think about their basic value proposition very differently. We need to create “purpose-driven” companies. The second is by Michael Schrage at MIT and it’s called the Innovator’s Hypothesis. Schrage is a foremost expert on innovation. His basic thesis is that the fundamental basis of innovation is not “good ideas” but creating a culture in the firm that rewards iterating constantly around “cheap experiments” and using data from those experiments to guide the business. It means injecting the scientific method. In other words, data is valuable only within the context of experiments. 

Thank you, Al. This has been super insightful.

Adam, thank you. It has been a pleasure.

Takeaways

Al provides several career-earned lessons on why and how to invest in edtech data for growth that resonate with my experiences growing businesses and managing data and insights teams. In particular:

1. Your data architecture foundations are your insights destiny. With a poorly conceived strategy you’ll miss capturing key user data, find future data discovery slow and expensive, and miss the opportunity to establish a customer- and data-driven culture. Make data and analytics foundational to your edtech and have your strategy externally reviewed.
2. Invest in data engineering, systems, and talent that are core to your business operations or competitive difference. Don’t waste investments, time, or talent on what is not (e.g. HR and Finance reporting)—buy them in.
3. Use the Goals, Obstacles, and Actions of your users to guide you on which data to engineer to capture and leverage. I’ve arrived at very similar principles for designing edtech dashboards.
4. Adaptivity has big potential—learn from the best. Identify exemplars in adaptivity and analytics. I always challenge startups with a related question—which competitors do you most admire and why? If you don’t know which products are best-in-class, don’t expect to beat them.
5. Stay abreast of emerging academic research. Build relationships with some leading academic researchers and/or attend learning analytics conferences for inspiration and ideas that might have legs.

If you want a fast and incisive assessment of your current analytics and practical tips for how to accelerate, please contact us today with the form provided below.