Case study: Babbel's North Star
How Babbel reestablished product-market fit by finding a new North Star Metric.
Geoff Stead describes himself as a Learning Technologist and inventor of products. From 2018 to 2022 he was Chief Product Officer for Babbel, the Berlin-based language learning app. Babbel has sold over 16m subscriptions and was named as the most innovative learning company by Fast Company in 2023. Geoff shared with me the journey that he and Babbel went on to redefine their North Star.
“If you work in an EdTech business focussed on learning, the most important thing you can do is find a metric that represents that learning impact,” says Geoff Stead, Babbel’s ex-CPO. “But it also needs to connect with your commercial business drivers to truly become a unifying force.”
Geoff is reflecting on the journey that he and Babbel went on from a “sales-oriented business,” to one focused on driving learner outcomes and ongoing innovation.
“The business I joined had millions of learners. It was well established… but growth had stagnated,” says Geoff. “We had a strong focus on our marketing engine. The internal language of the business was oriented around finding new customers, rather than improving the learning experience or encouraging users to engage more deeply.”
Babbel was founded in 2007 when the co-founders who had recently exited a music platform business went looking for online options to learn Spanish and found none. Spotting this opportunity, they decided their next mission was to use the internet to help people learn languages.
They started by licensing textbooks, creating digital versions and charging people for access. As well as being one of the first language learning websites, they quickly became one of the first internet content businesses to make a paywall work. In 2010 they were also early into mobile and apps.
Geoff joined in 2017 when a new breed of language learning apps like Duolingo and Busuu were starting to become established and provide stiff competition. “They were getting better and better… and were free,” says Geoff.
So in the face of this new threat, how did Babbel reinvent itself and reestablish their product-market fit? The answer was in finding a new North Star.
Recognising their strengths
“One of the USPs that Babbel had - and still has - is a lot of highly qualified educators and linguists who love languages and deeply care about the quality of the learning and the linguistic approach. And ensuring that it works for our learners,” says Geoff.
“Most of our app competitors at the time were not like that. Many only covered the lower levels of language ability. And in linguistically eccentric ways because they were using computer powered randomisations.”
“This approach may feel fun, but it can actually slow you down as a learner, making it harder to build more complex sentences and to have meaningful conversations. We knew our product was a better way to learn, but struggled to land this message with our customers.”
After some deep-dive user insight gathering, Geoff and the Babbel team came to realise that they were selling the wrong product.
“Learners didn’t pick Babbel for specific features,” he says. “Learners were picking Babbel because we helped them to fulfil a very personal dream of speaking another language. They saw us as the expert guides who could get them there.”
Redefining the North Star
Most of the metrics the team were tracking at the time were typical consumer commercial metrics. Number of sales, cost of acquisition, size of packages sold… “the typical things you would see in an online publishing type business” says Geoff.
He describes his initial job as helping the wider business understand that product engagement in the long term equaled healthier business, even if in the short term it didn't equal increased revenue.
“We were a very metric, data driven business, so we needed clear evidence to give us the confidence to shift from a directly commercial metric towards one that modelled learner impact.”
Geoff describes how they iterated through a series of different North Star metrics as they built stronger proof points. “We knew we wanted to focus on the actual learning achieved,” he explains, “but we didn't have the data to start there.”
So they began with a general measure of activity: Weekly Active Users. This helped them to increase engagement, but didn’t incentivise individual learners to build a healthier learning habit. They carried on exploring.
“We were looking at ways to encourage the same users to come back repeatedly,” says Geoff. “We knew learning works better if you make it a regular habit, so our next metric was Seven Day Return to Learning, which was about an individual coming back within the week.”
They kept going. “Once we were getting more of our learners to return more often, we started looking at the total time that an individual had spent actually learning,” says Geoff. “We ended up with the learner as the hero and the time they spent learning. And tying it all the way back to revenue. That became our overall company north star metric: Lifetime Learning Minutes.”
Connecting engagement with revenue
To get buy-in for the approach it was important to show that as well as driving learning, it also drove revenue. So how did they do this?
“It’s difficult in a business where users buy long term packages, maybe three or six-months, but then don’t use it after the first one or two,” he recalls.
“This pattern is quite common with aspirational, behaviour change products, like joining a gym. It is not obvious, commercially, why you should make product changes that keep users active when you are paid the same regardless.”
They ran multiple experiments trying to find a solid correlation between learner activity, and revenue.
“Luckily we had tons of learners going through our system, so we could run continual experiments to try and understand them better,” he says. “Our breakthrough was being able to connect app usage in the weeks building up to the end of a contract with renewal rate. The more someone was learning, the more likely they would renew.”
Designing a learning engagement ladder
Once he had gained broad support from all key stakeholders for the approach by demonstrating they could improve revenue, they undertook a piece of work to reflect on the underlying behavioural science and developed a ‘learning engagement ladder’.
“We built a theoretical model to understand learner behaviour a bit like a funnel. How could we move our learners from a cautious visit to a healthy learning habit? We mapped this out like a ladder and then ran lots of experiments against each piece to see whether that stacked up,” says Geoff explaining how they developed and tested their hypotheses.
This started to validate some of their theories: “We found out that if you completed a lesson, you are more likely to come back the next time,” he says, reflecting that completion was more powerful than simple engagement. “And so we would put things into the lesson to try and help you get to that completion moment.”
But there were more surprising findings: “We found that activities that made you speak or ones that helped you imagine yourself speaking in a conversation were more likely to make you come back,” he remembers. “Or even getting the blend right between recalling previous lessons and trying something new.”
These activities became a sophisticated funnel to move people along. “We chained these events together to represent the movement up our engagement ladder,” says Geoff adding that this engagement flow became one of the principles that they organised teams around.
Creating an ecosystem
This focus on learner engagement helped them to think beyond the app.
“Most of the language learning apps hit you at the lowest level of language learning. It's the vocab, it's the very basics,” says Geoff. “And then people struggle when they’re really trying to speak to people.”
This presented an interesting opportunity. Despite doing better at this than other apps due to the involvement of linguistic experts, they realised they needed to provide a more holistic experience.
“We came to realise that our learners really trusted our expertise, and came to us not just for an app, but for guidance on how to master their new language. Our deep language learning obsession could also be a gateway to guide learners into a wider portfolio or ecosystem,” he remembers.
They explored a language travel product that was quickly undermined by the emergence of Covid. They tried language games. They launched a live teaching product that is still going strong. And they launched an experimental podcast.
“We had fantastic success with podcasts, which we initially released as a bit of a sideline experiment but it proved very effective and popular,” Geoff remembers. Babbel’s podcasts have since been downloaded over 20m times.
“We had both free and paid users. What was cool was finding out that - despite being free - at least half of our listeners actually ended up using our app at the same time. It showed us the ecosystem idea really could work.”
He says the really cool thing was that they could see that even learning minutes on free products like the podcast, also had a positive impact on revenue.
Embrace the journey
Geoff says that moving to your perfect North star metric can be a multistage journey.
“It’s about educating people, building new insights, listening to experts,” he reflects. “It really is worth your while to do this because your North Star metric should encapsulate the reason you're all there. If you don't have a clear reason, you just set yourself up to have misunderstandings, misalignments and internal tensions.”
He acknowledges that this isn’t easy and requires tradeoffs. “We definitely struggled to find a single metric that mattered with an equal weighting to everybody, because whatever you pick, there's a reason that that's not the best one,” he says.
On one side, you have the purists who’d like a metric that is absolute proof of learning gains. “But you don’t have all the data so it’s kind of pointless to try,” he reckons. On the other, those that argue that as a subscription business all that matters is engagement. “This is true, but you don't want people just doing something trivial. Otherwise, what's the purpose?”
At the end of the day it’s about pragmatically finding a sensible balance. “You need to take the time to find a really simple, easy to measure but yet meaningful metric that you can hang your hat on,” concludes Geoff.
And for Babbel that was Total Learning Minutes.
Summary
We recap the points that might help others in a similar situation:
Recognise your strengths. For Babbel, that was their educational and linguistic approach that helped them understand and build an engagement ladder and later, a holistic ecosystem.
Find a simple, easy to understand metric that measures the impact you want to make e.g. Total Learning Minutes.
Make sure that it connects to the commercial drivers of the business and reflects learner engagement. It needs to do both.
Understand the behaviours and activities that drive that number and create an engagement ladder.
Keep experimenting and embrace the journey. It is likely to take several rounds to get to the perfect North Star.
As we finish I ask him for a final piece of advice for people working in EdTech.
“You need momentum in the learning tech space,” he says. “There's such a lot of change and so many things happening everywhere. You have to be motivated by learning new things, trying new things, solving old problems in new ways. It's really important to be curiously exploring and moving forward. Nobody knows everything, but you need to keep learning. The thing that you're most responsible for is momentum.”
Geoff has also co-written Engines of Engagement, which the authors describe as a curious book about generative AI. It’s a great read and you can buy it or download it for free on their website.