April 8, 2021 (about 2 years ago)
In April 2020, Oak National Academy was created as a rapid response to the Covid-19 pandemic to keep the UK’s school children learning virtually during lockdown. In February 2021, Oak National Academy reached over 100 million lessons. They also recently announced that their educational content has been ‘zero rated’ by eleven major telecoms providers in the UK.
Zero rating is the practice of Internet Service Providers (ISPs) not charging consumers for certain types of data consumption, or not counting certain types of usage against their data allowance. For many students, in the UK and elsewhere, access to the internet (and to virtual educational content) is only available by using mobile data. But watching hours of video can quickly add up - and quickly deplete the data available on a mobile plan.
Now, with agreements in place with major providers, families across the UK can access all of Oak National Academy’s lessons, without being charged for data or having it be counted towards any data limits.
We sat down with John Roberts, Director of Product and Technology at Oak National Academy, to cover everything from getting the virtual learning program up and running with quality content, to architecting for accessibility, to the product work required to make zero rating a reality, to the implications for net neutrality.
Here’s part one of the Oak National Academy story, in John’s words:
When the UK first went into lockdown, there was a conversation between a number of people who work in education and who’ve worked together on various projects about how we could help with the fact that children were not in school.
Teachers were already building video lessons themselves, but there's a huge inefficiency in trying to do that individually for all of their lessons, and ensuring it’s done in a high quality way.
So we came up with a solution. We brought a significant number of teachers and technical staff together, and focused on building structured lessons that could be delivered via streaming video.
The timeline was really short for this. We committed that we would do it by a particular date - basically the start of the Easter term, and we committed the first line of code... six days before the launch date.
If this had been any other project, there would have been months working on the program side - creating the content, delivering the content, and, on the tech side, managing post production and QA. But here we’re working flat out to get this over the line.
I remember me and our lead developer basically pressing the button at 6:00 the morning of launch and essentially falling asleep and hearing about it on a radio program about 45 minutes later, which was quite an amazing thing.
We had around a half million visits that first day, on a platform basically built on WordPress, so there was a lot of traffic to manage. Partners like Cloudflare really helped to make sure we stayed online with that huge spike of traffic, because, ultimately, you couldn’t really test in the time that we had available.
From there, we rolled out lessons every week, but it became increasingly clear that this was going to be a long-term challenge. Covid was not necessarily going to go away, like we initially assumed would be the case in summer.
We'd had a lot of feedback from users, teachers, parents, about how Oak was working them and that’s when we came up with the idea to build 10,000 lessons.
One of the initial pieces of feedback was that some schools and parents had challenges accessing the video content. That was partly because we were hosted on another vendor and that content is sometimes blocked on parents’ filters on their personal devices and filtered out in schools as well.
And getting video lessons into students’ hands and devices wasn’t the only challenge. Another was the scale of delivery - we were delivering 20 to 25 million minutes of video everyday in lockdown.
The second challenge was getting all of that video ingested. We had a very short period of time over the summer holidays to create 10,000 video lessons, get them ingested into a platform, and code and deliver them across a wide range of devices.
We had a number of conversations with different video providers, and Mux became the provider of choice because of the quality of the API, the testing that we were able to do, and the price point. And just the fact that you were able to be flexible and work with us. We were able to deliver high quality video content in a way that our developers could trust was going to work at scale.
We had about 48 hours to ingest 10 terabytes of video. Because we were able to ingest that over the weekend, we split up about 16 servers to move the videos out of our drive and send them to Mux.
Having the just-in-time encoding in place was critical because it meant that the video was there and ready. It was available on the Monday morning when we launched - thousands of videos and thousands of lessons available straight away.
Now the world's changed a little bit since we first started to spin this up. I think we thought we’d have the Monday off and we'd all come back, and everyone would go back to school. But that didn't turn out to be true.
So back in the summer, we'd modeled some likely scenarios. One of them was based on Christmas leading to a further lockdown. That was the worst case scenario, and we modeled out what video delivery would be required for six weeks worth of lockdown after Christmas.
All things pointed to schools reopening and the platform continuing to be relatively well used, in the order of hundreds of thousands of users, not necessarily in the order of millions of users per day. Then instantly, we were hitting that worst case scenario, in terms of what was required from Mux's hosting delivery, and Oak’s hosting and the platform stability itself.
On the 4th of January, usage skyrocketed. We’ve been able to scale from a modest amount of traffic to what is now a very significant amount of traffic, with significant spikes throughout the day. There is a very clear curve of when pupils are doing lessons and watching videos - mid morning. There's a huge spike around 10:30, 11:30. With Mux, we’re able to scale with that and haven't seen any significant issues going from a relatively modest amount of traffic to huge scale very, very quickly.
We have a very small tech team internally, building the glue to hold this all together and keep the platform and the product online. And your team has been at our side essentially maintaining the stability of the platform, ensuring it does what it's designed to do.
One of the key ideas was to get to a world where most things are unblocked on school networks. We had initial conversations originally early on, about “down the road it'd be good to also get on a custom domain” so we can make sure the video lessons are allowed to be listed on school networks.
But, that really all changed when everyone started doing it from home. The idea of custom domains and delivering over certain paths became a different requirement.
Most school environments have content filters blocking content that's not appropriate. That means if you have third party content, it can be incredibly difficult to list videos, particularly if you're hosting on some of the well-known providers. What we wanted to do was have a domain that was ours that would allow us to make sure that the content isn’t blocked by the school’s firewall.
Another big requirement that drove at this need for change was that there are about a million people in the UK for whom it's unaffordable to access the lessons because of the data charges that are in place.
From a data bandwidth perspective, delivering an Oak National Academy video lesson has a significant charge associated with it. And if you're on a pay-as-you-go contract, or pay-as-you-go SIM card, that will use your bandwidth day-to-day and low-income families in the UK struggle to pay the charges associated with those lessons.
So we started looking out for things that we could do, people that we could work with - mobile network operators (MNOs) - to find ways to solve this challenge and ensure children could access lessons and educational resources without being charged for that mobile data.
That came down to solving the same problem of traversing the school content filters and firewalls, which meant posting the video content and all the rest of the content on a single domain - having the apex domain off Oak National Academy, and then being able to host everything on either a sub-domain or on that apex domain.
It's taken a lot of time to get to that point. Our internal tech team essentially rebuilt the front end of our Google Forms and Slides so we can scrape all of that content and redeliver it from our own domain. It’s also been a huge piece of work for Mux to move a global architecture and global infrastructure for video delivery onto a domain that we can use.
Finding a solution within our timeline has already been impactful for people who can now access this content as a result of the work Mux and our internal team completed, as well as the mobile network operators that worked with us to zero rate the domains.
We've ended up in a really solid position where people with sim cards that are delivering mobile broadband to that device do not have to pay for videos and lessons. Now it means that there's a million people that are able to afford it, because it’s free. That's really exciting and really important in a world where a single lesson could be a couple of hundred megabytes of someone's data allowance. You can imagine how it goes really quickly, and then you’ve burned through five gigabytes a month. That's not many lessons when you add it all up.
There are also challenges around accessibility of devices as well. Isn't necessarily where we think it is.
A lot of families still might have one computer and maybe two or three kids. Then someone's going to have to learn on a mobile or tablet while the other kids may be on the computer, if that’s even available to them.
There was a long conversation about what resources can we get on, say, PlayStation browsers as well? Because having a gaming device instead of a computer might be common in some households.
The concurrency of PlayStation for users on our platform is something I've looked at before, and you see people using PlayStations all at the same time to view lessons. It’s great to see, and we're like - wow - does it work on that?
That’s the thing, every device out there is challenging, but we’ve built this on the fundamental principle of trying to make accessible lessons and make sure that no child misses their education due to COVID-19.
That comes down to two things, accessibility of device and building a platform that is accessible to the widest range of devices possible. And secondly, accessibility in terms of the learners’ needs as well. There are captions on every video by default and the research seems to suggest it has a positive impact on literacy. So when we’re doing tasks as part of our build process, we’re making sure that we're hitting the standards that are required to meet the Web Content Accessibility Guidelines (WCAG).
We needed to make sure that everything was served from a subdomain of Oak National Academy. So essentially we've provided mobile network operators (MNOs) with a rule that they can apply that allows that traffic to be listed, or zero rated, if it falls within a subset of one of our domains. Our video content is delivered through the content delivery networks (CDNs) of Mux, but using a subdomain of video.thenational.academy and we've delegated that subdomain to Mux to allow them to manage the content delivery network that sits underneath. Meanwhile, we control the rest of the subdomains there. We focused on classroom.thenational.academy and www.thenational.academy as the main subdomains, but essentially we’ve removed all third party requests across our platform.
That includes things like sending data back for Google Analytics or even some of the other providers we use, like New Relic or Bugsnag, to make sure that the data is zero rate-able. We've had to make sure that the data is all being proxied or being delivered through one of our subdomains.
The technical challenge is working with the MNOs. Modern internet architecture does not rely on IP addresses in the same way it used to, particularly with CDNs. And so doing allow listing by an IP address - a single dedicated IP address for a dedicated server that serves all the content - is just simply not viable.
The architecture that sits underneath both our side of the platform and the video side is challenging, and it's taken some time to bring all of that together, but we got there. Ultimately, the collaboration between the various parties and just the mission of trying to make this accessible to people has been engaging and inspiring.
Because we’re a really small team internally, to get to that point where we’re quickly rolling new serverless functions or workers out, and moving data in the right way so that it's not hitting third party domains has been a real journey and a lot of hard work. It was great that Mux was keen to try and make this happen for us - so a huge thank you from all at Oak National Academy. There's a lot of gratitude for making that happen. Everyone was here to get this traffic zero-rated so that we can enable as many kids to have virtual lessons at home as humanly possible.
I don't think there is another solution that we could have used actually to do this other than self-build. And I didn't want to go down that route.
Oak National Academy’s story doesn’t end here - but this post does! Stay tuned for the next installment of this two part series, where we’ll dive into the technical challenges of achieving zero-rating, and the product work that went into making it a reality. Spoiler alert: that part took longer than six days.
No credit card to start. $20 in free credits when you're ready.
Hack Club is an organization of high school students across the globe. To adapt with school closers they made 2020 the "Summer of Making" and are using Mux Video to power the summer scrapbook.
By Dylan Jhaveri
When 2020 pressed pause on live events creators had to get...creative. Maestro’s Ari Evans talks about developing an interactive streaming platform for the creator economy to stay connected with fans.
By Lynnette and Ed
Every organization struggles with balancing user support quality and efficiency. The typical solution is to scale the team up, by hiring more people, and this is absolutely a necessary step… to an ext ...
By Alex de Chaves