How to Guide: Content Recommendation & Discovery
Alan Wolk, Lead Analyst at KIT digital, tells a funny story about how, at its most basic level, content discovery and recommendation functionality works in many video services:
"I have two kids, a boy and a girl, and they come into the kitchen, stand in front of the refrigerator and ask "What do we have for lunch?". I can either read them full inventory of the fridge and pantry, give them recommendations based on what they're previously eaten for lunch or I can help narrow down the categories by suggesting a sandwich or soup and then subcategories like a grilled sandwich, a ham sandwich, a peanut butter sandwich etc."
It's a great analogy and begs the question of whether it's really the most efficient way of doing things. I've been speaking to the BBC, Blip TV and ThinkAnalytics about this topic and here are four options for surfacing and recommending content to your users:
#1. Incorporate off-the-shelf software
Various companies supply specialist, customizable, off-the-shelf content discovery and recommendations software that can be incorporated into your video services. Taboola, for example, works with companies like Fox, Hearst Television and CNN. Rovi is also a lead player in this space. Indeed, Michael Papish of Rovi recently wrote a great piece for us about the art + science approach to building a recommendations engine. For this article, however, I had a chat with Peter Docherty, Founder and CTO at ThinkAnalytics, which includes Virgin Media, Sky and ITV amongst its client list.
As background, ThinkAnalytics develops and markets, in their words, "the most widely deployed content recommendations engine in the market today". It combines personalised search with comprehensive media content recommendations. Its technology has been chosen by over 35 pay-TV providers and its customers have seen uplifts of between 30% and 100% in pay per view, VOD and other purchases, demonstrating a compelling return on investment. So, that's the sell but what about the mechanics? How does the software actually work? Peter Docherty explained:
"Our Intelligent Navigation and Recommendations software helps consumers find content they want to watch, when they want to watch it, and on any device. It makes the content discovery process simpler and easier by proactively being able to present content we know the consumer will like without having to page through the EPG or VOD catalogue, searching for something to watch.
We deliver a sophisticated system by employing a comprehensive range of algorithms, covering areas such as metadata and behaviour, designed to interact in many different combinations to enable multiple recommendations techniques across different devices/platforms. To demonstrate the requirement for a broad array of automated, interacting algorithmic techniques, consider the case of a pay-TV provider with 10 million subscribers and 80,000 programmes on offer across multiple devices; the number of possible combinations here are endless.
The ThinkAnalytics Recommendations Engine has a strong metadata element where we clean up and enhance the service provider's own metadata in order to improve the metadata keywords and tags. This process uses Natural Language understanding techniques across multiple languages to enhance the engine's understanding of the content. In addition, we have also developed ThinkMovies and ThinkTV, the largest sources of pre-tagged metadata for movies and TV, which are seamlessly integrated with our Recommendations Engine."
And there's also a social networking element. In 2011, ThinkAnalytics announced support for social media. This:
"… not only allows learning from behaviour such as Facebook 'likes' and other implicit and explicit feedback but pay-TV providers can also use information such as Twitter trending topics to deliver trending recommendations of content from multiple sources, including VOD, live linear TV and DVR recordings."
#2. Build your own solution based mainly on algorithms
Lately we've seen how powerful mathematics can be in predicting human behaviour. During the recent US presidential election, an analyst called Nate Silver correctly called the results of all 50 individual states and the exact number of Electoral College votes won by President Obama and Governor Romney by studying, weighting and modelling polls. Somebody in the new television industry should hire this guy!
So, understanding that there's usually a human, overseeing, element involved, can you build your own content discovery / recommendations engine based mainly on maths and algorithms? The BBC is one organisation that thinks you can and I asked Dan Taylor, the Executive Editor of the BBC iPlayer, about how they handle the task. We talked via email and I think it would be useful if I post some of the Q&A below. As a précis:
- The BBC is about to launch a new recommendations engine which it developed in-house;
- It used to use a third-party solution to deliver recommendations in iPlayer but switched to a BBC-built product in order to deliver a more tailored and responsive solution;
- It plans to build on the Favourites functionality within iPlayer to deliver more personalised recommendations
KANJI: How does the iPlayer 'do' recommendations and discovery? What are the advantages / disadvantages of this method for you?
TAYLOR: BBC iPlayer aims to offer users a range of ways of discovering great BBC programmes, including editorially curated Featured items and Collections, Categories (based on genre and format associations), Most Popular and A-Z lists and good old TV Channel and Radio Station schedules. Programme-to-programmes recommendations are another important discovery mechanism within iPlayer and are about to improve dramatically with the launch of a new BBC-built recommendation engine which aims to distil relevant and appropriate recommendations via an algorithm which weights various pieces of programme metadata (e.g. genre, format, broadcast channel, time of broadcast), whilst increasingly enabling editorial fine-tuning of those recommendations to meet users' high expectations of relevance and appropriateness.
KANJI: Have you ever considered using another approach? Indeed, do you already use another one?
TAYLOR: We are constantly looking for ways to improve content discovery within iPlayer and have plans to build on the already popular Favourites functionality within iPlayer to deliver more personalised recommendations. We have historically used a third-party solution to deliver recommendations in iPlayer but switched to a BBC-built product in order to deliver a more tailored and responsive solution.
KANJI: What are the factors that led you to use this method?
TAYLOR: The BBC has a unique on demand inventory, primarily focused on the last 7 days of the BBC's diverse television and radio output, but increasingly including gems from the BBC's archives. Audience expectations of BBC recommendations, shaped by years of effective editorial curation in broadcast environments, are understandably high (and their tolerance for irrelevant or inappropriate recommendations, correspondingly low). We consequently took the decision to move away from an off-the-shelf recommendation engine towards a more tailored solution which we can rapidly evolve in line with our on demand product and programme portfolio.
KANJI: How are humans involved in the recommendations process at the BBC?
TAYLOR: Monitoring the quality of our recommendations is one of a number of tasks performed by iPlayer editorial staff and is carried out by individuals whose familiarity with our output means they are able to gauge the relevance and appropriateness of recommendations without watching every episode of every programme. The volume and dynamic nature of the BBC's on demand offer means wherever possible, we try to translate editorial issues into new programmatic rules in the Recommendation Engine (e.g. children's programmes should never recommend programmes aimed at an adult audience) with a view to keeping on-going manual intervention to a minimum. We also aim to focus editorial attention on high impact / high reach programmes where we know millions of users will be viewing the recommendations, much as key programme junctions on broadcast TV are given particular attention.
KANJI: Finally, how can discovery / recommendation be made better?
TAYLOR: The key is understanding the specific user needs and business goals of your product and then defining what 'better' is in the light of these. For BBC iPlayer, a key aspect of 'better' is exposing users to the full range and breadth of the BBC's TV and radio output and introducing them to programmes they wouldn't otherwise have discovered. Another aspect is preserving the editorial integrity of programme recommendations, in line with licence fee payers' expectations of the BBC. We believe programme metadata, usage data and editorial curation can be combined to deliver the optimal programme discovery experience for users of BBC iPlayer.
#3. Build your own solution based mainly on human editors / curators
Dan Taylor's point above about the key to providing great discovery and recommendations tools is to understand the specific "business goals of your product" particularly resonates when we're looking at different video business models. Blip TV, an international service based in New York, has a simple mission: "to help people to discover the best in original web series and to help web series producers make a sustainable living."
So how do they do it? Kelly Day, the CEO of Blip, told me, "We believe that human-powered content discovery is a better user experience than an algorithm any day."
"By focusing on hand curation, the team at Blip has an intimate sense of what makes shows work and what kind of on-air talent that audiences respond to. Blip has a process whereby the producer applies [to get their content on to Blip], we watch the show, and make a determination as to whether to bring it on to the platform. As a result, we build personal relationships with producers and have a good sense of what is up-and-coming. Algorithms are great at telling you which shows are already "popping", but having human eyes on a show before most everyone else sees it, allows us to take a point of view and become tastemakers - very valuable in this industry.
In addition to our own point of view, we also rely heavily on social media to surface great shows. Social platforms like Facebook, Twitter and Tumblr are very effective at allowing communities to flourish around shows. And on-air talent, who are savvy about building their social media followings, can build very large audiences quickly. Blip's video player includes social sharing, as well as Facebook likes and commenting, right inside the player to make it a lot easier to share web series and discover content through social platforms. We are constantly looking for ways to build community and sharing tools into our platform."
#4. Use tools already supplied by your OVP (Online Video Platform)
Most of the bigger OVPs include some discovery and recommendations functionality out of the box but it's up to you to customize the solution. Brightcove's 'Video Cloud' for example, lets you display "Newest", "Most Viewed" and "Related Videos" without too much effort.
Similarly, Xstream's OVP solution, 'MediaMaker', gives you the tools to surface content as long as you're willing to write some business rules about how to present it.
If you're already using an OVP maybe this is a good place to start. The resulting data will give you a feel - and probably some great data - for how your users interact with your content and can serve as a stepping stone to a different approach.
What do you think? Is there a right way to 'do' content discovery and recommendations or is this something that, like so many things in the new television industry, we're going to keep learning from and making better over time? I'd love to hear your comments below.