AI recommendation systems: Key takeaways and insights

The abundant content availability today doesn't astonish anyone. AI can help you generate more content, study your users’ preferences and recommend relevant articles, products or services. But what are the challenges you might encounter when implementing AI recommendation systems? And what should you consider before you deploy them?

The first experiments

There’s a huge potential for growth in recommending relevant products or content to the right audience. Online statistics show that Netflix's recommendation system accounts for a striking 80% of its streaming time.

Similarly, Amazon's recommendation engine fuels approximately 35% of its sales. Not to forget another significant player in the media industry - Spotify. It employs a diverse mix of techniques and strategies in its recommendation system, even analyzing the audio characteristics of each song.

But just like any innovation, AI recommendations have not been without many failed attempts. If badly designed, they can lead you down a path of dissatisfaction and user churn.  

Here are three global brands that weren’t afraid to experiment with new ideas. We can now learn from their first tries, whether they were successful or not:

• Netflix and the Cinematch debacle: In its early days, Netflix faced severe criticism for its Cinematch recommendation system. It was notorious for its narrow algorithm, often suggesting movies that were too similar to those users had already watched This lack of diversity was a significant drawback.
• Apple Music's recommendations: Apple Music's recommendation system was under fire right from its inception. It struggled to accurately suggest songs and artists that aligned with users' tastes, resulting in widespread dissatisfaction. This experience demonstrated that even tech giants can stumble when it comes to personalization.
• Spotify's Release Radar misfire: Spotify's Release Radar, a playlist designed to recommend new releases from artists that users follow or might enjoy, encountered its fair share of missteps. It often included songs from artists that users didn't listen to or older tracks that were recently added to Spotify, rather than focusing on actual new releases.

These examples demonstrate that even industry giants with substantial budgets and vast data repositories must navigate the challenges of effectively utilizing recommendation systems. 

From complexity to simplicity: H&M case study

The big fashion company H&M showed us a great example of this journey. They unveiled the development process of their recommendation engine at the ACM RecSys 2021 conference, giving us valuable information about what they learned.

Their primary business motivation for deploying the recommender system was the following:

1. Boost the sales of expensive items: One of the key objectives is to promote high-value items in their inventory, thereby increasing the average order value and overall revenue.
2. Support H&M's "Great Design for Everyone" concept: The recommendation system aligns with H&M's ethos of making fashion accessible to everyone. Providing personalized recommendations ensures every customer can find something that suits their style and budget.
3. Help find optimal price points: The system also aims to help H&M understand the price sensitivity of their customers better. By analyzing which price points generate the most engagement and sales, they can optimize their pricing strategy to maximize profitability.

H&M's journey began with a basic version of their recommendation system – the minimum viable product. They initially used collaborative filtering (base recommendation algorithm). However, they encountered issues like recommending too many common and older items, such as jeans and socks.

The over-recommendation of socks indicated that the collaborative filtering method favored items that were frequently bought but less expensive. The abundance of older items was a result of relying heavily on past purchase data instead of user interactions like clicks on the website. 

To tackle these problems, they added filters for specific items and aimed to strike a balance between new and old data.

The next step was to enhance the basic version. H&M refined their approach by adding score-weighted boosters (additional models), taking various factors into account, such as category preferences, current trends, price preferences, color preferences, item similarities, and market basket analysis. 

While these models worked well on their own, they started to interact in complex ways when combined, leading to unpredictable outcomes. For instance, changes in category preferences influenced trends, while the impact of recency weighting was minimal.

One likely reason for these mixed results was that each model was optimized separately and in isolation. Although joint optimization was possible, it would have required significant computing time. 

Consequently, when several simple models were combined, the system's complexity increased substantially. This raised challenges related to scalability, maintenance, and cost, as it demanded substantial computational resources for retraining and running the models.

A simple algorithm – the key to success

Recognizing these challenges, H&M opted for a simpler approach. They reverted to the basic collaborative filtering algorithm, refocusing on their business objectives and the unique attributes and constraints of their use cases. 

For instance, many of their clothing items have a short shelf life, and a significant portion of their recommendations was for items that were no longer available. Predicting past behavior didn't align with the need to predict future engaging content.

A pivotal moment came when H&M chose to exclusively use collaborative filtering to map user buying behavior. Think of it as creating a network (graph) that connects each user to the products they've purchased over time. Users were then grouped into clusters based on similar buying patterns within this graph.

Within these clusters, recommendations were made for content that was popular in that group. This included new products bought by other customers in the last 14 days, products purchased in the last 100 days, products featured in campaigns, and category recommendations. 

Instead of individual user recommendations, the focus shifted to making recommendations for the entire group.

This simplified approach offered several advantages. It was easier to use, explain, adjust, and operate. Moreover, it was cheaper to maintain from both an algorithmic perspective and in terms of computing time. 

The result was a simple yet powerful method for suggesting fashion items to customers, aligning with H&M's business goals while providing a more streamlined and cost-effective solution.

Start thinking about your own AI recommendations

Do you believe that your business can benefit from an AI recommendation system? Before you start, consider these six points. They will help you be crystal clear about your business objectives and understand the legal or reputational risks associated with recommending inappropriate or biased content.

Here's an actionable checklist to consider:

1. Define business goals and metrics: What specific business objectives do you aim to achieve with the recommendation system? What metrics will be used to measure success, and how soon can these metrics be accurately measured?
2. Evaluate recommendation relevance vs. diversity: Do you need the recommendations to be relevant to the user's preferences or rather introducing them to new options (for instance, favorite songs or rather new songs)?
3. Consider user perspective on recommendations: Does the user typically have a specific item in mind, just a seed of an idea or a path they wish to explore? Understanding user intent can help refine the recommendation process.
4. Anticipate changes in user behavior: What changes in user behavior do you expect as a result of the recommendations? Which user segments will this apply to? What product/content metrics need to be considered?
5. Understand user and content/product data: What is the typical lifespan of the featured content - Is it seasonal or timeless? Are you confident about the attributes used to describe the user? How will you address the 'cold start' problem where the system lacks sufficient information about users or items? Can you ensure the data is unbiased and free from malicious content? Is all personal data processed in compliance with legal regulations? What signals will you use - implicit (e.g., rating stars) or explicit (e.g., time spent on an article)?
6. Assess model complexity and maintenance: How complex is the model? Is it clear how different factors influence each other, and can the model be maintained and adjusted? What computing resources are needed to run the model? How often will the model need to be tuned? 

With answers to these questions in hand, you can significantly enhance your chances of successfully implementing and deploying an AI recommendation system while also reducing potential risks. While it's important to approach AI with caution, we obviously can't shut the door on it.  

This innovative technology offers so many exciting opportunities to enhance the customer experience. And Kentico developers are working hard to bring the best of AI to your DXP. You might have already experienced content personalization with Recombee but we won't stop there. More AI features are on the horizon! Stay tuned, and in the meantime, delve into a thought-provoking article on brand trust and content authenticity in the era of generative AI.