Harry Potter and Kanye West

One single sentence describes personalization in e-commerce: Everyone talks about it, everyone claims to have it, and it rarely works.

E-commerce Web sites use recommendation algorithms to personalize the experience of each customer. But most Web sites end up recommending bestsellers or category bestsellers. So if you buy a pair of sunglasses, you are recommended five additional pairs of sunglasses.

The two main reasons most recommendation algorithms fail are Harry Potter and Paris Hilton.

Harry Potter

The Harry Potter effect is relatively well known, thanks to Amazon. The name was coined in the mid-2000s, when the Harry Potter books popped up as suggested items in the Amazon “customers who bought this also bought” widget, even for completely unrelated books and products.


The “customers who bought this also bought” widget is based on similarities. The recommendations are based on the items that the customer has previously purchased. The algorithm matches the user’s purchased items to similar items and then combines the similar items into a recommendation list. By definition, a very popular item is often purchased with other items, and therefore the popular item is suggested to be similar to several other items.

Enter Kanye

Kanye West spends so much money that he claims to be broken. He buys a great deal of merchandise; he is likely to consumed two items together. Compare his behavior to a casual shopper who buys just one item. This nonobvious problem makes Kanye West more dangerous than the little magician—and more subtle.

How Big Are These Problems?

The bigger the ,the more skewed the data. The Internet is much more skewed than the off-line world is. Anyone running an e-commerce Web site with thousands of customers knows that there is no such thing as an “average order” or “average customer spending.” Few customers spend a great deal of money; most customers spend a small amount of money.

No Easy Way Out

Easy solutions call for a normalization of the data, specifically for normalizing similarities for the number of times an item has been consumed. Some of these solutions are even worse than the original problem was. If we were to divide the similarity of a Harry Potter book by the number of books sold to create a relative measure of similarity, then an item with one single sale would end up being considered extremely similar to every item that customer consumed.

At Muse, we developed a proprietary system that not only solves both the Harry Potter and Kanye West effects but also avoids the grave dangers of commonly suggested “naïve” solutions.

Our item-to-item collaborative filtering algorithm scales independently from the number of items and users. And unlike traditional collaborative filtering, the algorithm generates high-quality recommendations, even with limited user data and even with as few as three or four items.

The End of Search

The End of Search

Billions of people, photos, products. The Internet is so great because of its magnitude, yet this magnitude has a downside.

Online discovery is broken. While servers have become faster, front-end devices have become smaller and more reliable, and new services have flourished, online discovery has stayed the same since 1995 and is still based on search boxes.

Google Shopping. Amazon. eBay. Product discovery is driven by search boxes. And search is effective only when people already know what they need.

In the words of Rakuten founder Hiroshi Mikitani: “E-commerce is search. And search is not shopping.”

When it’s not, it is driven by advertising or discounts.

Discovery is a problem that needs to be solved.

Lifestyle suffers the most from this problem because purchase decisions are driven by inspiration and tastemakers, not by needs. Lifestyle as a category matters because it includes everything we don’t need to survive yet makes our lives unique.

What if people won’t have to use search in the future?

What if we can build a better e-commerce experience that does not rely on search?