Active Preference Learning with Discrete Choice Data

I just released an unofficial implementation of the Active Preference Learning with Discrete Choice Data by Brochu et al. as published in NIPS 2007.

Why would this package be useful for you?

Imagine a scenario where you are trying to find a place to have lunch today. There are tons of places to eat around. An app suggests you two restaurants to compare at a time and can help you reach a good enough restaurant in as few queries as possible. Each time you pick a restaurant, the model gradually learns what you want and hopefully, its suggestions get gradually better. This will save you a lot of time and when designed well, can be a much more fun way to search as opposed to going through a boring list view.

In general, if the following conditions are present, active preference learning can be useful for you:

• User is searching for an item in a very large set of items that’s impossible go through one by one.
• User is okay with a good enough solution if it is going to be found shortly.
• Items can be embedded in a vectors space where proximity in that space implies similarity in preference between items.

Why not other rating-based methods as a way to model preferences?

Amazon uses 5-star ratings as feedback for items by users, also called Likert scale. Stitch Fix uses Style Shuffle, which is basically a liked or didn’t like decision. Then, why would you use binary comparisons instead of these two options? According to Kingsley and Brown (2010), people get more consistent as they progress towards binary comparisons of alternatives. This might be a cleaner feedback from the user as opposed to absolute ratings such as 1-to-5 scale or like-or-dislike input. This is the motivation behind this paper.

Why I built this

I just think amazing things can be made with active preference learning. Especially given that we are in the age of GANs and other powerful generative models. Imagine a GAN creating music pairs of music and the user rates them to quickly generate enjoyable tunes. Or quick art forms that reflect the current mood of the user.

I couldn’t find well-maintained repos that provide active preference learning alternatives. I wanted to spark a fire. I hope me or someone else can extend this library further and built great things with it.

Roadmap

• Extend the package such that it models two more options: ‘liked both’ and ‘disliked both’.
• Allow the option to search in the (possible bounded) continous space as opposed to the finite set of elements.
• Incorporate a more accurate posterior inference engine than Laplace’s approximation.
• Create more examples.
• Package for PyPI, add CI/CD procedures and more tests.