Preference learning

Preference learning is a subfield in machine learning, which is a classification method based on observed preference information.^[1] In the view of supervised learning, preference learning trains on a set of items which have preferences toward labels or other items and predicts the preferences for all items.

While the concept of preference learning has been emerged for some time in many fields such as economics,^[2] it's a relatively new topic in Artificial Intelligence research. Several workshops have been discussing preference learning and related topics in the past decade.^[3]

Tasks

The main task in preference learning concerns problems in "learning to rank". According to different types of preference information observed, the tasks are categorized as three main problems in the book Preference Learning:^[4]

Label ranking

In label ranking, the model has an instance space $X=\{x_{i}\}\,\!$ and a finite set of labels $Y=\{y_{i}|i=1,2,\cdots ,k\}\,\!$ . The preference information is given in the form $y_{i}\succ _{x}y_{j}\,\!$ indicating instance $x\,\!$ shows preference in $y_{i}\,\!$ rather than $y_{j}\,\!$ . A set of preference information is used as training data in the model. The task of this model is to find a preference ranking among the labels for any instance.

It was observed some conventional classification problems can be generalized in the framework of label ranking problem:^[5] if a training instance $x\,\!$ is labeled as class $y_{i}\,\!$ , it implies that $\forall j\neq i,y_{i}\succ _{x}y_{j}\,\!$ . In the multi-label case, $x\,\!$ is associated with a set of labels $L\subseteq Y\,\!$ and thus the model can extract a set of preference information $\{y_{i}\succ _{x}y_{j}|y_{i}\in L,y_{j}\in Y\backslash L\}\,\!$ . Training a preference model on this preference information and the classification result of an instance is just the corresponding top ranking label.

Instance ranking

Instance ranking also has the instance space $X\,\!$ and label set $Y\,\!$ . In this task, labels are defined to have a fixed order $y_{1}\succ y_{2}\succ \cdots \succ y_{k}\,\!$ and each instance $x_{l}\,\!$ is associated with a label $y_{l}\,\!$ . Giving a set of instances as training data, the goal of this task is to find the ranking order for a new set of instances.

Object ranking

Object ranking is similar to instance ranking except that no labels are associated with instances. Given a set of pairwise preference information in the form $x_{i}\succ x_{j}\,\!$ and the model should find out a ranking order among instances.

Techniques

There are two practical representations of the preference information $A\succ B\,\!$ . One is assigning $A\,\!$ and $B\,\!$ with two real numbers $a\,\!$ and $b\,\!$ respectively such that $a>b\,\!$ . Another one is assigning a binary value $V(A,B)\in \{0,1\}\,\!$ for all pairs $(A,B)\,\!$ denoting whether $A\succ B\,\!$ or $B\succ A\,\!$ . Corresponding to these two different representations, there are two different techniques applied to the learning process.

Utility function

If we can find a mapping from data to real numbers, ranking the data can be solved by ranking the real numbers. This mapping is called utility function. For label ranking the mapping is a function $f:X\times Y\rightarrow \mathbb {R} \,\!$ such that $y_{i}\succ _{x}y_{j}\Rightarrow f(x,y_{i})>f(x,y_{j})\,\!$ . For instance ranking and object ranking, the mapping is a function $f:X\rightarrow \mathbb {R} \,\!$ .

Finding the utility function is a regression learning problem which is well developed in machine learning.

Preference relations

The binary representation of preference information is called preference relation. For each pair of alternatives (instances or labels), a binary predicate can be learned by conventional supervising learning approach. Fürnkranz and Hüllermeier proposed this approach in label ranking problem.^[6] For object ranking, there is an early approach by Cohen et al.^[7]

Using preference relations to predict the ranking will not be so intuitive. Since preference relation is not transitive, it implies that the solution of ranking satisfying those relations would sometimes be unreachable, or there could be more than one solution. A more common approach is to find a ranking solution which is maximally consistent with the preference relations. This approach is a natural extension of pairwise classification.^[6]

Uses

Preference learning can be used in ranking search results according to feedback of user preference. Given a query and a set of documents, a learning model is used to find the ranking of documents corresponding to the relevance with this query. More discussions on research in this field can be found in Tie-Yan Liu's survey paper.^[8]

Another application of preference learning is recommender systems.^[9] Online store may analyze customer's purchase record to learn a preference model and then recommend similar products to customers. Internet content providers can make use of user's ratings to provide more user preferred contents.

References

^ Mohri, Mehryar; Rostamizadeh, Afshin; Talwalkar, Ameet (2012). Foundations of Machine Learning. US, Massachusetts: MIT Press. ISBN 9780262018258.
^ Shogren, Jason F.; List, John A.; Hayes, Dermot J. (2000). "Preference Learning in Consecutive Experimental Auctions". American Journal of Agricultural Economics. 82 (4): 1016–1021. doi:10.1111/0002-9092.00099. S2CID 151493631.
^ "Preference learning workshops". 23 January 2024.
^ Fürnkranz, Johannes; Hüllermeier, Eyke (2011). "Preference Learning: An Introduction". Preference Learning. Springer-Verlag New York, Inc. pp. 3–8. ISBN 978-3-642-14124-9.
^ Har-peled, Sariel; Roth, Dan; Zimak, Dav (2003). "Constraint classification for multiclass classification and ranking". In Proceedings of the 16th Annual Conference on Neural Information Processing Systems, NIPS-02: 785–792.
^ ^a ^b Fürnkranz, Johannes; Hüllermeier, Eyke (2003). "Pairwise Preference Learning and Ranking". Proceedings of the 14th European Conference on Machine Learning: 145–156.
^ Cohen, William W.; Schapire, Robert E.; Singer, Yoram (1998). "Learning to order things". In Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems: 451–457. ISBN 978-0-262-10076-2.
^ Liu, Tie-Yan (2009). "Learning to Rank for Information Retrieval". Foundations and Trends in Information Retrieval. 3 (3): 225–331. doi:10.1561/1500000016.
^ Gemmis, Marco De; Iaquinta, Leo; Lops, Pasquale; Musto, Cataldo; Narducci, Fedelucio; Semeraro, Giovanni (2009). "Learning Preference Models in Recommender Systems". Preference Learning in Recommender Systems (PDF). Vol. 41. pp. 387–407. doi:10.1007/978-3-642-14125-6_18. ISBN 978-3-642-14124-9. {{cite book}}: |journal= ignored (help)

External links

Preference Learning site

[1] Mohri, Mehryar; Rostamizadeh, Afshin; Talwalkar, Ameet (2012). Foundations of Machine Learning. US, Massachusetts: MIT Press. ISBN 9780262018258.

[SHOG00-2] Shogren, Jason F.; List, John A.; Hayes, Dermot J. (2000). "Preference Learning in Consecutive Experimental Auctions". American Journal of Agricultural Economics. 82 (4): 1016–1021. doi:10.1111/0002-9092.00099. S2CID 151493631.

[WEB:WORKSHOP-3] "Preference learning workshops". 23 January 2024.

[FURN11-4] Fürnkranz, Johannes; Hüllermeier, Eyke (2011). "Preference Learning: An Introduction". Preference Learning. Springer-Verlag New York, Inc. pp. 3–8. ISBN 978-3-642-14124-9.

[HARP03-5] Har-peled, Sariel; Roth, Dan; Zimak, Dav (2003). "Constraint classification for multiclass classification and ranking". In Proceedings of the 16th Annual Conference on Neural Information Processing Systems, NIPS-02: 785–792.

[FURN03-6] Fürnkranz, Johannes; Hüllermeier, Eyke (2003). "Pairwise Preference Learning and Ranking". Proceedings of the 14th European Conference on Machine Learning: 145–156.

[COHE98-7] Cohen, William W.; Schapire, Robert E.; Singer, Yoram (1998). "Learning to order things". In Proceedings of the 1997 Conference on Advances in Neural Information Processing Systems: 451–457. ISBN 978-0-262-10076-2.

[LIU09-8] Liu, Tie-Yan (2009). "Learning to Rank for Information Retrieval". Foundations and Trends in Information Retrieval. 3 (3): 225–331. doi:10.1561/1500000016.

[GEMM09-9] Gemmis, Marco De; Iaquinta, Leo; Lops, Pasquale; Musto, Cataldo; Narducci, Fedelucio; Semeraro, Giovanni (2009). "Learning Preference Models in Recommender Systems". Preference Learning in Recommender Systems (PDF). Vol. 41. pp. 387–407. doi:10.1007/978-3-642-14125-6_18. ISBN 978-3-642-14124-9. {{cite book}}: |journal= ignored (help)

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]