A new learning paradigm: Learning using privileged information - Vapnik, Vladimir; Vashist, Akshay (Neural networks, 2009)
In the Afterword to the second edition of the book “Estimation of Dependences Based on Empirical Data” by V. Vapnik, an advanced learning paradigm called Learning Using Hidden Information (LUHI) was introduced. This Afterword also suggested an extension of the SVM method (the so called SVM γ + method) to implement algorithms which address the LUHI paradigm ( Vapnik, 1982–2006, Sections 2.4.2 and 2.5.3 of the Afterword). See also ( Vapnik, Vashist, & Pavlovitch, 2008, 2009) for further development of the algorithms. In contrast to the existing machine learning paradigm where a teacher does not play an important role, the advanced learning paradigm considers some elements of human teaching. In the new paradigm along with examples, a teacher can provide students with hidden information that exists in explanations, comments, comparisons, and so on. This paper discusses details of the new paradigm 1 1 In this article we changed the terminology. We will call this paradigm Learning Using Privileged Information (LUPI) (instead of LUHI) since the word privilege better reflects the core idea of the new paradigm. and corresponding algorithms, introduces some new algorithms, considers several specific forms of privileged information, demonstrates superiority of the new learning paradigm over the classical learning paradigm when solving practical problems, and discusses general questions related to the new ideas.