A new learning paradigm: Learning using privileged information

• Published in: Neural networks Vol. 22; no. 5; pp. 544 - 557
• Main Authors: Vapnik, Vladimir, Vashist, Akshay
• Format: Journal Article Conference Proceeding
• Language: English
• Published: Kidlington Elsevier Ltd 01.07.2009; Elsevier
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Bayes Theorem; Computer science; control theory; systems; Databases, Genetic; Exact sciences and technology; Forecasting - methods; Hidden information; Information Dissemination; Information, signal and communications theory; Language; Learning; Learning and adaptive systems; Learning with teacher; Machine learning; Mathematical Concepts; Oracle SVM; Privileged information; Protein Conformation; Proteins - classification; Sequence Analysis, Protein - methods; Signal and communications theory; Signal representation. Spectral analysis; Signal, noise; SVM; Telecommunications and information theory; Time Factors; Privileged information; Oracle SVM; Learning with teacher; Hidden information; SVM; Machine learning; Human; Automatic classification; Teacher; Support vector machine; Student; Algorithm; Signal classification; Contrast; Implementation; Teaching; Learning; Acquisition process; Dependence; Development; Oracle; Artificial intelligence
• ISSN: 0893-6080; 1879-2782; 1879-2782
• DOI: 10.1016/j.neunet.2009.06.042

In the Afterword to the second edition of the book “Estimation of Dependences Based on Empirical Data” by V. Vapnik, an advanced learning paradigm called Learning Using Hidden Information (LUHI) was introduced. This Afterword also suggested an extension of the SVM method (the so called SVM γ + method) to implement algorithms which address the LUHI paradigm ( Vapnik, 1982–2006, Sections 2.4.2 and 2.5.3 of the Afterword). See also ( Vapnik, Vashist, & Pavlovitch, 2008, 2009) for further development of the algorithms. In contrast to the existing machine learning paradigm where a teacher does not play an important role, the advanced learning paradigm considers some elements of human teaching. In the new paradigm along with examples, a teacher can provide students with hidden information that exists in explanations, comments, comparisons, and so on. This paper discusses details of the new paradigm 1 1 In this article we changed the terminology. We will call this paradigm Learning Using Privileged Information (LUPI) (instead of LUHI) since the word privilege better reflects the core idea of the new paradigm. and corresponding algorithms, introduces some new algorithms, considers several specific forms of privileged information, demonstrates superiority of the new learning paradigm over the classical learning paradigm when solving practical problems, and discusses general questions related to the new ideas.