Deep Conditional Transformation Models

• Published in: Machine Learning and Knowledge Discovery in Databases. Research Track Vol. 12977; pp. 3 - 18
• Main Authors: Baumann, Philipp F. M., Hothorn, Torsten, Rügamer, David
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Deep learning; Distributional regression; Normalizing flows; Semi-structured regression; Transformation models
• ISBN: 3030865223; 9783030865221
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-86523-8_1

Learning the cumulative distribution function (CDF) of an outcome variable conditional on a set of features remains challenging, especially in high-dimensional settings. Conditional transformation models provide a semi-parametric approach that allows to model a large class of conditional CDFs without an explicit parametric distribution assumption and with only a few parameters. Existing estimation approaches within this class are, however, either limited in their complexity and applicability to unstructured data sources such as images or text, lack interpretability, or are restricted to certain types of outcomes. We close this gap by introducing the class of deep conditional transformation models which unifies existing approaches and allows to learn both interpretable (non-)linear model terms and more complex neural network predictors in one holistic framework. To this end we propose a novel network architecture, provide details on different model definitions and derive suitable constraints as well as network regularization terms. We demonstrate the efficacy of our approach through numerical experiments and applications.