Semantics for probabilistic programming higher-order functions, continuous distributions, and soft constraints

• Published in: Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science pp. 525 - 534
• Main Authors: Staton, Sam, Yang, Hongseok, Wood, Frank, Heunen, Chris, Kammar, Ohad
• Format: Conference Proceeding
• Language: English
• Published: New York, NY, USA ACM 05.07.2016
• Series: ACM Conferences
• Subjects: Software and its engineering > Software notations and tools > General programming languages
• ISBN: 9781450343916; 1450343910
• DOI: 10.1145/2933575.2935313

We study the semantic foundation of expressive probabilistic programming languages, that support higher-order functions, continuous distributions, and soft constraints (such as Anglican, Church, and Venture). We define a metalanguage (an idealised version of Anglican) for probabilistic computation with the above features, develop both operational and denotational semantics, and prove soundness, adequacy, and termination. This involves measure theory, stochastic labelled transition systems, and functor categories, but admits intuitive computational readings, one of which views sampled random variables as dynamically allocated read-only variables. We apply our semantics to validate nontrivial equations underlying the correctness of certain compiler optimisations and inference algorithms such as sequential Monte Carlo simulation. The language enables defining probability distributions on higher-order functions, and we study their properties.