Universal probabilistic programming offers a powerful approach to statistical phylogenetics

Statistical phylogenetic analysis currently relies on complex, dedicated software packages, making it difficult for evolutionary biologists to explore new models and inference strategies. Recent years have seen more generic solutions based on probabilistic graphical models, but this formalism can only partly express phylogenetic problems. Here we show that universal probabilistic programming languages (PPLs) solve the expressivity problem, while still supporting automated generation of efficient inference algorithms. To prove the latter point, we develop automated generation of sequential Monte Carlo (SMC) algorithms for PPL descriptions of arbitrary biological diversification (birth-death) models. SMC is a new inference strategy for these problems, supporting both parameter inference and efficient estimation of Bayes factors that are used in model testing. We take advantage of this in automatically generating SMC algorithms for several recent diversification models that have been difficult or impossible to tackle previously. Finally, applying these algorithms to 40 bird phylogenies, we show that models with slowing diversification, constant turnover and many small shifts generally explain the data best. Our work opens up several related problem domains to PPL approaches, and shows that few hurdles remain before these techniques can be effectively applied to the full range of phylogenetic models.

Download Full-text

Universal probabilistic programming offers a powerful approach to statistical phylogenetics

Communications Biology ◽

10.1038/s42003-021-01753-7 ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Fredrik Ronquist ◽

Jan Kudlicka ◽

Viktor Senderov ◽

Johannes Borgström ◽

Nicolas Lartillot ◽

...

Keyword(s):

Programming Languages ◽

Graphical Models ◽

Sequential Monte Carlo ◽

Full Range ◽

Efficient Estimation ◽

Probabilistic Programming ◽

Automated Generation ◽

Inference Algorithms ◽

Powerful Approach ◽

Inference Strategy

AbstractStatistical phylogenetic analysis currently relies on complex, dedicated software packages, making it difficult for evolutionary biologists to explore new models and inference strategies. Recent years have seen more generic solutions based on probabilistic graphical models, but this formalism can only partly express phylogenetic problems. Here, we show that universal probabilistic programming languages (PPLs) solve the expressivity problem, while still supporting automated generation of efficient inference algorithms. To prove the latter point, we develop automated generation of sequential Monte Carlo (SMC) algorithms for PPL descriptions of arbitrary biological diversification (birth-death) models. SMC is a new inference strategy for these problems, supporting both parameter inference and efficient estimation of Bayes factors that are used in model testing. We take advantage of this in automatically generating SMC algorithms for several recent diversification models that have been difficult or impossible to tackle previously. Finally, applying these algorithms to 40 bird phylogenies, we show that models with slowing diversification, constant turnover and many small shifts generally explain the data best. Our work opens up several related problem domains to PPL approaches, and shows that few hurdles remain before these techniques can be effectively applied to the full range of phylogenetic models.

Download Full-text

Correctness of Sequential Monte Carlo Inference for Probabilistic Programming Languages

10.26226/morressier.604907f41a80aac83ca25d23 ◽

2021 ◽

Cited By ~ 1

Author(s):

Daniel Lundén ◽

Johannes Borgström ◽

David Broman

Keyword(s):

Monte Carlo ◽

Programming Languages ◽

Sequential Monte Carlo ◽

Probabilistic Programming

Download Full-text

Correctness of Sequential Monte Carlo Inference for Probabilistic Programming Languages

Programming Languages and Systems - Lecture Notes in Computer Science ◽

10.1007/978-3-030-72019-3_15 ◽

2021 ◽

pp. 404-431

Author(s):

Daniel Lundén ◽

Johannes Borgström ◽

David Broman

Keyword(s):

Monte Carlo ◽

Programming Languages ◽

Sequential Monte Carlo ◽

Fundamental Problem ◽

Operational Semantics ◽

Correctness Proof ◽

Probabilistic Programming ◽

Inference Problems ◽

Large Numbers ◽

Open Question

AbstractProbabilistic programming is an approach to reasoning under uncertainty by encoding inference problems as programs. In order to solve these inference problems, probabilistic programming languages (PPLs) employ different inference algorithms, such as sequential Monte Carlo (SMC), Markov chain Monte Carlo (MCMC), or variational methods. Existing research on such algorithms mainly concerns their implementation and efficiency, rather than the correctness of the algorithms themselves when applied in the context of expressive PPLs. To remedy this, we give a correctness proof for SMC methods in the context of an expressive PPL calculus, representative of popular PPLs such as WebPPL, Anglican, and Birch. Previous work have studied correctness of MCMC using an operational semantics, and correctness of SMC and MCMC in a denotational setting without term recursion. However, for SMC inference—one of the most commonly used algorithms in PPLs as of today—no formal correctness proof exists in an operational setting. In particular, an open question is if the resample locations in a probabilistic program affects the correctness of SMC. We solve this fundamental problem, and make four novel contributions: (i) we extend an untyped PPL lambda calculus and operational semantics to include explicit resample terms, expressing synchronization points in SMC inference; (ii) we prove, for the first time, that subject to mild restrictions, any placement of the explicit resample terms is valid for a generic form of SMC inference; (iii) as a result of (ii), our calculus benefits from classic results from the SMC literature: a law of large numbers and an unbiased estimate of the model evidence; and (iv) we formalize the bootstrap particle filter for the calculus and discuss how our results can be further extended to other SMC algorithms.

Download Full-text

Model-based machine learning

Philosophical Transactions of The Royal Society A Mathematical Physical and Engineering Sciences ◽

10.1098/rsta.2012.0222 ◽

2013 ◽

Vol 371 (1984) ◽

pp. 20120222 ◽

Cited By ~ 55

Author(s):

Christopher M. Bishop

Keyword(s):

Machine Learning ◽

Graphical Models ◽

Large Scale ◽

Commercial Application ◽

Software Environment ◽

Probabilistic Programming ◽

Practical Applications ◽

Model Based ◽

Inference Algorithms ◽

Modelling Environment

Several decades of research in the field of machine learning have resulted in a multitude of different algorithms for solving a broad range of problems. To tackle a new application, a researcher typically tries to map their problem onto one of these existing methods, often influenced by their familiarity with specific algorithms and by the availability of corresponding software implementations. In this study, we describe an alternative methodology for applying machine learning, in which a bespoke solution is formulated for each new application. The solution is expressed through a compact modelling language, and the corresponding custom machine learning code is then generated automatically. This model-based approach offers several major advantages, including the opportunity to create highly tailored models for specific scenarios, as well as rapid prototyping and comparison of a range of alternative models. Furthermore, newcomers to the field of machine learning do not have to learn about the huge range of traditional methods, but instead can focus their attention on understanding a single modelling environment. In this study, we show how probabilistic graphical models, coupled with efficient inference algorithms, provide a very flexible foundation for model-based machine learning, and we outline a large-scale commercial application of this framework involving tens of millions of users. We also describe the concept of probabilistic programming as a powerful software environment for model-based machine learning, and we discuss a specific probabilistic programming language called Infer.NET , which has been widely used in practical applications.

Download Full-text

Publisher Correction: Universal probabilistic programming offers a powerful approach to statistical phylogenetics

Communications Biology ◽

10.1038/s42003-021-01922-8 ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Fredrik Ronquist ◽

Jan Kudlicka ◽

Viktor Senderov ◽

Johannes Borgström ◽

Nicolas Lartillot ◽

...

Keyword(s):

Probabilistic Programming ◽

Powerful Approach

A Correction to this paper has been published: https://doi.org/10.1038/s42003-021-01922-8

Download Full-text

NENOK — A SOFTWARE ARCHITECTURE FOR GENERIC INFERENCE

International Journal of Artificial Intelligence Tools ◽

10.1142/s0218213010000042 ◽

2010 ◽

Vol 19 (01) ◽

pp. 65-99 ◽

Cited By ~ 6

Author(s):

MARC POULY

Keyword(s):

Programming Languages ◽

Software Library ◽

Inference Process ◽

Software Projects ◽

Open Problems ◽

Constraint Systems ◽

Fast Prototyping ◽

Inference Algorithms ◽

Key Competences ◽

High Level

Computing inference from a given knowledgebase is one of the key competences of computer science. Therefore, numerous formalisms and specialized inference routines have been introduced and implemented for this task. Typical examples are Bayesian networks, constraint systems or different kinds of logic. It is known today that these formalisms can be unified under a common algebraic roof called valuation algebra. Based on this system, generic inference algorithms for the processing of arbitrary valuation algebras can be defined. Researchers benefit from this high level of abstraction to address open problems independently of the underlying formalism. It is therefore all the more astonishing that this theory did not find its way into concrete software projects. Indeed, all modern programming languages for example provide generic sorting procedures, but generic inference algorithms are still mythical creatures. NENOK breaks a new ground and offers an extensive library of generic inference tools based on the valuation algebra framework. All methods are implemented as distributed algorithms that process local and remote knowledgebases in a transparent manner. Besides its main purpose as software library, NENOK also provides a sophisticated graphical user interface to inspect the inference process and the involved graphical structures. This can be used for educational purposes but also as a fast prototyping architecture for inference formalisms.

Download Full-text