Research Outputs

2023 Publications

Benton, J., Shi, Y., De Bortoli, V., Deligiannidis, G. and Doucet, A. (2023) Royal Statistical Society, Series B, together with its discussion A unified framework for designing generative models with general noising processes. The importance of this work is that it enables the ideas of diffusion generative models to be applied to non-real valued data. This could include discrete data, network, graphs, or e.g. data that is restricted to lie on a manifold.

Rimella, L., Alderton, S., Sammarro, M., Rowlingson, B., Cocker, D., Feasey, N., Fearnhead, P. & Jewell, C. (2023) Journal of the Royal Statistical Society Series C: Applied Statistics, qlad055 (Short Summary Below)

Rimella, L., Jewell, C. & Fearnhead, P. (2023) To appear in Statistica Sinica,doi:10.5705/ss.202022.0198. (Short Summary Below)

Cornish, R., Taufiq Muhammad, F., Doucet, A. & Holmes, Chris. (2023) arXiv e-prints, pp. arXiv:2301.07210.

Sutton, M., Fearnhead, P., (2023) To appear in Journal of Computational and Graphical Statistics. (Short Summary Below)

2022 Publications

Christophe, A., Lee, A., Power, S. & Wang A.Q., (2022) arXiv preprint arXiv:2208.05239

Chevallier A., Fearnhead P. & Sutton M. (2022) To appear in Journal of the American Statistical Association, doi:10.1080/01621459.2022.2099402 (Short Summary Below)

Corbella, A., Spencer, S. & Roberts, G. (2022) Statistics and Computing, 32, 107 doi:10.1007/s11222-022-10142-x

Sutton, M., Salomone, R., Chevallier, A. & Fearnhead, P., (2022) Advances in Neural Information Processing Systems

Andrieu, C., Lee, A., Power, S. & Wang A.Q. (2022) arXiv preprint arXiv:2211.08959

Andrieu, C., Lee, A., Power, S. & Wang, A. (2022) The Annals of Statistics, 50, pp.3592-3618. (Short Summary Below)

Fong, E., Holmes, C. & Walker, S. G. (2022) To appear in Journal of the Royal Statistical Society, Series B. (Short Summary Below)

Buchholz, A., Ahfock, D. & Richardson, S. (2022) Bayesian Analysis, 18 pp. 607-638

Jersakova, R., Lomax, J., Hetherington, J., Lehmann, B., Nicholson, G., Briers, M. & Holmes, C. (2022) Applied Statistics 71 834-860

Nicholson, G., Blangiardo, M., Briers, M., Diggle, P. J., Erlend Fjelde, T., Ge, H., Goudie, R.J.B., Jersakova, R., King, R.E., Lehmann, B.C.L., Mallon, A., Padellini, T., Teh, Y.W., Holmes, C. & Richardson, S. (2022) Statistical Science 37, pp. 183–206

Aicher, C., Putcha, S., Nemeth, C., Fearnhead, P., & Fox, E. B. (2022) Society for Industrial and Applied Mathematics Vol.1, Iss.3 (Short Summary Below)

Vono, M., Paulin, D., & Doucet, A. (2022) Journal of Machine Learning Research 23(25):1?69, 2022

2021 Publications

Read, J.M., Bridgen, J.R.E, Cummings, D.A.T., Ho,A. & Jewell C.P. (2021) Philosophical Transactions of the Royal Society B 376 (1829), 20200265

Birrell, P., Blake, J., Van Leeuwen, E., Gent, N. & De Angelis, D. (2021) Philosophical Transactions of the Royal Society B 376 (1829), 20200279

Nicholson, G., Lehmann, B., Padellini, T., Pouwels, K.B., Jersakova, R., Lomax, J., King, R.E., Mallon, A.M., Diggle, P.J., Richardson, S., Blangiardo, M. & Holmes, C. (2021) MedRxiv

Nemeth, C., & Fearnhead, P. (2021) Taylor & Francis Online (Short Summary Below)

2020 Publications

Behr, M., Ansari, M. A., Munk, A., & Holmes, C. (2020) Proceedings of the National Academy of Sciences of the United States of America, 117(18), 9787–9792 (Short Summary Below)

Vats, D., Gon?alves, F., ?atuszyński, K., & Roberts, G. O. (2020)

Touloupou, P., Finkenst?dt, B., & Spencer, S. E. F. (2020) Journal of Computational and Graphical Statistics, 29(2), 238–249

Chapman, L. A. C., Spencer, S. E. F., Pollington, T. M., Jewell, C. P., Mondal, D., Alvar, J., Deirdre Hollingsworth, T., Cameron, M. M., Bern, C., & Medley, G. F. (2020) medRxiv, 2020.02.24.20023325

Pollock, M., Fearnhead, P., Johansen, A. M., & Roberts, G. O. (2020) Journal of the Royal Statistical Society. Series B, Statistical Methodology

Chevallier, A., Fearnhead, P., & Sutton, M. (2020) Taylor & Francis Online

Bierkens, J., Grazzi, S., Kamatani, K., & Roberts, G. (2020) Proceedings of the 37th International Conference on Machine Learning, PMLR 119:908-918, 2020

Fong, E., & Holmes, C. C. (2020) Biometrika, 107(2), 489–496 (Short Summary Below)

2019 Publications

Wang, A. Q., Pollock, M., Roberts, G. O., & Steinsaltz, D. (2019)

Buchholz, A., Ahfock, D., & Richardson, S. (2019)

Mider, M., Jenkins, P. A., Pollock, M., Roberts, G. O., & S?rensen, M. (2019)

Short Paper Summaries

Quasi-stationary Monte Carlo and the ScaLE Algorithm

The key idea of the paper is to try and draw samples from the posterior distribution by simulating a Markov process with killing. Such processes will eventually die out, and there long-term behaviour is often described by its quasi-stationary distribution. That is we consider the limiting distribution of the process conditional on it not dying.

This leads to new algorithms for sampling from a posterior, though this does not come without some challenges -- simulating from the quasi-stationary distribution is much harder than simulating from the stationary distribution in MCMC. The algorithm presented in the paper is based on constructing a killed Brownian motion, for which it is straightforward to derive the killing rate so that the killed Brownian motion has the posterior distribution as its quasi-stationary distribution. However simulating killed Brownian motion is non-trivial, and requires the use of ideas from exact simulation of diffusions. Simulating from the quasi-stationary distribution is then achieved by embedding the forward simulation of the killed Brownian motion within a Sequential Monte Carlo Algorithm. The paper shows that the resulting algorithm, called ScaLE, can have good properties in terms of how it scales with the number of data points -- as one can simulate the killed Brownian motion exactly whilst only using subsamples of data at each iteration.

This work was developed as part of the i-like research project, but has also been supported by both the Bayes4Health and CoSInES grants.

Stochastic gradient Markov chain Monte Carlo

Stochastic gradient Markov chain Monte Carlo algorithms are increasingly popular approaches for approximate sampling from posterior distributions in big data. One example is stochastic gradient Langevin dynamics, which can be viewed as a computationally-efficient approximation to the MALA algorithm – but where the Metropolis-Hastings accept-reject step is removed and the gradient of the posterior is approximated using a small sub-sample of the data at each iteration. This algorithm is much faster to run than MALA, but no longer target the true posterior. However in many applications where there is limited compute resource, it can give more accurate results than standard MCMC algorithms.

This paper reviews this class of algorithms, gives an informal overview of recent theoretical results on their Monte Carlo and approximation error, compares them against standard MCMC algorithms, and points to open research challenges.

On the marginal likelihood and cross-validation

In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through k-fold partitioning or leave-p-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-p-out cross-validation averaged over all values of p and all held-out test sets when using the log posterior predictive probability as the scoring rule.

Testing for dependence on tree structures

Tree-like structures are abundant in the empirical sciences as they can summarize high-dimensional data and show latent structure among many samples in a single framework. Prominent examples include phylogenetic trees or hierarchical clustering derived from genetic data. Currently, users employ ad hoc methods to test for association between a given tree and a response variable, which reduces reproducibility and robustness. This paper introduces treeSeg, a simple to use and widely applicable methodology with high power for testing between all levels of hierarchy for a given tree and the response while accounting for the overall false positive rate. This method allows for precise uncertainty quantification and therefore, increases interpretability and reproducibility of such studies across many fields of science.

Stochastic gradient MCMC for nonlinear state space models

Current stochastic gradient MCMC methods have been designed for models where the log-likelihood can be written as a sum and it is easy to calculate the gradient of each term in the sum. Unfortunately, the log-likelihood for state-space models cannot be written in this way as the gradient terms involve calculating integrals. This paper shows how one can still apply stochastic gradient MCMC to state space models by introducing a further approximation (using buffering ideas) when calculating the gradients; it presents theory on the resulting error of the method and gives comparisons that show when the method can be more efficient than standard MCMC approaches.

Reversible Jump PDMP Samplers for Variable Selection

This paper extends recent PDMP-based sampling methods so that they can be used for variable choice problems, such as fitting GLMs jointly with deciding which covariates to include in the model. The algorithm developed is easy to implement if we have a PDMP sampler that can explore the posterior for a given model (i.e. a given set of covariates to include). The only difference is two additional moves, one which re-introduces covariates into the model with a constant rate; and the other removes covariates if the co-efficient associated with their effect is ever 0.

Martingale posterior distributions

This paper presents a new approach to sampling from posterior distributions. It is based on the idea that parameters of interest would be known exactly if we had an infinitely large data set. Thus if we could simulate new data given the observed data, then we could simulate a large data set and hence obtain a parameter value from the posterior. This idea turns sampling from a posterior distribution into repeated sampling from a predictive distribution. This opens up a new approach to Bayesian non-parametrics, and has potential as a way to marry Bayesian approaches to uncertainty quantification with powerful machine learning algorithms for prediction.

Concave-Convex PDMP-based sampling

The implementational bottleneck for PDMP samplers is simulating the event times at which the dynamics of the sampler change. Simulating these event times is equivalent to simulating events in a time-inhomogeneous Poisson process. We show that if the rate of this time-inhomogeneous Poisson process can be decomposed into the sum of a concave and convex function then there are efficient and automatic methods for simulating the event times that just need to evaluate the concave and convex functions. These are simple to implement and we provide code for doing so, and can be more efficient that other existing simulation methods.

Comparison of Markov chains via weak Poincaré inequalities

When carrying out Bayesian inference in complex statistical models, it can be the case that standard computational methods cannot be applied, as certain key quantities (likelihoods, normalising constants, etc.) cannot be evaluated. Nevertheless, for some important model classes, there exist “exact-approximate” methods which are able to bypass this intractability and deliver an implementable algorithm which emulates the ‘ideal’ method which one would like to use, without incurring systematic bias.

In this paper, we study the theoretical properties of some of these “exact-approximate” MCMC algorithms, using the perspective of Markov chain comparison theory. This approach neatly mirrors the practical construction of the algorithms themselves, showing that for well-designed algorithms, one can perform “almost as well as” the ideal algorithm, in a precise quantitative sense. Moreover, relative to previous studies, the comparison techniques used herein are particularly straightforward and transparent.

With this perspective, we are able to provide new insights into the practical use of so-called “pseudo-marginal” algorithms, with applications to Approximate Bayesian Computation (ABC), particle marginal Metropolis--Hastings (PMMH), and related procedures.

Inference on extended-spectrum beta-lactamase E- coli and Klebsiella pneumoniae

The paper introduces an innovative modelling approach to investigate the spread of antimicrobial-resistant bacteria in Malawi. The model adopts an individual-based framework considering the colonisation status of each individual in the population while encompassing various relevant factors in the colonisation rate, like individuals’ covariates, seasonal influence, and environmental effect. Inference is pursued through the algorithm SMC2, where an SMC nested in another SMC is used to perform particles approximation of both the posterior distributions over the individuals and the parameters.

Approximating optimal SMC proposal distribution in... epidemic models

Conventional SMC implementations, such as the bootstrap particle filter and the auxiliary particle filter, prove to be inefficient when applied to high-dimensional state-space models, and, in particular, in individuals-based models for infectious disease modelling. This inefficiency arises mainly from the discrepancy between the proposal distribution on the individuals’ states and future observations. To overcome this issue, the paper proposes to build proposal distributions that take into account future observations by exploiting two crucial properties of the model:

  1. The ability to analytically calculate the optimal proposal distribution for a single individual given future observations and the future infection rate of each individual,
  2. The independence of dynamics among individuals when conditioned on their infection rates.