Research

Your Own Best Critic: Extrapolating Predictive Checks for Stronger Model Criticism from Prior Information Alone
Cademartori, C.

(Draft manuscript available upon request.)
The prior and posterior predictive checks were originally conceived as methods for criticizing Bayesian models based on their ability to reproduce features of observed data. More recently, it has been observed that the prior predictive check is also useful for determining if a model is consistent with our prior information. We argue that this idea remains underutilized, and we introduce an extention of the classical check -- the prior extrapolatory check -- to expose the model to more powerful criticism on the basis of prior information. With this new check, we demonstrate that prior information can be uniquely valuable for model critcism, potentially revealing deficits which the data alone would obscure, such as misspecified model features which promote overfitting rather than underfitting.

Variance Deltas for Visualizing and Explaining Posterior Uncertainty
Cademartori, C.

When working with limited, observational data and a complex, uncontrolled data generating process, applied statisticians are often beset with not only statistical uncertainty, but also meta-uncertainty: We do not know why our inference is uncertain. While the first-order uncertainty problem can only be solved with more data or prior information, the meta-uncertainty problem can often be addressed by peering more deeply into our probability models. In the Bayesian setting, we propose a method for explaining posterior uncertainty for a quantity of interest in terms of other quantities in our model. We leverage the factorization structure of the Bayesian model to automatically generate plausible explanations, which are arranged into a tree-structured visualization (which we refer to as a variance delta). The statistician can then interactively refine the potential explanations through opreations that merge, divide, and extend them.

Impact of Hurricane Florence on buprenorphine transactions for opioid use disorder: A dual-perspective synthetic control study
Murphy, E., Cademartori, C., Kline, D., Hurley, R., Yong, J., McDonnell, C., Helper, S., Adams, M.

This project applies a Bayesian latent factor model of the untreated potential outcome to estimate the treatment effect of Hurricane Florence evacuation orders on access to Buprenorphine for opioid use disorder in coastal North Carolina. The approach is essentially a Bayesian adaptation of synthetic control methods, relying on correlations between treated and untreated units in the pre-treatment period to estimate the untreated outcome of the treated unit in the post-treatment period. Using prescription transactions data from the IQVIA datbase, we estimate a (statistically and practically) significant effect of the hurricane evacuation orders in the weeks immediately following landfall, followed by a rebound.

Identifiability and Falsifiability: Two Challenges for Bayesian Model Expansion.
Cademartori, C.

(An interactive blog post explaining the main ideas of this work may be found here.)
In this work, we use information-theoretic tools to investigate the properties of Bayesian models under a process of model expansion, whereby a simpler base model is extended to a larger, higher-dimensional model which embeds the base model as a special case. We find that this process tends to lead to weakening identification of model parameters and degrading power of model checks. We argue that these problems may be characteristic of model expansion in general, establishing bounds that indicate a tradeoff between them: the more an expansion avoids one problem, the more it is likely to increase the severity of the other. Finally, we consider the methodological consequences of these conclusions. In particular, we demonstrate in examples that methods capable of leveraging the dependence structure of the posterior distribution can partly overcome the challenges that model expansion poses for many classic inferential tools.

A Non-asymptotic Analysis of Generalized Approximate Message Passing Algorithms with Right Rotationally Invariant Designs
Cademartori, C., Rush, C.

Approximate message passing procedures are a class of algorithms derivable as Gaussian approximations to certain belief and expectation propagation algorithms for high-dimensional regression problems. Many of these algorithms have the remarkable property that their error at any iteration can be predicted to high accuracy by a computable recursion called the state evolution. This work studies a state evolution for the Generalized Vector Approximate Message Passing algorithm, which extends the classic AMP algorithm to apply to generalized linear models and to design matrices which are potentially severely ill-conditioned. We show under general conditions that the average error of GVAMP at any iteration converges at exponentially fast rates to these state evolution predictions.

Strength in Numbers: A Joint Posterior $p$-Value for Increasing the Frequentist Power of Bayesian Model Diagnostics.
Cademartori, C.

(In progress) This project is motivated by the old observation that the posterior predictive $p$-value is conservative in the sense that the probability of observing $p$-value $p_{obs}$ under frequentist replications of the data generating process is usually less than $p_{obs}$. We argue that this problem increases in severity as the model dimension grows, putting pressure on previous arguments that the conservativity property is unproblematic in practice. We propose a joint $p$-value computed for multiple test statistics $T_i$ simultaneously, which we develop a frequency bound for and argue is capable of overcoming the conservativity problem in many cases.