Drew Prinster

Drew Prinster

PhD student in computer science at Johns Hopkins, focusing on trustworthy machine learning in healthcare.

Publications

Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)

Published in The International Conference on Machine Learning (ICML), 2024

Paper at ICML 2024. Demonstrates how conformal prediction can theoretically extend to any data distribution (i.e., not only exchangeable or quasi-exchangeable ones), with practical experiments focused on common settings of AI/ML agents including multiround synthetic protein design and active learning.

Recommended citation: Prinster, D.*, Stanton, S.*, Saria, S., & Liu, A. (2023). JAWS-X: Addressing Efficiency Bottlenecks of Conformal Prediction Under Standard and Feedback Covariate Shift. In International Conference on Machine Learning. PMLR. https://arxiv.org/abs/2405.06627

JAWS-X: Addressing Efficiency Bottlenecks of Conformal Prediction Under Standard and Feedback Covariate Shift

Published in The Fortieth International Conference on Machine Learning (ICML), 2023

Accepted for an Oral Presentation at ICML 2023 (top ~2% of submissions) and building on our previous “JAWS” framework (NeurIPS 2022), this paper presents JAWS-X, a collection of methods for efficiently estimating predictive confidence intervals for black-box predictors under standard and feedback covariate shift. Our JAWS-X methods achieve distribution-free, finite-sample guarantees while flexibly balancing statistical and computational efficiency.

Recommended citation: Prinster, D., Saria, S., & Liu, A. (2023). JAWS-X: Addressing Efficiency Bottlenecks of Conformal Prediction Under Standard and Feedback Covariate Shift. In International Conference on Machine Learning. PMLR. https://proceedings.mlr.press/v202/prinster23a.html

JAWS: Auditing Predictive Uncertainty Under Covariate Shift ([previously] Predictive Inference Under Covariate Shift)

Published in Advances in Neural Information Processing Systems (NeurIPS), 2022

JAWS is a collection of wrapper methods for distribution-free predictive inference when the common data exchangeability (e.g., i.i.d.) assumption is violated due to shifts in the input data distribution (standard covariate shifts). JAWS is based on our core method JAW–the JAckknife+ Weighted for standard covariate shift–and also includes computationally efficient Approximations of JAW (JAWA) using higher-order influence functions.

Recommended citation: Prinster, D., Liu, A., & Saria, S. (2022). JAWS: Auditing Predictive Uncertainty Under Covariate Shift. In Advances in Neural Information Processing Systems. https://papers.nips.cc/paper_files/paper/2022/hash/e944bacecce6b06374ac39b260348db0-Abstract-Conference.html