Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)

Drew Prinster, Samuel Stanton, Anqi Liu, Suchi Saria

July, 2024

@InProceedings{pmlr-v235-prinster24a,
  title = {Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)},
  author = {Prinster, Drew and Stanton, Samuel Don and Liu, Anqi and Saria, Suchi},
  booktitle = {Proceedings of the 41st International Conference on Machine Learning},
  pages = {41086--41118},
  year = {2024},
  editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix},
  volume = {235},
  series = {Proceedings of Machine Learning Research},
  month = {21--27 Jul},
  publisher = {PMLR},
  url = {https://proceedings.mlr.press/v235/prinster24a.html}
}

Informal Summary

Demonstrates how conformal prediction can theoretically extend to any data distribution (i.e., not only exchangeable or quasi-exchangeable ones), with practical experiments focused on common settings of AI/ML agents including multiround synthetic protein design and active learning.