1. Goff, DC, Lloyd-Jones, DM, Bennett, G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk. Circulation. 2014;129(25suppl 2):S49–73.
Google Scholar | Medline2. DeFilippis, AP, Young, R, Carrubba, CJ, et al. An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Ann Intern Med. 2015;162(4):266–75.
Google Scholar | Crossref | Medline3. Ridker, PM, Cook, NR. Statins: new American guidelines for prevention of cardiovascular disease. Lancet. 2013;382(9907):1762–5.
Google Scholar | Crossref | Medline4. Davis, SE, Lasko, TA, Chen, G, Siew, ED, Matheny, ME. Calibration drift in regression and machine learning models for acute kidney injury. J Am Med Inform Assoc. 2017;24(6):1052–61.
Google Scholar | Crossref5. Finlayson, SG, Subbaswamy, A, Singh, K, et al. The clinician and dataset shift in artificial intelligence. N Engl J Med. 2021;385:283–6.
Google Scholar | Crossref | Medline6. Van Calster, B, Nieboer, D, Vergouwe, Y, De Cock, B, Pencina, MJ, Steyerberg, EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167–76.
Google Scholar | Crossref | Medline7. Hosmer, DW, Lemeshow, S, Sturdivant, RX. Applied Logistic Regression. Vol. 398. New York: John Wiley & Sons; 2013.
Google Scholar | Crossref8. Harrell, F . Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. New York: Springer; 2015.
Google Scholar | Crossref9. Cox, DR . Two further applications of a model for binary regression. Biometrika. 1958;45(3/4):562–5.
Google Scholar | Crossref10. Dalton, JE . Flexible recalibration of binary clinical prediction models. Stat Med. 2013;32(2):282–9.
Google Scholar | Crossref | Medline11. Platt, JC . Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classifiers. 1999;10(3):61–74.
Google Scholar12. Zadrozny, B, Elkan, C. Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers. In: ICML. Vol. 1. Citeseer; 2001. p 609–16.
Google Scholar13. Zadrozny, B, Elkan, C. Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; ACM; 2002. p 694–9.
Google Scholar | Crossref14. Niculescu-Mizil, A, Caruana, R. Predicting good probabilities with supervised learning. In: Proceedings of the 22nd International Conference on Machine Learning; ACM; 2005. p 625–32.
Google Scholar | Crossref15. Jiang, X, Osl, M, Kim, J, Ohno-Machado, L. Calibrating predictive model estimates to support personalized medicine. J Am Med Inform Assoc. 2011;19(2):263–74.
Google Scholar | Crossref16. Baker, SG, Cook, NR, Vickers, A, Kramer, BS. Using relative utility curves to evaluate risk prediction. J R Stat Soc Ser A Stat Soc. 2009;172(4):729–48.
Google Scholar | Crossref | Medline17. Vickers, AJ, Elkin, EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74.
Google Scholar | SAGE Journals18. Steyerberg, EW, Vickers, AJ. Decision curve analysis: a discussion. Med Decis Making. 2008;28(1):146–9.
Google Scholar | SAGE Journals19. Pepe, M, Janes, H. Methods for Evaluating Prediction Performance of Biomarkers and Tests. New York: Springer; 2013.
Google Scholar | Crossref20. Kerr, KF, Brown, MD, Zhu, K, Janes, H. Assessing the clinical impact of risk prediction models with decision curves: guidance for correct interpretation and appropriate use. J Clin Oncol. 2016;34(21):2534–40.
Google Scholar | Crossref21. Pauker, SG, Kassirer, JP. The threshold approach to clinical decision making. N Engl J Med. 1980;302(20):1109–17.
Google Scholar | Crossref | Medline22. Baker, SG, Kramer, BS. Peirce, Youden, and receiver operating characteristic curves. Am Stat. 2007;61(4):343–6.
Google Scholar | Crossref23. Kerr, KF, Marsh, TL, Janes, H. The importance of uncertainty and opt in vs. opt out: best practices for decision curve analysis. Med Decis Making. 2019;39(5):491–2.
Google Scholar | SAGE Journals24. Kerr, KF, Brown, MD, Marsh, TL, Janes, H. Assessing the clinical impact of risk models for opting out of treatment. Med Decis Making. 2019;39(2):86–90.
Google Scholar | SAGE Journals | ISI25. Van Calster, B, Vickers, AJ. Calibration of risk prediction models impact on decision-analytic performance. Med Decis Making. 2015;35(2):162–9.
Google Scholar | SAGE Journals26. Pepe, MS, Fan, J, Feng, Z, Gerds, T, Hilden, J. The net reclassification index (NRI): a misleading measure of prediction improvement even with independent test data sets. Stat Biosci. 2015;7(2):282–95.
Google Scholar | Crossref | Medline27. Baker, SG, Van Calster, B, Steyerberg, EW. Evaluating a new marker for risk prediction using the test tradeoff: an update. Int J Biostat. 2012;8(1):1–37.
Google Scholar | Crossref | ISI28. Metz, CE . Basic principles of ROC analysis. In: Seminars in Nuclear Medicine. Vol. 8. Amsterdam: Elsevier; 1978. p 283–98.
Google Scholar | Crossref29. Friedman, J, Hastie, T, Tibshirani, R. The Elements of Statistical Learning. Vol. 1. Springer Series in Statistics . New York: Springer; 2001.
Google Scholar30. Vickers, AJ, Van Calster, B, Steyerberg, E. Decision curves, calibration, and subgroups. J Clin Oncol. 2017;35(4):472–3.
Google Scholar | Crossref31. Bild, DE, Bluemke, DA, Burke, GL, Detrano, R, Diez Roux, AV, Folsom, AR, et al. Multi-ethnic study of atherosclerosis: objectives and design. Am J Epidemiol. 2002;156(9):871–81.
Google Scholar | Crossref | Medline32. Mishra, A . ClinicalUtilityRecal: recalibration methods for improved clinical utility of risk scores; 2020. R package version 0.1.0. Available from: https://CRAN.R-project.org/package=ClinicalUtilityRecal
Google Scholar33. Steyerberg, EW, Borsboom, GJ, van Houwelingen, HC, Eijkemans, MJ, Habbema, JDF. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med. 2004;23(16):2567–86.
Google Scholar | Crossref | Medline34. Vergouwe, Y, Nieboer, D, Oostenbrink, R, Debray, TP, Murray, GD, Kattan, MW, et al. A closed testing procedure to select an appropriate method for updating prediction models. Stat Med. 2017;36(28):4529–39.
Google Scholar | Crossref | Medline35. Steyerberg, EW, Harrell, FE, Borsboom, GJ, Eijkemans, M, Vergouwe, Y, Habbema, JDF. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54(8):774–81.
Google Scholar | Crossref | Medline36. Kull, M, Silva Filho, T, Flach, P. Beta calibration: a well-founded and easily implemented improvement on logistic calibration for binary classifiers. In: Artificial Intelligence and Statistics; Proceedings of Machine Learning Research; 2017. p 623–31.
Google Scholar37. Kerr, KF, Brown, M, Janes, H. Reply to AJ Vickers et al. J Clin Oncol. 2016;35(4):473–5.
Google Scholar | Crossref | Medline