CoChat
Sign Up
PreprintCited by 1

Designing Decision Support Systems Using Counterfactual Prediction Sets

Straitouri, Eleni, Rodriguez, Manuel Gomez·2023·arXiv
DOI 10.48550/arxiv.2306.03928arXiv 2306.03928
Open this paper in CoChat — free account

Abstract

Decision support systems for classification tasks are predominantly designed to predict the value of the ground truth labels. However, since their predictions are not perfect, these systems also need to make human experts understand when and how to use these predictions to update their own predictions. Unfortunately, this has been proven challenging. In this context, it has been recently argued that an alternative type of decision support systems may circumvent this challenge. Rather than providing a single label prediction, these systems provide a set of label prediction values constructed using a conformal predictor, namely a prediction set, and forcefully ask experts to predict a label value from the prediction set. However, the design and evaluation of these systems have so far relied on stylized expert models, questioning their promise. In this paper, we revisit the design of this type of systems from the perspective of online learning and develop a methodology that does not require, nor assumes, an expert model. Our methodology leverages the nested structure of the prediction sets provided by any conformal predictor and a natural counterfactual monotonicity assumption to achieve an exponential improvement in regret in comparison to vanilla bandit algorithms. We conduct a large-scale human subject study ($n = 2{,}751$) to compare our methodology to several competitive baselines. The results show that, for decision support systems based on prediction sets, limiting experts' level of agency leads to greater performance than allowing experts to always exercise their own agency. We have made available the data gathered in our human subject study as well as an open source implementation of our system at https://github.com/Networks-Learning/counterfactual-prediction-sets.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2306.03928

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Human-system decision accuracy is higher when the human choice is restricted to the outputs that are in the proposed set.

    “It was shown that for certain tasks, the human-system decision accuracy indeed is higher when the human choice is restricted to the outputs that are in the proposed set [78].”

    From On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives (2024) · cites this paper as [78]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 64

  1. Agrawal, S., Avadhanula, V., Goyal, V., and Zeevi, A. (2019). Mnl-bandit: A dynamic learning approach to assortment selection. Operations Research, 67(5):1453-1485.
  2. Angelopoulos, A. N. and Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends® in Machine Learning, 16(4):494-591.
  3. Audibert, J.-Y., Bubeck, S., and Munos, R. (2010). Best arm identification in multi-armed bandits. In COLT, pages 41-53.
  4. Babbar, V., Bhatt, U., and Weller, A. (2022). On the utility of prediction sets in human-ai teams. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence.
  5. Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., and Horvitz, E. (2019). Beyond accuracy: The role of mental models in human-ai team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing.
  6. Bates, S., Angelopoulos, A., Lei, L., Malik, J., and Jordan, M. (2021). Distribution-free, risk-controlling prediction sets. Journal of the ACM (JACM), 68(6):1-34.
  7. Beach, L. R. (1993). Broadening the definition of decision making: The role of prechoice screening of options. Psychological Science, 4(4):215-220.
  8. Ben-Akiva, M. and Boccara, B. (1995). Discrete choice models with latent choice sets. International Journal of Research in Marketing, 12(1):9-24.
  9. Bordt, S. and von Luxburg, U. (2020). When humans and machines make joint decisions: A non-symmetric bandit model. arXiv preprint arXiv:2007.04800. · arXiv
  10. Bouneffouf, D., Rish, I., and Aggarwal, C. (2020). Survey on applications of multi-armed and contextual bandits. In 2020 IEEE Congress on Evolutionary Computation (CEC), pages 1-8. IEEE.
  11. Chen, W., Wang, Y., and Yuan, Y. (2013). Combinatorial multi-armed bandit: General framework and applications. In International conference on machine learning, pages 151-159.
  12. PMLR.
  13. Chernev, A., Böckenholt, U., and Goodman, J. (2015). Choice overload: A conceptual review and meta- analysis. Journal of Consumer Psychology, 25(2):333-358.
  14. Chzhen, E., Denis, C., Hebiri, M., and Lorieul, T. (2021). Set-valued classification-overview via a unified framework. arXiv preprint arXiv:2102.12318. · arXiv
  15. De, A., Koley, P., Ganguly, N., and Gomez-Rodriguez, M. (2020). Regression under human assistance. In Proceedings of the AAAI Conference on Artificial Intelligence.
  16. De, A., Okati, N., Zarezade, A., and Gomez-Rodriguez, M. (2021). Classification under human assistance. In Proceedings of the AAAI Conference on Artificial Intelligence.
  17. Ding, K., Li, J., and Liu, H. (2019). Interactive anomaly detection on attributed networks. In Proceedings of the twelfth ACM international conference on web search and data mining, pages 357-365.
  18. Durand, A., Achilleos, C., Iacovides, D., Strati, K., Mitsis, G. D., and Pineau, J. (2018). Contextual bandits for adapting treatment in a mouse model of de novo carcinogenesis. In Machine learning for healthcare conference, pages 67-82. PMLR.
  19. Einbinder, B.-S., Bates, S., Angelopoulos, A. N., Gendler, A., and Romano, Y. (2022). Conformal prediction is robust to label noise. arXiv preprint arXiv:2209.14295. · arXiv
  20. Even-Dar, E., Mannor, S., Mansour, Y., and Mahadevan, S. (2006). Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. Journal of machine learning research, 7(6).
  21. Gibbs, I. and Candes, E. (2021). Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34:1660-1672.
  22. Haynes, G. A. (2009). Testing the boundaries of the choice overload phenomenon: The effect of number of options and time pressure on decision difficulty and satisfaction. Psychology & Marketing, 26(3):204-212.
  23. Heiss, F. (2016). Discrete choice methods with simulation. Taylor & Francis.
  24. Jiao, W., Atwal, G., Polak, P., Karlic, R., Cuppen, E., Danyi, A., de Ridder, J., van Herpen, C., Lolkema, M. P., Steeghs, N., et al. (2020). A deep learning system accurately classifies primary and metastatic cancers using passenger mutation patterns. Nature communications, 11(1):1-12.
  25. Kahneman, D. and Tversky, A. (1973). On the psychology of prediction. Psychological review, 80(4):237.
  26. Kahneman, D. and Tversky, A. (1984). Choices, values, and frames. American psychologist, 39(4):341.
  27. Kroon, A. D., Mooij, J., and Belgrave, D. (2022). Causal bandits without prior knowledge using separating sets. In First Conference on Causal Learning and Reasoning.
  28. Kuksov, D. and Villas-Boas, J. M. (2010). When more alternatives lead to less choice. Marketing Science, 29(3):507-524.
  29. Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., and Tan, C. (2021). Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471. · arXiv
  30. Lattimore, F., Lattimore, T., and Reid, M. D. (2016). Causal bandits: Learning good interventions via causal inference. Advances in Neural Information Processing Systems, 29.
  31. Lee, S. and Bareinboim, E. (2018). Structural causal bandits: where to intervene? Advances in Neural Information Processing Systems, 31.
  32. Liu, H., Tian, Y., Chen, C., Feng, S., Chen, Y., and Tan, C. (2023). Learning human-compatible representations for case-based decision support. In The Eleventh International Conference on Learning Representations.
  33. Loewenstein, G., Lerner, J. S., et al. (2003). The role of affect in decision making. Handbook of affective science, 619(642):3.
  34. Loewenstein, G. and Prelec, D. (1992). Anomalies in intertemporal choice: Evidence and an interpretation. The Quarterly Journal of Economics, 107(2):573-597.
  35. Lubars, B. and Tan, C. (2019). Ask not what ai can do, but what ai should do: Towards a framework of task delegability. Advances in Neural Information Processing Systems, 32:57-67.
  36. Misra, K., Schwartz, E. M., and Abernethy, J. (2019). Dynamic online pricing with incomplete information using multiarmed bandit experiments. Marketing Science, 38(2):226-252.
  37. Mortier, T., Wydmuch, M., Dembczyński, K., Hüllermeier, E., and Waegeman, W. (2021). Efficient set-valued prediction in multi-class classification. Data Mining and Knowledge Discovery, 35(4):1435-1469.
  38. Mozannar, H. and Sontag, D. (2020). Consistent estimators for learning to defer to an expert. In International Conference on Machine Learning, pages 7076-7087.
  39. Mueller, J. W., Syrgkanis, V., and Taddy, M. (2019). Low-rank bandit methods for high-dimensional dynamic pricing. Advances in Neural Information Processing Systems, 32.
  40. Nguyen, V.-L. and Hüllermeier, E. (2021). Multilabel classification with partial abstention: Bayes-optimal prediction under label independence. Journal of Artificial Intelligence Research, 72:613-665.
  41. Okati, N., De, A., and Gomez-Rodriguez, M. (2021). Differentiable learning under triage. In Advances in Neural Information Processing Systems.
  42. Papenmeier, A., Englebienne, G., and Seifert, C. (2019). How model accuracy and explanation fidelity influence user trust. arXiv preprint arXiv:1907.12652. · arXiv
  43. Podkopaev, A. and Ramdas, A. (2021). Distribution-free uncertainty quantification for classification under label shift. In Uncertainty in Artificial Intelligence, pages 844-853.
  44. PMLR.
  45. Raghu, M., Blumer, K., Corrado, G., Kleinberg, J., Obermeyer, Z., and Mullainathan, S. (2019). The algorithmic automation problem: Prediction, triage, and human effort. arXiv preprint arXiv:1903.12220. · arXiv
  46. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015). Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-252.
  47. Sadinle, M., Lei, J., and Wasserman, L. (2019). Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association, 114(525):223-234.
  48. Schölkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K., and Mooij, J. (2012). On causal and anticausal learning. In Proceedings of the 29th International Coference on International Conference on Machine Learning, pages 459-466.
  49. In Bengio, Y. and LeCun, Y., editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
  50. Slivkins, A. (2019). Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning, 12(1-2):1-286.
  51. Steyvers, M., Tejeda, H., Kerrigan, G., and Smyth, P. (2022). Bayesian modeling of human-ai complementarity. Proceedings of the National Academy of Sciences, 119(11):e2111547119.
  52. Straitouri, E., Wang, L., Okati, N., and Gomez-Rodriguez, M. (2023). Improving expert predictions with conformal prediction. In Proceedings of the 40th International Conference on Machine Learning.
  53. Stutz, D., Roy, A. G., Matejovicova, T., Strachan, P., Cemgil, A. T., and Doucet, A. (2023). Conformal prediction under ambiguous ground truth. arXiv preprint arXiv:2307.09302. · arXiv
  54. Suresh, H., Lao, N., and Liccardi, I. (2020). Misplaced trust: Measuring the interference of machine learning in human decision-making. In 12th ACM Conference on Web Science, pages 315-324.
  55. Tibshirani, R. J., Barber, R. F., Candes, E. J., and Ramdas, A. (2020). Conformal prediction under covariate shift. Tversky, A. and Kahneman, D. (1981). The framing of decisions and the psychology of choice. science, 211(4481):453-458.
  56. Vodrahalli, K., Gerstenberg, T., and Zou, J. (2022). Uncalibrated models can improve human-ai collaboration. In Advances in Neural Information Processing Systems.
  57. Vovk, V. (2012). Conditional validity of inductive conformal predictors. In Asian conference on machine learning, pages 475-490.
  58. PMLR.
  59. Vovk, V., Gammerman, A., and Shafer, G. (2005). Algorithmic Learning in a Random World. Springer-Verlag, Berlin, Heidelberg.
  60. Wang, X. and Yin, M. (2021). Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making. In 26th International Conference on Intelligent User Interfaces, pages 318-328.
  61. Whitehill, J., Mohan, K., Seaton, D., Rosen, Y., and Tingley, D. (2017). Mooc dropout prediction: How to measure accuracy? In Proceedings of the fourth (2017) acm conference on learning@ scale, pages 161-164.
  62. Yang, G., Destercke, S., and Masson, M.-H. (2017). Cautious classification with nested dichotomies and imprecise probabilities. Soft Computing, 21(24):7447-7462.
  63. Yin, M., Wortman Vaughan, J., and Wallach, H. (2019). Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems, pages 1-12.
  64. Zhang, Y., Liao, Q. V., and Bellamy, R. K. (2020). Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 295-305.

Cited by

Sorted by citations

Authors
Straitouri, E., & Rodriguez, M. G.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2306.03928
arXiv
2306.03928
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review