Effectiveness and Information Quality Perception of an AI Model Card: A Study Among Non-Experts

Bracamonte, V.; Pape, S.; Löbner, S. and Tronnier, F.

In 20th Annual International Conference on Privacy, Security & Trust (PST 2023), Copenhagen, Denmark, 21-23 August, 2023.

Abstract

With the rising popularity of artificial intelligence (AI) applications, the use of the underlying AI models has spread to the general public. These AI models have limitations and biases, and knowing about their characteristics could promote their safe use. Although there is some information about AI models available, in the form of AI Model Cards, there is little research on how useful this information is for non-expert users. In this paper, we conduct an experiment to evaluate the effectiveness and perception of information quality of the Model Card of a currently available AI, and compare it with shorter versions. The results show that participants can use the Model Card to answer questions about the AI but they are less confident about their answers compared to shorter versions. In addition, the full Model Card is considered less understandable and interpretable compared with a short version. On the other hand, a short version had a negative effect effect on perceived trustworthiness of the AI, but in all cases the participants had a positive attitude towards seeking information about the AI.

PDFDOILinkLinkLinkLinkBibtexmachine learning

Bibtex

@InProceedings{BPLT23pst,
  author    = {Vanessa Bracamonte and Sebastian Pape and Sascha L\"obner and Frederic Tronnier},
  title     = {Effectiveness and Information Quality Perception of an AI Model Card: A Study Among Non-Experts},
  booktitle = {20th Annual International Conference on Privacy, Security \& Trust ({PST} 2023), Copenhagen, Denmark, 21-23 August},
  year      = {2023},
  month     = {08},
  doi       = {10.1109/PST58708.2023.10320197},
  keywords  = {machine learning},
  url       = {https://ieeexplore.ieee.org/document/10320197},
}

PDF