Artificial intelligence and validity

In: New directions for evaluation: a publication of the American Evaluation Association, Band 2023, Heft 178-179, S. 85-95

Abstract

AbstractThis article explores the interaction between artificial intelligence (AI) and validity and identifies areas where AI can help build validity arguments, and where AI might not be ready to contribute to our work in establishing validity. The validity of claims made in an evaluation is critical to the field, since it highlights the strengths and limitations of findings and can contribute to the utilization of the evaluation. Within this article, validity will be discussed within two broad categories: quantitative validity and qualitative trustworthiness. Within these categories, there are multiple types of validity, including internal validity, measurement validity, establishing trustworthiness, and credibility, to name a few. Each validity type will be discussed within the context of AI, examining if and how AI can be leveraged (or not) to help establish a specific validity type, or where it might not be possible for AI (in its current form) to contribute to the development of a validity argument. Multiple examples will be provided throughout the article to highlight the concepts introduced.