Historical Methods for AI Evaluations, Assessments, and Audits

In this paper, we present historical analysis as a critical method for integration within AI evaluation, assessments, and audit protocols. Drawing from our own experiences conducting historical analysis as part of an internal impact assessment on AI-enabled virtual agents, we provide a framework for how to integrate historical analysis with other evaluation methodologies. We argue that historical analysis can help point to a wider range of risks and impacts, contributing to the harm discovery phase of evaluation protocols. Additionally, we argue that historical analysis can help substantiate findings and present them in their appropriate social, cultural, political, and economic context.