Abstract The successes of speech and language technology research in recent years has led to the deployment of  human language technology (HLT) in an increasing number of high-visibility, high-consequence applications.  Well-established approaches to evaluating HLT[...]

Abstract Large language models (LMs) are pretrained on diverse data sources—news, discussion forums, books, online encyclopedias. A significant portion of this data includes facts and opinions which, on one hand, celebrate democracy and diversity of[...]