https://www.metaculus.com/notebooks/9613/explainable-ai-and-trust-issues/

AI researchers exploring ways to increase trust in AI recognize that one barrier to trust, often, is a lack of explanation. This recognition has led to the development of the field of Explainable Artificial Intelligence (XAI). In their paper Formalizing Trust in Artificial Intelligence, Jacovi et al. classify an AI system as trustworthy to a contract if it is capable of maintaining this contract: A recommender algorithm might be trusted to make good recommendations, and a classification algorithm might be trusted to classify things appropriately. When a classification algorithm makes grossly inappropriate classifications, we feel betrayed, and the algorithm loses our trust. (Of course, a system may be untrustworthy even as we continue to place trust in it.) This essay explores current legal implementations of XAI as they relate to explanation, trust, and human data subjects (e.g. users of Google or Facebook)—while forecasting outcomes relevant to XAI.