How accurately do leading AI models respond to questions about Black history? “I would give them probably a C,” Black history educator and content creator Ernest Crim III told Proof News in an interview on our YouTube channel.

Using our AI testing software, which simultaneously queries five leading AI models, Crim asked the models questions he thought his students might ask related to Black history and experiences. He published the results on his TikTok channel and spoke with us about his work on our brand new YouTube and TikTok channels.

In his testing, Crim found that some of the AI models not only flubbed easy questions — like “who is Charlotte Ray?” (she is the first Black woman to become a lawyer in the U.S.) — but also were not nuanced enough on questions that they probably shouldn’t try to answer definitively like “how would you define racism?”

Crim’s project kicks off a new initiative at Proof, where we are making our investigative tools available to content creators to use for their own research in their own domains of expertise. 

We also believe news should be labeled the way food is labeled – with its ingredients, so you know exactly what you’re ingesting. So we put together an ingredients label for Crim’s investigation.

We believe that journalism should seek first and foremost to inform the public – and so, rather than keeping our tools to ourselves, we are collaborating with creators on YouTube and TikTok who have much larger audiences than we do. 

We will interview the creators about their research in a video interview series called “Proof Ingredients.” Our first interview in the series showcased the work of Proof reporter Annie Gilbertson, whose investigation revealed the YouTube videos that were swiped to train leading AI models.

Subscribe to our YouTube and TikTok channels to follow along on this journey!