Engadget has been testing and reviewing consumer tech since 2004. Our stories may include affiliate links; if you buy something through a link, we may earn a commission. Read more about how we evaluate products.
Meta made a fact-checking AI to help verify Wikipedia citations
The tool could one day allow editors to correct footnotes at scale.
In 2020, the Wikipedia community was engulfed in scandal when it came out that a US teen had written 27,000 entries in a language they didn’t speak. The episode was a reminder that the online encyclopedia is not a perfect source of information. Sometimes people will attempt to edit Wikipedia entries out of malice, but frequently factual errors come from some well-intentioned individual making a mistake.
That's a problem Facebook parent company Meta thinks it can help address with the help of AI. The social media giant set its sights on citations. The problem with Wikipedia footnotes is that there are almost too many for the platform's volunteer editors to verify. With the website growing by more than 17,000 articles every month, countless citations are incomplete, missing or just plain inaccurate.
Meta developed an AI model that can automatically scan citations at scale to verify their accuracy. It can also suggest alternative citations when it finds a poorly sourced passage. When Wikipedia's human editors evaluate citations, they rely on common sense and experience. When an AI does the same work, it uses a Natural Language Understanding (NLU) transformation model that attempts to understand the various relationships of words and phrases within a sentence. Meta’s Sphere database, consisting of more than 134 million web pages, acts as the system's knowledge index. As it goes about its job of checking the citations in an article, the model is designed to find a single source to verify every claim.
To illustrate the capabilities of the AI, Meta shared an example of an incomplete citation the model found on the Wikipedia page for the Blackfoot Confederacy. Under the Notable Blackfoot people section, the article mentions Joe Hipp, the first Native American to compete for the WBA World Heavyweight title. The linked website doesn’t mention Hipp or boxing. Searching the Sphere database, the model found a more suitable citation in a 2015 article from the Great Falls Tribune. Here’s the passage the model flagged:
In 1989 at the twilight of his career, [Marvin] Camel fought Joe Hipp of the Blackfeet Nation. Hipp, who became the first Native American to challenge for the world heavyweight championship, said the fight was one of the weirdest of his career.
What’s notable about the above passage is that it doesn’t explicitly mention boxing. Meta’s model found a suitable reference thanks to its natural language capabilities. To be clear, Wikipedia isn't currently using the AI model to automatically verify citations, but it's something Meta hopes to build into a platform for the encyclopedia's editors to use one day. "These models are the first components of potential editors that could help verify documents in real time," the company said. "In addition to proposing citations, the system would suggest auto-complete text — informed by relevant documents found on the web — and offer proofreading corrections."
Update 07/12/22 3:25PM ET: Updated the post to clarify Meta and the Wikimedia Foundation didn't partner on the AI model.
