Latest in Tomorrow

Image credit: MIT

MIT algorithm finds subtle connections between art pieces

Researchers developed the system to compare works at the MET and Amsterdam’s Rijksmuseum.
Ann Smajstrla
July 29, 2020
307 Shares
Share
Tweet
Share

Sponsored Links

The researchers were inspired by an unlikely, yet similar pairing: Francisco de Zurbarán’s, The Martyrdom of Saint Serapion (left) and Jan Asselijn’s The Threatened Swan (right).
MIT

A new system developed by MIT researchers called “MosAIc” is finding hard-to-spot similarities between art pieces at the Metropolitan Museum of Art and Amsterdam’s Rijksmuseum. MosAIc scans an image, then uses deep networks to find similarities in pieces that span cultures, artists and media that may not have otherwise been noticed, according to a blog post from MIT’s Computer Science & Artificial Intelligence Lab.

To use MosAIc, the user inputs an image and MosAIc’s algorithm finds similar art pieces. In one example, MosAIc linked Francisco de Zurbarán’s, The Martyrdom of Saint Serapion and Jan Asselijn’s The Threatened Swan. “These two artists did not have a correspondence or meet each other during their lives, yet their paintings hinted at a rich, latent structure that underlies both of their works,” said MIT CSAIL PhD student Mark Hamilton, lead author on a paper about MosAIc.

“Anthropoides paradisea” and “Seth Slaying a Serpent, Temple of Amun at Hibis” demonstrate a shared artistic form in color and theme.
MIT

A particularly difficult aspect of developing MosAIc was creating an algorithm that could find not only similarities in color or style, but also in meaning and theme, Hamilton said. Researchers examined a deep network of “activations,” or features, for each image in the open access collections of both museums. The distance between the “activations” of the deep network was how researchers judged similarity.

Researchers also used a new image search data structure called a “KNN Tree,” which groups images together in a tree-like structure. To find one image’s closest match, the algorithm starts at the “trunk” of the grouping, then follows the most promising “branch” until it’s found the closest image. The data structure improves on itself by allowing the tree to “prune” itself based on characteristics of the image.

Hamilton said he hopes the work started on MosAIc can be expanded upon to other fields, like humanities, social sciences and medicine. “These fields are rich with information that has never been processed with these techniques and can be a source for great inspiration for both computer scientists and domain experts,” he said.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
307 Shares
Share
Tweet
Share

Popular on Engadget

Best Buy slashes 50 percent off TCL's 75-inch 8-series Roku TV

Best Buy slashes 50 percent off TCL's 75-inch 8-series Roku TV

View
Netflix will only stream in 4K to Macs that have a T2 security chip

Netflix will only stream in 4K to Macs that have a T2 security chip

View
Nintendo agrees to $2 million settlement in Switch hacking lawsuit

Nintendo agrees to $2 million settlement in Switch hacking lawsuit

View
The next 'Super Smash Bros. Ultimate' fighter is Steve from 'Minecraft'

The next 'Super Smash Bros. Ultimate' fighter is Steve from 'Minecraft'

View
Peter Parker has been recast in ‘Marvel’s Spider-Man: Remastered’

Peter Parker has been recast in ‘Marvel’s Spider-Man: Remastered’

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr