Latest in Tomorrow

Image credit: MIT

MIT algorithm finds subtle connections between art pieces

Researchers developed the system to compare works at the MET and Amsterdam’s Rijksmuseum.
Ann Smajstrla
July 29, 2020
300 Shares
Share
Tweet
Share

Sponsored Links

The researchers were inspired by an unlikely, yet similar pairing: Francisco de Zurbarán’s, The Martyrdom of Saint Serapion (left) and Jan Asselijn’s The Threatened Swan (right).
MIT

A new system developed by MIT researchers called “MosAIc” is finding hard-to-spot similarities between art pieces at the Metropolitan Museum of Art and Amsterdam’s Rijksmuseum. MosAIc scans an image, then uses deep networks to find similarities in pieces that span cultures, artists and media that may not have otherwise been noticed, according to a blog post from MIT’s Computer Science & Artificial Intelligence Lab.

To use MosAIc, the user inputs an image and MosAIc’s algorithm finds similar art pieces. In one example, MosAIc linked Francisco de Zurbarán’s, The Martyrdom of Saint Serapion and Jan Asselijn’s The Threatened Swan. “These two artists did not have a correspondence or meet each other during their lives, yet their paintings hinted at a rich, latent structure that underlies both of their works,” said MIT CSAIL PhD student Mark Hamilton, lead author on a paper about MosAIc.

“Anthropoides paradisea” and “Seth Slaying a Serpent, Temple of Amun at Hibis” demonstrate a shared artistic form in color and theme.
MIT

A particularly difficult aspect of developing MosAIc was creating an algorithm that could find not only similarities in color or style, but also in meaning and theme, Hamilton said. Researchers examined a deep network of “activations,” or features, for each image in the open access collections of both museums. The distance between the “activations” of the deep network was how researchers judged similarity.

Researchers also used a new image search data structure called a “KNN Tree,” which groups images together in a tree-like structure. To find one image’s closest match, the algorithm starts at the “trunk” of the grouping, then follows the most promising “branch” until it’s found the closest image. The data structure improves on itself by allowing the tree to “prune” itself based on characteristics of the image.

Hamilton said he hopes the work started on MosAIc can be expanded upon to other fields, like humanities, social sciences and medicine. “These fields are rich with information that has never been processed with these techniques and can be a source for great inspiration for both computer scientists and domain experts,” he said.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission.
Comment
Comments
Share
300 Shares
Share
Tweet
Share

Popular on Engadget

Engadget's 2020 Back-to-School Guide

Engadget's 2020 Back-to-School Guide

View
'Xbox Series S' console revealed by controller packaging

'Xbox Series S' console revealed by controller packaging

View
Space Force official logo and motto unveiled

Space Force official logo and motto unveiled

View
Watch AI-controlled virtual fighters take on an Air Force pilot on August 18th

Watch AI-controlled virtual fighters take on an Air Force pilot on August 18th

View
Hyundai is turning Ioniq into its own EV sub-brand

Hyundai is turning Ioniq into its own EV sub-brand

View

From around the web

Page 1Page 1ear iconeye iconFill 23text filevr