Despite its uncertain future, Flickr continues to add features, doing their best to remain a relevant part of today’s photo sharing landscape. Their latest update adds something called “similarity search“, a feature that helps you find visually similar photos without guessing at proper search terms and tags.
Just like Google’s “visually similar” image results when you search for a specific photo, Yahoo uses a deep neural network to find photos that contain the same subject, colors, and style of the photo you’re using as a reference.
Flickr demonstrates the new feature in this GIF for… cactus lovers?:
The technology behind “similarity search” is, of course, some pretty advanced machine learning. First, they train a neural network to “map the raw pixels of a photo into a set of relevant tags” such as “hot air balloon.” That’s usually where auto-tagging algorithms stop, tossing aside information like where the balloon is in space, its color, and more.
Yahoo’s similarity search goes the extra step, keeping this information and plotting it as something called a “feature vector” using some advanced math we won’t get into here. It can then compare one images’ feature vector to other photos, and images with similarly plotted vectors will tend to feature similar content and structure.
This illustration shows how this works. Photos with feature vectors in the “same neighborhood” will be much more similar to your image than those further away:
“This measure of similarity is not perfect and cannot capture all possible notions of similarity—it will be constrained by the particular task the network was trained to perform, i.e., scene recognition,” explains Yahoo on their Flickr code blog. “However, it is effective for our purposes, and, importantly, it contains information beyond merely the semantic content of the image, such as appearance, composition, and texture.”
To dive deeper into this technology, click here. Alternatively, just head over to Flickr, search for “puppies,” and give the feature a shot. That’s what we did.
