Researchers at Cornell Tech and the University of California San Diego (UCSD) have been building a searchable, visual encyclopedia – and with help from Cornell’s Lab of Ornithology, their first experiment in identifying bird species proves that they’re making great strides in the field of computer vision.
Cornell Tech professor Serge Belongie and UCSD master’s student Grant Van Horn – along with collaborators at Caltech, Berkeley, BYU and NYU – have been working for five years on a new technology called Visipedia – a visual encyclopedia that lets you search images rather than text.
Like Wikipedia, Visipedia’s success lies in engaging users. For their first project in identifying bird species, Belongie and Van Horn teamed up with Cornell’s Lab of Ornithology, which reached out to its network of bird enthusiasts around the world.
The experts classified thousands of photos for Visipedia, detailing birds’ color, beak shape, wing span, size and more. Thanks to their work, Visipedia can analyze images of birds and detect those details so that any user can now upload a photo and have it matched it to the proper bird species. Visipedia’s bird search engine has reached 85% accuracy, about the same rate of success as one of the human experts.
Visipedia is aiming to expand its detecting abilities to cover everything from plants to insects, furniture to food. It could even one day be used by cancer pathologists to better classify groups of tumors.
Rather than pigeon-holing the program to one field or app, Belongie plans on making Visipeda a non-profit, universally available Application Programming Interface (API). They want the tool to be open to the public so web and app developers can use the data to build new programs.
In the short term, Belongie and Van Horn hope developers will use Visipedia’s successful bird search to build mobile apps that could be used while bird watching, encouraging more people to explore nature.
