Omnity's search engine uses rare word matching to find unexpected results

When it comes to search, there's Google and there's everyone else -- the company is basically synonymous with searching the internet. But Omnity, a relatively new company from San Francisco, thinks its own search that's based on "semantic mapping" offers something that Google can't do. Omnity's trick is that it looks for the connections between documents on the internet based on rare words -- the theory that research that has several of the same rare words will likely be about related topics, even if that research doesn't directly link to or cite each other.

Thus far, Omnity has operated primarily by selling enterprise plans to companies and educational institutions. Omnity can search not only all of the public datasets it scans (like patents, scientific, engineering and medical documents, clinical trials, case law, SEC filings and so forth) but also a company's internal documents -- for some companies, Omnity indexes 150 petabytes of data.

That may be useful to massive institutions, but plenty of ordinary people could benefit from Omnity's research features -- so today, the company has announced that anyone can search the public databases it indexes for free. Omnity groups the free datasets into four groups: biomedicine, engineering, finance and law, and each set pulls from a wide variety of publicly available sources. Previously, the company offered limited demo searches for free, but now anyone can look up whatever they want.

Once you've signed up for a free Omnity account, you can initiate a search by typing in the Google-like search bar. You'll be served up results grouped by primary sources (those directly related to your query) and secondary sources (documents that share key vocabulary with the primary documents). From there, Omnity offers lots of different ways of visualizing the connections between various documents so you can see what's most potentially related before diving down the research long tail.

There are a lot of ways to extend your search from there, including clicking a word cloud to see specific documents containing those words or seeing a map which shows where the research originated. But one of Omnity's most interesting features is that you can upload documents of your own for it to analyze. Once the document is uploaded, it'll automatically look for those "rare words" and find other documents in its databases that match up with the one you added yourself. It's worth noting that those documents you upload stay private to you -- they aren't added to Omnity's overall database.

This adds up to a search tool that's decidedly not for your average, day-to-day basic informational queries. But, if you work in a field and spend lots of time going down the rabbit hole of the internet, it's entirely possible Omnity can reveal documents that you might have otherwise missed using a traditional keyword-based search engine like Google.

Students and researchers alike may find the tool useful -- and now that it's fully open and free, there's no reason not to give it a shot. And Omnity expects this free version to serve as a good proof-of-concept for its work with larger enterprise companies. If a company or university gets hooked on the free version, it'll probably be a lot easier for Omnity to show them the benefits of its paid service.