Reducing friction in search with natural language

Keyword Search is incredibly powerful, but it is often a pretty blunt instrument. People are typically more expressive with natural language, and a system that uses natural language as an interface can reduce the friction to users. This can make PC based interfaces more effective, and will have an even more significant effect with mobile devices (which are moving very quickly to speech based interfaces).

We are developing a technology to extract more meaning from end-user queries than a straight keyword search can. Our first proposed application would be for Amazon's product search.

Our technology can do two major things:

  1. It can look at the search results from a query targeted to ALL categories, and move higher in the results set those entries that most closely match what the user intended to look for, which improves the customer experience by reducing friction in the shopping cycle.
  2. It can look at the search results, and if the confidence level of a sufficient number do not match what we think the user intended, we can provide results of a second query reflecting what we think the user intended.

We believe that option 1 can be demonstrated to show measurably better performance on actual user queries without having to be integrated into the Amazon system. If we are given a set of historical data containing user queries, along with the users clickstream, we plan to demonstrate that our filter raises the position of the items the users actually wanted, relative to a simple keyword search.

We recognize that the Natural Language problem is one that many have tried to solve previously -- mostly with limited success. We believe we have an approach that will provide demonstrably useful, scalable results with small processing loads.

Next level of detail

Amazon uses keyword search. It depends on the user using perfectly matching keywords. If the user uses the wrong keywords, search degrades (e.g. ladies instead of womens or maroon instead of red). If the user adds extra words (as a Siri user saying "Show me a red blouse", then search degrades. Even a perfect keyword search can be derailed by a product description mentioning things unrelated to the product (like a description of a sheet saying the company makes futons and sheets), -- a search on futon may return that sheet.

Our Natural Language capability complements the existing search by determining what the user actually wants and restricting the results to those relevant to that request. It can determine the correct browse node categories to be allowed and examine returned products' specifications to confirm they match (like color or size). If the Amazon search fails to bring up relevant answers, our system can replace the given search with one that will get better answers, either doing a better job of removing extraneous items (by removing less important words and passing that search to Amazon's existing engine) or by converting words to synonyms that are more likely to get the desired hits.

Our NL capability also enables the user to communicate in simple natural English sentences instead of unnatural keyword combinations. It can be extended in the future to involve interactive querying of the user, to help the user refine vague queries into well-targeted ones. It could even accept vague requests such as "Show me potential presents for my wife", and coordinate with a gift recommendation system.

Here is a link to our demo.