One of the annoying things that old school Artificial Intelligence researchers have to deal with is the fact that simple brute force is such a daunting foe. Back in the dawn era of the field, attempts to replicate human thought processes used deductive reasoning, symbolic representation, and incremental learning to solve problems.

As an example, look at what the AI consensus might have been 30 years ago for championship chess programs, and compare it to the massive database searches used by Deep Blue to pummel human opponents. I think you’ll find that things haven’t worked out quite the way they were expected to.

The feeling of the hoi polloi, of course, is that Artificial Intelligence is dead, and that’s probably the best thing that could happen to what is still a pretty exciting field. But it’s quite a rarity for the public to get a hands-on look at exciting developments in AI.

Unfortunately, the public beta of the Swingly search engine is not going to be the exception to this rule.

Swingly

Swingly is a search engine that purports to answer questions in plain English by searching the web. They give some examples which work quite well, such as:

  • How much money did Avatar make?
  • Who won the World Series in 2004?
  • Who killed Inigo’s father?

Natural language queries are to AI what register optimization is to compilers - a fundamental problem that has been studied to death. As a result, I though maybe Swingly might have some interesting results to present here.

Swingly vs. Google

Of course, it is only fair to compare its results to those of Google, the brute force ghost at the banquet. I threw out a random selection of questions and examined the results, judging them only on whether or not they answered the question. My sample list included:

  • Who is the editor of Dr. Dobb’s Journal?
  • Is same sex marriage legal in Texas?
  • How does Swingly work?
  • What books did Mark Nelson write?
  • How do I tie a half hitch?
  • What song has the lyrics “My loneliness is killing me?”

I rated the test results as Good, Okay, or Bad. Google got 5 Goods and 1 Okay. Swingly got 2 Okays and 4 Bad - bad meaning that it totally missed the point.

Just as an example, the answer to the last question is “Baby One More Time” by Britney Spears. Google’s first result was a video for that song - a clear win. Swingly’s first result was a Wikipedia entry for a song called “Killing Me” by the Japanese rock band L’Arc-en-Ciel - fail.

So today at least, chalk one up for finely tuned brute force queries, and mark it a loss for natural language parsing.