We Will Survive the Linguistic Shift

Keywords or questions: is the difference really that remarkable?

**Keep this in mind: Anyone familiar with the game of Jeopardy knows that an integral part of getting the answer right is the ability to quickly frame it as a question….

A couple of sources I’ve recently read have tried to make a valid argument against any seamless implementation of a semantic or natural language web. An arugument? People are trained to keyword search. I hesitate to consider really the root of their beef–are they saying we won’t make the shift from phrases to questions? That’s the bare bones difference between a keyword (Google-type) search query and a natural language (Hakia, Powerset) search query. Query being the actual thing you type into the search field, the “prompters” intended to stir up the information for which you’re looking.

Okay, I know that journalists and pundits are in the biz of creating talk, the more controversy the better, but an arugment like this–that we’re already patterned to work in a keyword inspired environment, just seems lame to me. It also relegates our species back to that of caveman. Remember the Jeopardy rules…

Steven Pinker, in his brand new book, The Stuff of Thought, argues that even as children we are driven to instinctively emulate simple to complex language patterns even well before we have formal instruction. Linguistic pattern as a right of belonging, as part of our human-ness, is elementary, it is fundamental in our collective DNA.

So, NO, we’ll catch on quickly if and when a natural language engine rises to the top. AND if our businesses rest on it, you better believe the learning curve will be short-lived. I mean look at AdWords….when it first launched only the very maverick web marketers and SEOs took to it–were able to instantly rope and tie it. But now, just a few years later (and, yes, many millions of businesses make their money from it) it’s an absolute essential component in any savvy business strategy. If people can learn: how to build a marketing strategy with AdWords, how to navigate and sell everything on eBay, and shop for anything online, then casting a search query in natural language, framed as a question, seems quite….um, natural.

True Knowledge: Natural Language Search Engine

Making a natural language search engine for the masses–keep it on the DL.

True Knowledge is an internet search company that has produced as a key product, a new natural language search engine, although they don’t say that. Also missing is “semantic.” I only mention this because it seems to me thattrue knowledge new search True Knowledge and Freebase before it, have put high-level techno terms like these at arm’s length. Maybe it makes them more palatable, maybe it is less intimidating, maybe everyday folks will be a lot more inclined to add such a search engine to their daily search habits if it looks and smells not too unlike another.

Here’s what True Knowledge proposes to do: deliver “direct” and concise answers to natural language queries. For example, TK’s example “is jennifer lopez single?” received a direct “NO” — parsed from the data existing in the SE’s database, plus highlighted results for the natural language query, PLUS ordinary keyword search results in case TK is unable to find the information–“at least you’ll have the results you’d have on any other given day.”

Right now the engine has just launched a private beta version, so it’s anyone’s guess how long it will be before she’s open for business. But if you’d like a preview of the API and the technology behind this newest search engine effort, then take a look at the site. There’s a video introduction that gives about a 7 minute run-down of the features and benefits of True Knowledge.

Who is Cyc?

Some search engines vie for fairy dust, others just ante up the goods.

tolkien’s middle earth map

Semantic search engines vie to harness the same fairy dust as did Google–once upon a time. But charismatic, enigmatic, and dismissive geeky upstarts that make billions upon billions of dollars of course earn as many foes as they do dough. My point is that Google’s limelight is still enviable and new search engines like the mysterious Powerset and Hakia are in line.

But, outside the West Coast Search celebrity there are other semantic web forces to be reckoned with that have actually been chipping away at the natural language lexicon for years.

Cyc, “the world’s largest and most complete general knowledge base and commonsense reasoning engine,” is already a working product for corporate and industrial users. I’ve just happened on it and am still trying to digest the literature, but it reminds me of a mini IBM WebFountain, without thecycorp logo supercomputing gusto, but, a powerful engine, nevertheless, that has already “learned” <…..THIS much……>

“What does Cyc know?” According to founder and developer, Doug Lenat, Cyc is able to negotiate this question: “Which American city would be most vulnerable to an anthrax attack during summer?”

Where can I get my very own Cyc?

Like most open source systems designed for industrial applications, Cyc is not nearly as consumer-friendly as a Powerset or Hakia. There is no intuitive, slick little interface. Instead the Cyc main page smacks of the magic language of software developers and Unix users and computer wizards. This is middleware land, Middle Earth, Hobbit-ville. Which makes it all the more enigmatic–I want one.