Communications - September 2008

based on the words in the questions that are sometimes expanded with synonyms. For example, ASU QA takes dozens of patterns for each question type from the questions and answers seen previously during the training process. It transforms the question, say, “Who is the CEO of IBM?” into the Google query “became the CEO of IBM” because it has previously seen the answer “Washington” to the question “What is the capital of the U.S.?” in the sentence “Washington became the capital of the U.S. on June 11, 1800.”

Since many of the factual answers are named entities (such as people, organizations, countries, and cities), QA systems typically employ third-party named-entity-identification techniques to extract candidate answers (such as Minipar). ¹All named entities in the proximity of the question words and that match the desired semantic category are identified as candidate answers. Meanwhile, ASU QA, employs a pattern-matching mechanism to perform answer extraction. A sentence like “Samuel Palmisano recently became the CEO of IBM” matches the pattern “<A> became <Q>,” where <A> = “ Samuel Palmisano recently” is the candidate answer, and <Q> = “the CEO of IBM” is the question’s focus. ASU QA also treats all subphrases from each candidate answer as candidates themselves. In the example, the subphrases are “Samuel Palmisano recently,” “Samuel Palmisano,” “Palmisano recently,” “Samuel,” “Palmisano,” and “recently.”

In order to identify the most probable (supported) answer, ASU QA has gone several steps beyond frequency counts explored earlier by Dumais et al. ¹ and other groups involved in TREC competitions that involved a probabilistic triangulation mechanism. Triangulation is a term widely used in the intelligence and journalism fields for confirming or disconfirming facts by checking multiple sources. Roussinov’s and Robles’s algorithm is demonstrated through the following intuitive example: Imagine that we have two candidate answers for the question “What was the purpose of the Manhattan Project?”: ( 1) “To develop a nuclear bomb” or ( 2) “To create an atomic weapon.” They support (triangulate) with each other since they are semantically similar. In the example involving the CEO of IBM, “Samuel

Palmisano” and “Sam Palmisano” win because they reinforce each other.

Although QA technology is maturing quickly and seems promising for a number of practical applications (such as commonsense reasoning and database federation), few QA systems go beyond information seeking. Although the Ford Motor Company and Nike, Inc. began using Ask.com as their site search engine in 2005, they’ve never reported if QA features are indeed practical and useful. In 2005, Roussinov and Robles demonstrated empirically that ASU QA helps locate potentially malevolent online content, potentially helping law-en-forcement and public oversight groups combat the proliferation of materials that threaten cybersecurity or promote terrorism.

feature comparison When comparing features and performing our informal evaluation, we chose only the QA systems (see Table 1) mentioned in popular IT magazines or academic publications and that were (and still are) available online during the first run of our study in spring 2005. We did not include Google or MSN since their QA capabilities were (and still are) quite limited. Google occasionally produces precise answers with respect to geogra-phy-related questions (such as “What is the population of Cambodia?”) but

does not attempt to target more general or dynamic topics (such as “Who is the CEO of Motorola?”) or more grammatically or semantically challenging questions (“How long can a British Prime Minister serve in office?”). MSN uses only Encyclopedia Encarta as a source of precise answers and is similarly limited in terms of complexity and coverage.

Although AskJeeves enjoyed immense popularity and investor interest at the time it was acquired, its QA capabilities are limited in practice. Its answers to natural-language questions could be provided only from manually created databases, and the topics of inquiry were limited to simple “ encyclopedic” requests (such as “What is the population of Uganda?”). When the question does not match any of the anticipated questions, Ask.com would reroute the question as a simple keyword query to its underlying keyword search engine—Teoma, which was acquired by Ask.com in 2001 when it was a failing dot-com based on technology originally created by IBM and further developed at Rutgers University. In 2005, Ask.com introduced certain answer-matching capabilities over the entire Web but is still short of specifying the precise answer while displaying a set of ordered snippets (up to 200) with the words from the highlighted question, similar to Google’s approach.