[Corpora-List] INEX 2011: Call for Participation

Jaap Kamps kamps at science.uva.nl
Thu Apr 7 14:40:34 CEST 2011


INEX 2011 Call for Participation

REGISTRATION NOW OPEN AT http://inex.mmci.uni-saarland.de/

OVERVIEW

Traditional IR focused on pure text retrieval over "bags of words" but the use of structure---such as document structure, semantic metadata, entities, or genre/topical structure---is of increasing importance on the Web and in professional search. INEX has been pioneering structured retrieval since 2002, by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. INEX 2011 will proudly continue pushing research boundaries, with a wide range of tasks including Social Search, Faceted Search, and Snippet Retrieval.

TASKS AND TRACKS

- Books and Social Search Track

Investigating techniques to support users in searching and navigating books, metadata and complementary social media. The _Social Search for Best Books Task_ studies the relative value of authoritative metadata and user-generated content using a collection based on data from Amazon and LibraryThing. The _Prove It Task_ asks for pages confirming or refuting a factual statement, using a corpus of the full texts of 50k digitized books.

- Data Centric Track

Investigating retrieval over a strongly structured collection of documents based on IMDb. The _Ad Hoc Search Task_ has informational requests to be answered by the entities in IMDb (movies, actors, directors, etc.). The _Faceted Search Task_ asks for a restricted list of facets and facet-values that will optimally guide the searcher toward relevant information.

- Interactive Track

Investigating the behavior of users when interacting with structured documents, using a collection based on Amazon book records and additional social data from LibraryThing, as well as developing access tools which are effective in user-based environments.

- Question Answering Track

Investigating real-world focused information needs formulated as natural language questions using a Wikipedia corpus with additional annotation (document structure, sentences, and named-entities), evaluated by both a readable summary of question context and a list of answers.

- Relevance Feedback Track

Investigate the utility of incremental passage level relevance feedback by simulating a searcher's interaction. An unconventional evaluation track where submissions are executable computer programs rather than search results.

- Snippet Retrieval Track

Investigate how to generate informative snippets for search results.

Such snippets should provide sufficient information to allow the user to determine the relevance of each document, without needing to view the document itself.

- Web Service Discovery

Investigate techniques for discovery of Web services based on searching service descriptions provided in WSDL. The answer may be a single service that matches the requirements of a request, or a group of services that combined together match the requirements.

A detailed description of each of the tracks can be found at http://inex.mmci.uni-saarland.de/

IMPORTANT DATES

Schedules vary per track, with collections and topics coming available in the spring (now!), submissions due in the summer, and results coming available in the fall. INEX 2011 concludes with a workshop in December:

12-14 Dec 2011: INEX Workshop near Saarbruecken

RELEVANCE ASSESSMENTS

Some of the tracks rely on participating groups to build the test collection though topic creation and relevance assessments. The Books and Social Search Track uses crowdsourcing.

WORKSHOP AND PROCEEDINGS

Participants will be able to present their approaches and the final results at the INEX workshop to be held in December near Saarbruecken. All participants will be able to publish their work in the workshop pre-proceedings. Fully peer-reviewed and revised papers will be published the INEX post-workshop final proceedings. Since 2004 the final proceedings have been published in the Springer's Lecture Notes in Computer Science (LNCS) series, and we seek to continue that tradition.



More information about the Corpora mailing list