WebCLEF is the crosslingual web track at the Cross-Language Evaluation Forum campaign (CLEF: http://www.clef-campaign.org). In the past three years the track focused on evaluation of systems providing multi- and cross-lingual access to web data.
WebCLEF 2008 will repeat the task setup of the 2007 edition. In 2008, we evaluate a multilingual information synthesis task, where, for a given topic, participating systems are asked to extract important snippets from web pages (fetched from the live web and provided by the task organizers). The systems will have to focus on extracting, summarizing, filtering and presenting information relevant to the topic, rather than on large scale web search and retrieval per se. In the 2008 edition of the task, we will focus on refining the assessment procedure and evaluation measures.
WebCLEF 2008 has lots of similarities with (topic-oriented) multi-document summarization and with answering complex questions. An important difference is that at WebCLEF, topics may come with extensive descriptions and with many thousands of documents from which important facts have to be mined. In addition, WebCLEF works with web documents, that may be very noisy and redundant.
For WebCLEF 2008 participants, the test collection, topics, assessments, and source code of the best performing 2007 system are available, together with evaluation scripts, so as to create a shared baseline.
Tentative timeline for WebCLEF 2008:
* February 2008: task guidelines and development data available
* April 2008: topic development (shared effort by participants)
* May 2008: release of test collection and topic
* June 2008: result submission
* July 2008: evaluation (shared effort by participants)
* August 2008: paper submission for CLEF Working Notes
* September 2008: CLEF workshop
WebCLEF 2008 is organized by the University of Amsterdam and backed by the following advisory board:
* Julio Gonzalo, Universidad Nacional de Educacion a Distancia
* Valentin Jijkoun, University of Amsterdam
* Jimmy Lin, University of Maryland
* Constantin Orasan, University of Wolverhampton
* Maarten de Rijke, University of Amsterdam
For more details, tools, resources, and signing up please visit http://ilps.science.uva.nl/WebCLEF/WebCLEF2008/