[Corpora-List] CLEF QA Track - Entrance Exams - 1st Call for Participation

Alvaro Rodrigo Yuste alvarory at lsi.uned.es
Wed Feb 18 16:18:06 CET 2015


**** Apologies for cross posting *****

---------------------------------------------

Entrance Exams 2015 http://nlp.uned.es/entrance-exams/

---------------------------------------------

CLEF Question Answering Track (CLEF QA Track) a CLEF 2015 Lab http://nlp.uned.es/clef-qa/

---------------------------------------------

Summary -------

Japanese University Entrance Exams include questions formulated at various levels of complexity and test a wide range of capabilities. The challenge of "Entrance Exams" aims at evaluating systems under the same conditions humans are evaluated to enter the University. Exams are created by the Japanese National Center for University Admissions Tests. The "Entrance Exams" corpus is provided by NII's Todai Robot Project and NTCIR RITE.

Task Description ----------------

Participant systems are asked to read a given document and answer questions. Questions are given in multiple-choice format, with four options from which a single answer must be selected. Questions are taken from reading tests of real university entrance exams, and may include a variety of question types that require a wide range of inference. Systems have to answer questions by referring to "common sense knowledge" that high school students who aim to enter the university are expected to have. Another important difference with previous QA exercises is that we do not intend to restrict question types. Any types of reading comprehension questions in real entrance exams will be included in the test data.

Language --------

Test documents, questions, and options will be made available in English, French and Spanish. Japanese, Chinese, German and Russian are under consideration. Please contact the organization for expressions of interest in any particular language.

Evaluation ----------

Each test will receive an evaluation score between 0 and 1 using c at 1. This measure, used in previous CLEF QA Tracks, encourages systems to reduce the number of incorrect answers while maintaining the number of correct ones by leaving some questions unanswered.

Important Dates ---------------

Send the agreement form by April 30, 2015 Test set release May 6, 2015 Run submissions May 15, 2015 Individual results to participants May 20, 2015 Submission of Working Notes Papers June 7, 2015 CLEF Workshop in Toulouse, FR September 15-18, 2015

Contact -------

Anselmo Peñas (anselmo at lsi.uned.es) Yusuke Miyao (yusuke at nii.ac.jp)

Organizing Committee --------------------

Anselmo Peñas (UNED, Spain) Yusuke Miyao (National Institute of Informatics, Japan) Eduard Hovy (Carnegie Mellon University, USA) Noriko Kando (National Institute of Informatics, Japan) Teruko Mitamura (Carnegie Mellon University, USA) Alvaro Rodrigo (UNED, Spain)

-- Dr. Alvaro Rodrigo Yuste UNED - E.T.S.I. Informática Dpto. Lenguajes y Sistemas Informáticos Ciudad Universitaria C/ Juan del Rosal nº 16 28040 Madrid - Spain Tef. +34 - 91 - 398 96 93 Fax +34 - 91 - 398 93 82 email: alvarory at lsi.uned.es

-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 4345 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20150218/0a4a4057/attachment.txt>



More information about the Corpora mailing list