[Corpora-List] BioNLP Shared Task 2013 - Open Call for Supporting Analyses

Pontus Stenetorp pontus at stenetorp.se
Wed Dec 12 08:11:18 CET 2012

Summary: For the BioNLP 2013 Shared Task we are issuing an open call for automated analyses from existing Natural Language Processing systems to ease the burden on shared task participants and increase visibility of research efforts that could benefit state-of-the-art Information Extraction systems.

We are issuing an open call to the Natural Language Processing to provide supporting analyses of the texts that have been manually annotated for the BioNLP 2013 Shared Task (http://2013.bionlp-st.org/) on information extraction. As for the previous tasks in 2009 and 2011, we aim to provide supporting analyses to the participants in order to ease the workload required to participate in the shared task. Syntactic analyses have been and continue to be a key component of the state-of-the-art methods for the shared task [1,2] and semantic analyses have been used to improve upon the performance existing methods [3]. If you and/or your group has designed an NLP system which you think could assist the extraction targets for the 2013 task, we ask you to consider providing these supporting analyses for the benefit of the shared task participants and the community at large and increase the visibility of your tool among your fellow researchers. Researchers who provide supporting analyses will be acknowledged with mention and references on the shared task homepage and in related shared task publications.

The procedure to participate is as follows:

1.) Send a statement of interest to the shared task organisers. We ask those interested to notify us by December 19th 2012 (Wednesday). Please send your messages to <bionlp-st-adm at googlegroups.com>.

2.) The raw text for the respective tasks will be made publicly available one week prior to the release of the training and development sets in early January 2013. Those that want to submit supporting analyses are expected to run their tools within this week and send the resulting analyses to the shared task organisers for inclusion in the public release of the annotations. Sample data for all the tasks is already available from the shared task homepage (http://2013.bionlp-st.org/tasks).

3.) The same proceduce take place for the test set releases, each supporting analyses provider will once again have a full week to run their tools and submit their analyses prior to the test set releases.

Tentative schedule:

* 8th of January 2013: Release of the training/development set texts * 15th of January 2013: Public release of the training/development data sets * 22nd of February 2013: Release of the test set texts (Tentative) * 1st of March 2013: Public Release of the test data sets (Tentative)

In order to allow for the shared task participants to replicate and apply the tools used to produced the automated analyses for purposes other than the shared task, the following are emphasised (but not strictly required) for reproducibility purposes:

* Availability of the tool and tool source-code for research purposes

* Support for well-established formats (if the tool uses its own format, conversions to for example PTB for constituency output is encouraged to be included along with the native format)

* Availability of scientific and technical documentation

When sending your statement of interest to the organisers, please include the following information:

1.) Contact details for the members of your group that seek to participate 2.) Describe, briefly, how you think the tool could be used by the shared task participants and ease their workload 3.) Homepage of the tool (if any) 4.) Relevant publications for the tool (if any) to be referenced in the shared task publications and homepage 5.) Status of availability (source code, web service, binary, non-public and so forth)

We warmly welcome your contributions towards advancing the state-of-the-art in information extraction and we look forward to your automated analyses.

On behalf of the BioNLP 2013 Shared Task organisers,

Pontus Stenetorp

[1]: "Fast and Robust Joint Models for Biomedical Event Extraction", Riedel and McCallum (2011) [2]: "Generalizing Biomedical Event Extraction", Björne and Salakoski (2011) [3]: "Bridging the Gap Between Scope-based and Event-based Negation/Speculation Annotations: A Bridge Not Too Far", Stenetorp et al. (2012)

More information about the Corpora mailing list