[Corpora-List] [CfP] CL-SciSumm Shared Task 2018 @SIGIR’2018: The Scientific Summarization Shared Task

Muthu muthu.chandra at comp.nus.edu.sg
Mon Mar 19 04:13:04 CET 2018


*=== Call for Participation ===The 4th Computational Linguistics Scientific Summarization Shared Task, CL-SciSumm-18 @ SIGIR 2018: http://wing.comp.nus.edu.sg/~cl-scisumm2018/ <http://wing.comp.nus.edu.sg/~cl-scisumm2018/>You are invited to participate in the CL-SciSumm 2018 Shared Task, as part of the 3rd Joint Workshop of Bibliometric-enhanced IR and NLP for Digital Libraries (BIRNDL) at SIGIR 2018 on Thursday, July 12, 2018.The 4th CL-SciSumm Shared Task on scientific paper summarization follows up on and extends the corpus sizes of the successful Shared Tasks conducted as a part of the BIRNDL workshops in 2017, 2016 and the Pilot Task conducted as a part of the BiomedSumm Track at the Text Analysis Conference 2014. In the CL-SciSumm 2017 Shared Task, fifteen teams from six countries signed up, and ten teams ultimately submitted and presented their results. The task is run on CL-Scisumm corpus, the largest annotated corpus for scientific summarization, comprising over 500 computational linguistics (CL) research papers, interlinked through a citation network. The corpus is available for free download and use at https://github.com/WING-NUS/scisumm-corpus <https://github.com/WING-NUS/scisumm-corpus>. The repository also archives all results from all prior runs of the Shared Task.The Shared Task comprises three sub-tasks in automatic research paper summarization on a new corpus of research papers. This task is expected to be of interest to a broad community including those working in CL and NLP, especially in the sub-disciplines of text summarization, natural language generation, text reuse, discourse structure in scholarly discourse, paraphrase, textual entailment and text simplification.=== The Task ===Given: A topic consisting of a Reference Paper (RP) and ten or moreCiting Papers (CPs) that all contain citations to the RP. In each CP,the text spans (i.e., citances) have been identified that pertain to aparticular citation to the RP.Task 1a: For each citance, identify the spans of text (cited textspans) in the RP that most accurately reflect the citance. These areof the granularity of a sentence fragment, a full sentence, or severalconsecutive sentences (no more than 5).Task 1b: For each cited text span, identify what facet of the paper itbelongs to, from a predefined set of facets.Evaluation: Task 1 will be scored by overlap of text spans in thesystem output vs the gold standard created by human annotatorsTask 2: (optional bonus task): Finally, generate a structured summary of the RP from the cited text spans of the RP. The length of the summary should not exceed 250 words.Evaluation: Task 2 will be scored using the ROUGE evaluation metric to compare automatic summaries against paper abstracts, human written summaries and community summaries constructed using the output of Task 1a.=== Important Dates ===March 19: Training set postedApril 8: Deadline for expression of interest and short systemdescriptions dueMay 1: Test set postedMay 20: System runs from the test set dueMay 27: System reports (paper) dueJune 25: Camera ready contributions dueJuly 12, 2018: Participants present at the BIRNDL 2018 workshop in Ann Arbor, MI, USA=== The Corpus ===The CL-SciSumm corpus is created by randomly sampling documents fromthe ACL Anthology corpus and selecting their citing papers. Citing paper may Include papers from outside the Anthology. ForCL-SciSumm 2018, we have selected three portions of this sourcecollection to be annotated and serve as training, development and testcollections. The training set of articles is available for downloadat GitHub <https://github.com/WING-NUS/scisumm-corpus <https://github.com/WING-NUS/scisumm-corpus>> and can be usedby participants to pilot their systems. Watch for updates to theGitHub repository, as we will be updating the repository with announcements and new files. The system outputs from the test set should be submitted to the task organizers, for the collation of the final results to be presented at the workshop.=== Registration ===Organizations wishing to participate in the CL Shared Task track atBIRNDL 2018 are invited to register on EasyChair:<https://easychair.org/conferences/?conf=birndl2018 <https://easychair.org/conferences/?conf=birndl2018>> by April 8th with a tentative abstract. Please prefix “CLSciSumm Shared Task: ” to the title of your submission. Participants are advised to register as soon as possible in order to receive timely access to evaluation resources, including training development and testing data. Registration for thetask does not commit you to participation - but is helpful to know for planning. All participants who submit system runs are welcome to present their systems as posters/selected presentations at the BIRNDL 2018 Workshop at Ann Arbor, MI, USA.Dissemination of CL-SciSumm work and results other than in theworkshop proceedings is welcomed, but the conditions of participationspecifically preclude any advertising claims based on these results.Any questions about conference participation may be sent to theorganizers mentioned below.=== Organising Committee ===- Kokil Jaidka, <http://kokiljaidka.wordpress.com/ <http://kokiljaidka.wordpress.com/>>- Muthu Kumar Chandrasekaran, <http://wing.comp.nus.edu.sg/~cmkumar/ <http://wing.comp.nus.edu.sg/~cmkumar/>>- Michihiro Yasunaga <https://www.linkedin.com/in/michihiro-yasunaga-616762136 <https://www.linkedin.com/in/michihiro-yasunaga-616762136>>- Dragomir Radev <https://cpsc.yale.edu/people/dragomir-radev <https://cpsc.yale.edu/people/dragomir-radev>>- Min-Yen Kan <https://www.comp.nus.edu.sg/~kanmy/ <https://www.comp.nus.edu.sg/~kanmy/>>*

Thanks! Muthu

Muthu Kumar Chandrasekaran Ph.D. Candidate | Web Information Retrieval / Natural Language Processing Group (WING) School of Computing | National University of Singapore (NUS) wing.comp.nus.edu.sg/~cmkumar -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 40152 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20180318/3f46b28e/attachment.txt>



More information about the Corpora mailing list