[Corpora-List] Summary - evaluation of UD and other dependencies

Amir Zeldes Amir.Zeldes at georgetown.edu
Fri Jul 20 03:24:30 CEST 2018

(apologies for cross-postings)

Hi everyone,

Last week I asked for papers about evaluating dependency annotation, and specifically reporting scores on UD inter-annotator agreement. I received many interesting references, some of which are direct evaluations of UD annotation and others which discuss specific issues and metrics for evaluating dependency annotations – here is the complete list:

– Bamman, David, Francesco Mambrini & Gregory Crane (2009), An ownership model of annotation: The Ancient Greek dependency treebank. In: Proceedings of the Eighth International Workshop on Treebanks and Linguistic Theories (TLT 8). Groningen, 5–15. Available at: http://www.perseus.tufts.edu/~ababeu/tlt8.pdf.

– Berzak, Yevgeni, Yan Huang, Andrei Barbu, Anna Korhonen & Boris Katz (2016), Anchoring and Agreement in Syntactic Annotations. In: Proceedings of EMNLP 2016. Austin, TX, 2215–2224.

– Berzak, Yevgeni, Jessica Kenney, Carolyn Spadine, Jing Xian Wang, Lucia Lam, Keiko Sophie Mori, Sebastian Garza & Boris Katz (2016), Universal Dependencies for Learner English. In: Proceedings of ACL 2016. Berlin, Germany, 737–746. Available at: http://www.aclweb.org/anthology/P16-1070.

– Liu, Yijia, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider & Noah A. Smith (2018), Parsing Tweets into Universal Dependencies. In: Proceedings of NAACL 2018. New Orleans, LA, 965–975. Available at: http://aclweb.org/anthology/N18-1088.

– Nguyen, Kiem-Hieu (2018), BKTreebank: Building a Vietnamese Dependency Treebank. In: Proceedings of LREC 2018. Miyazaki, Japan, 2164–2168. Available at: http://www.lrec-conf.org/proceedings/lrec2018/pdf/69.pdf.

– Seddah, Djamé, Eric De La Clergerie, Benoît Sagot, Héctor Martínez Alonso & Marie Candito (2018), Cheating a Parser to Death: Data-driven Cross-Treebank Annotation Transfer. In: Proceedings of LREC 2018. Miyazaki, Japan, 4535–4539. Available at: http://www.lrec-conf.org/proceedings/lrec2018/pdf/1101.pdf.

– Seyoum, Binyam Ephrem, Yusuke Miyao & Baye Yimam Mekonnen (2018), Universal Dependencies for Amharic. In: Proceedings of LREC 2018. Miyazaki, Japan, 2216–2222. Available at: http://www.lrec-conf.org/proceedings/lrec2018/pdf/565.pdf.

– Skjærholt, Arne (2014), A Chance-corrected Measure of Inter-annotator Agreement for Syntax. In: Proceedings of ACL 2014. Baltimore, MD, 934–944. Available at: http://www.aclweb.org/anthology/P14-1088.

Thanks to everyone who contributed!



Dr. Amir Zeldes

Asst. Prof. of Computational Linguistics

Department of Linguistics

Georgetown University

1437 37th St. NW

Washington, DC 20057

<http://corpling.uis.georgetown.edu/amir> http://corpling.uis.georgetown.edu/amir

-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 10925 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20180719/129c2a6b/attachment.txt>

More information about the Corpora mailing list