[Corpora-List] Word Sense Disambiguation: a Unified Evaluation Framework and Empirical Comparison (Rocco Tripodi)

Rocco Tripodi rocco.trip at gmail.com
Tue Jan 17 10:30:10 CET 2017

Thanks, Alessandro for the great job.

I will evaluate my system (Tripodi R. and Pelillo M., 2017, A game-theoretic approach to word sense disambiguation <https://www.researchgate.net/publication/304469533_A_Game-Theoretic_Approach_to_Word_Sense_Disambiguation>, to appear in Computational Linguistics, 43,1) on your framework soon, since it has been cited in your paper but not evaluated.

Let me just add to your description that this system, even if it is graph and knowledge-based, it is completely different from the algorithms based on PageRank. For example, it disambiguates all words simultaneously and produces consistent labelings that correspond to Nash equilibria. That means, for example, that the word bank in "there is a financial institution near the river bank", will not be allowed to take the "financial institution" sense, since two related words in the text will be always disambiguated with coherent senses (because this choice gives higher payoff).

Hope that your advice will be taken! :)

Best regards, Rocco Tripodi

Postdoc Researcher European Centre for Living Technology Dipartimento di Studi Linguistici e Culturali Comparati Ca' Foscari University researchgate <https://www.researchgate.net/profile/Rocco_Tripodi>

> Message: 4
> Date: Mon, 16 Jan 2017 15:12:29 +0100
> From: Alessandro Raganato <raganato at di.uniroma1.it>
> Subject: [Corpora-List] Word Sense Disambiguation: a Unified
> Evaluation Framework and Empirical Comparison
> To: corpora at uib.no
> The unified evaluation framework for Word Sense Disambiguation (WSD) is
> available at http://lcl.uniroma1.it/wsdeval .
> We have gathered together five popular all-words WSD evaluation datasets
> and two training datasets, standardizing their format and sense inventory,
> providing a unified evaluation framework. WSD is a long-standing task in
> Natural Language Processing, lying at the core of human language
> understanding. However, the field seems to be slowing down due to the lack
> of groundbreaking improvements. We argue that this is partly due to the
> lack
> of a standard benchmark, which prevents new approaches to be easily
> compared with old approaches. Current benchmarks tend to differ in format,
> construction guidelines and underlying sense inventory.
> In our work we used this framework to perform an empirical comparison among
> a set of heterogeneous approaches, including latest advances based on
> neural networks. All supervised approaches were trained on the same
> preprocessed corpora, ensuring a fair comparison among all systems.
> Additionally, we have enabled a competition in CodaLab
> <https://competitions.codalab.org/competitions/15984> for testing new
> models (or models not considered in our empirical comparison).
> If you would like to contribute to the framework with sense-annotated
> training data or other evaluation datasets, you can share it with us
> (instructions in the website
> <http://lcl.uniroma1.it/wsdeval/share-your-data>).
> Let?s make WSD great again! :)
> For more information, please read the reference paper:
> Alessandro Raganato, Jose Camacho-Collados and Roberto Navigli.
> Word Sense Disambiguation: A Unified Evaluation Framework and Empirical
> Comparison
> <http://lcl.uniroma1.it/wsdeval/data/EACL17_WSD_EvaluationFramework.pdf>.
> Proceedings of EACL 2017, Valencia, Spain
> --
> =====================================
> Alessandro Raganato
> Dipartimento di Informatica
> Sapienza University of Rome
> Viale Regina Elena 295
> 00161 Roma Italy
> Home Page: http://wwwusers.di.uniroma1.it/~raganato
> =====================================
-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/html Size: 5044 bytes Desc: not available URL: <https://mailman.uib.no/public/corpora/attachments/20170117/70ee7a9d/attachment.txt>

More information about the Corpora mailing list