[Corpora-List] Evaluation Measures for Weakly CC in MTs

Serge Sharoff s.sharoff at leeds.ac.uk
Tue Feb 17 16:43:16 CET 2015

Dear Javid,

This doesn't answer your question directly, but we have recently announced a shared task on comparable corpora, http://comparable.limsi.fr/bucc2015/bucc2015-task.html

It includes the initial training set for experiments, as well as the evaluation procedure.

I guess that in your case you're interested in an extrinsic evaluation measure. My feeling is that for evaluation purposes you need to have a small parallel corpus to access reference translations. However, in comparison to the traditional parallel corpora for MT development, this corpus can be quite small.

Best wishes, Serge

On 14/02/15 08:34, javid dadashkarimi wrote:
> Dear Everybody,
> Does anybody know, in the machine translation task, how we can evaluate
> a weakly comparable corpus? In such a corpus there are not necessarily
> parallel sentences in the corpus. In the best of our knowledge measures
> like BLUE heavily depend on existing parallel sentences. On the other
> hand, comparability measures like binary compatibility or the vector
> space measure, which exploit bilingual dictionaries in their
> computations, are not specific for the machine translation purpose.
> Best,
> Javid

More information about the Corpora mailing list