This doesn't answer your question directly, but we have recently announced a shared task on comparable corpora, http://comparable.limsi.fr/bucc2015/bucc2015-task.html
It includes the initial training set for experiments, as well as the evaluation procedure.
I guess that in your case you're interested in an extrinsic evaluation measure. My feeling is that for evaluation purposes you need to have a small parallel corpus to access reference translations. However, in comparison to the traditional parallel corpora for MT development, this corpus can be quite small.
Best wishes, Serge
On 14/02/15 08:34, javid dadashkarimi wrote:
> Dear Everybody,
> Does anybody know, in the machine translation task, how we can evaluate
> a weakly comparable corpus? In such a corpus there are not necessarily
> parallel sentences in the corpus. In the best of our knowledge measures
> like BLUE heavily depend on existing parallel sentences. On the other
> hand, comparability measures like binary compatibility or the vector
> space measure, which exploit bilingual dictionaries in their
> computations, are not specific for the machine translation purpose.