Moses-support Digest, Vol 168, Issue 3

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: [Chat translation task WMT20] updates on manual
evaluations and reference translations (Amin Farajian)


----------------------------------------------------------------------

Message: 1
Date: Thu, 8 Oct 2020 17:46:34 +0100
From: Amin Farajian <ma.farajian@gmail.com>
Subject: Re: [Moses-support] [Chat translation task WMT20] updates on
manual evaluations and reference translations
To: wmt-tasks@googlegroups.com
Cc: moses-support@mit.edu, vincentwang0229@gmail.com
Message-ID: <D61600AE-B60A-445D-BE2C-279C54BD3C1C@gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Liam,

There were several factors that led us to perform the manual evaluation only on the agent side (English-German). The customer source text was originally in English and was translated into German. We realised that showing the noisy English text to the evaluators as the reference could be misleading for them impacting their evaluations.
Moreover, in this setting it is usually fine for the agent to have a gist of the message; however there is extra responsibility in translating the agent since it has the company brand with it.
Therefore, we preferred to focus more on evaluating the agent translations more rigorously than to spend resources in evaluating the customer.

Hope this answers your question.
Best,
Amin

> On 8 Oct 2020, at 12:35, LIANG DING <liangding.liam@gmail.com> wrote:
>
> Dear Dr. Amin Farajian,
>
> Thank you for the effort to organize the manual evaluations of WMT Chat Translations~
>
> I have a question, why is there only English-German in this manual evaluation result, but no opposite direction, i.e., German-English?
>
> Thank you again for your efforts and look forward to your Reply
>
> Liam
>
> Amin Farajian <ma.farajian@gmail.com <mailto:ma.farajian@gmail.com>> ?2020?10?8??? ??6:40???
> Dear Participants,
>
> We would like to inform you that the results of the manual document-level direct assessments of all the primary submissions of the Chat Translation shared task at WMT 2020 are now available on the website http://www.statmt.org/wmt20/chat-task.html <http://www.statmt.org/wmt20/chat-task.html>.
> Moreover, the reference translations of the test set are also available on the website.
>
> Thank you very much,
> Amin
>
> --
> You received this message because you are subscribed to the Google Groups "Workshop on Statistical Machine Translation" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wmt-tasks+unsubscribe@googlegroups.com <mailto:wmt-tasks+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wmt-tasks/8ED074F8-D847-4516-9473-52924C05645D%40gmail.com <https://groups.google.com/d/msgid/wmt-tasks/8ED074F8-D847-4516-9473-52924C05645D%40gmail.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to the Google Groups "Workshop on Statistical Machine Translation" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to wmt-tasks+unsubscribe@googlegroups.com <mailto:wmt-tasks+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/wmt-tasks/CAHZN6XBuPc8iq4Fh%3DG3PSQPAdQRxawYEC9aA%3DO1Uszxv0DSAvA%40mail.gmail.com <https://groups.google.com/d/msgid/wmt-tasks/CAHZN6XBuPc8iq4Fh%3DG3PSQPAdQRxawYEC9aA%3DO1Uszxv0DSAvA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20201008/d7fbdecb/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 168, Issue 3
*********************************************

Tweet