Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Moses SMT Evaluation (Maxim Khalilov)
2. NMT vs Moses (Nat Gillin)
3. Re: NMT vs Moses (Marcin Junczys-Dowmunt)
4. Re: NMT vs Moses (Barry Haddow)
----------------------------------------------------------------------
Message: 1
Date: Thu, 24 Nov 2016 09:09:52 +0100
From: Maxim Khalilov <maxkhalilov@gmail.com>
Subject: Re: [Moses-support] Moses SMT Evaluation
To: Emmanuel Dennis <emmanueldennisb@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAPKeiSD4Q0ZdYDFx+kRbG3rnhEKPTt9-ykErL89_16ZgC6atjA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Try http://asiya.cs.upc.edu/demo/asiya_online.php
On Thu, Nov 24, 2016 at 8:41 AM, Emmanuel Dennis <emmanueldennisb@gmail.com>
wrote:
> Is there a way to evaluate SMT system online using BLEU??
>
>
> I will appreciate your feedback.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
*Maxim Khalilov*
*Mob.: +31 615 602 017*
*Skype: Maxim Khalilov (desperbcn)*
*Twitter: twitter.com/maximkhalilov <http://twitter.com/maximkhalilov>*
*LinkedIn: https://nl.linkedin.com/in/maximkhalilov
<https://nl.linkedin.com/in/maximkhalilov>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161124/95459b2e/attachment-0001.html
------------------------------
Message: 2
Date: Thu, 24 Nov 2016 18:22:02 +0800
From: Nat Gillin <nat.gillin@gmail.com>
Subject: [Moses-support] NMT vs Moses
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAD2EOZjzKXMKx5F=1Y0d-G4kZeWQ5oomsXw0Dszn2-DyG6MzfQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Moses Community,
This seems to be prickly topic to discuss but my experiments on a different
kind of data set than WMT or WAT (workshop for asian translation) has not
been able to achieve the stella scores that the recent advancement in MT
has been reporting.
Using state-of-art encoder-attention-decoder framework, just by running
things like lamtram or tensorflow, I'm unable to beat Moses' scores from
sentences that appears both in the train and test data.
Imagine it's a translator using MT and somehow he/she has translated the
sentence before and just wants the exact translation. A TM would solve the
problem and Moses surely could emulate the TM but NMT tends to go overly
creative and produces something else. Although it is consistent in giving
the same output for the same sentence, it's just unable to regurgitate the
sentence that was seen in the training data. In that matter, Moses does it
pretty well.
For sentences that is not in train but in test, NMT does fairly the same or
sometimes better than Moses.
So the question is 'has anyone encounter similar problems?' Is the solution
simply to do a fetch in the train set before translating? Or a
system/output chooser to rerank outputs?
Are there any other ways to resolve such a problem? What could have
happened such that NMT is not "remembering"? (Maybe it needs some
memberberries)
Any tips/hints/discussion on this is much appreciated.
Regards,
Nat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161124/236352d8/attachment-0001.html
------------------------------
Message: 3
Date: Thu, 24 Nov 2016 10:32:19 +0000
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] NMT vs Moses
To: moses-support@mit.edu
Message-ID: <f5e26040-7a0d-3ac7-57e1-44420a0044fb@amu.edu.pl>
Content-Type: text/plain; charset="windows-1252"
Hi,
the short answer to your problem would be, that the typical encoder
decoder models are not really meant to do what you want it to do, there
is however interesting new work on archive:
https://arxiv.org/abs/1611.01874
which could exactly solve your problem. However, I am always weary of
results of that particular group of researchers. It seems reproducing
their results for anything but Chinese does not really work, also their
train sets are really small, so it is not clear what the effects really
are. Maybe those models are just dealing better with smaller data.
Best,
Marcin
W dniu 24/11/16 o 10:22, Nat Gillin pisze:
> Dear Moses Community,
>
> This seems to be prickly topic to discuss but my experiments on a
> different kind of data set than WMT or WAT (workshop for asian
> translation) has not been able to achieve the stella scores that the
> recent advancement in MT has been reporting.
>
> Using state-of-art encoder-attention-decoder framework, just by
> running things like lamtram or tensorflow, I'm unable to beat Moses'
> scores from sentences that appears both in the train and test data.
>
> Imagine it's a translator using MT and somehow he/she has translated
> the sentence before and just wants the exact translation. A TM would
> solve the problem and Moses surely could emulate the TM but NMT tends
> to go overly creative and produces something else. Although it is
> consistent in giving the same output for the same sentence, it's just
> unable to regurgitate the sentence that was seen in the training data.
> In that matter, Moses does it pretty well.
>
> For sentences that is not in train but in test, NMT does fairly the
> same or sometimes better than Moses.
>
> So the question is 'has anyone encounter similar problems?' Is the
> solution simply to do a fetch in the train set before translating? Or
> a system/output chooser to rerank outputs?
>
> Are there any other ways to resolve such a problem? What could have
> happened such that NMT is not "remembering"? (Maybe it needs some
> memberberries)
>
> Any tips/hints/discussion on this is much appreciated.
>
> Regards,
> Nat
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161124/ebbae001/attachment-0001.html
------------------------------
Message: 4
Date: Thu, 24 Nov 2016 11:00:13 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] NMT vs Moses
To: Nat Gillin <nat.gillin@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <050a6778-b8ff-e61b-8199-83802018ef8e@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="windows-1252"
Hi Nat
> Imagine it's a translator using MT and somehow he/she has translated
> the sentence before and just wants the exact translation. A TM would
> solve the problem and Moses surely could emulate the TM but NMT tends
> to go overly creative and produces something else.
Then just use a TM for this. Fast and simple.
You can probably create a seq2seq model which will do the copying when
appropriate (see e.g.
https://www.aclweb.org/anthology/P/P16/P16-1154.pdf), but in the
scenario you describe I think there is really no need.
cheers - Barry
On 24/11/16 10:22, Nat Gillin wrote:
> Dear Moses Community,
>
> This seems to be prickly topic to discuss but my experiments on a
> different kind of data set than WMT or WAT (workshop for asian
> translation) has not been able to achieve the stella scores that the
> recent advancement in MT has been reporting.
>
> Using state-of-art encoder-attention-decoder framework, just by
> running things like lamtram or tensorflow, I'm unable to beat Moses'
> scores from sentences that appears both in the train and test data.
>
> Imagine it's a translator using MT and somehow he/she has translated
> the sentence before and just wants the exact translation. A TM would
> solve the problem and Moses surely could emulate the TM but NMT tends
> to go overly creative and produces something else. Although it is
> consistent in giving the same output for the same sentence, it's just
> unable to regurgitate the sentence that was seen in the training data.
> In that matter, Moses does it pretty well.
>
> For sentences that is not in train but in test, NMT does fairly the
> same or sometimes better than Moses.
>
> So the question is 'has anyone encounter similar problems?' Is the
> solution simply to do a fetch in the train set before translating? Or
> a system/output chooser to rerank outputs?
>
> Are there any other ways to resolve such a problem? What could have
> happened such that NMT is not "remembering"? (Maybe it needs some
> memberberries)
>
> Any tips/hints/discussion on this is much appreciated.
>
> Regards,
> Nat
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161124/e96be87d/attachment.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161124/e96be87d/attachment.pl
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 121, Issue 39
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 121, Issue 39"
Post a Comment