Moses-support Digest, Vol 104, Issue 58

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Major bug found in Moses (Lane Schwartz)
2. Dependencies in EMS/Experiment.perl (Evgeny Matusov)
3. Re: Major bug found in Moses (Read, James C)


----------------------------------------------------------------------

Message: 1
Date: Fri, 19 Jun 2015 11:40:49 -0500
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] Major bug found in Moses
To: "Read, James C" <jcread@essex.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, Philipp Koehn
<phi@jhu.edu>
Message-ID:
<CABv3vZnKz4GRH4zLZpf1aS8kwjejbmN=29ecQafpMRZ+Tab3Uw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

On Fri, Jun 19, 2015 at 11:28 AM, Read, James C <jcread@essex.ac.uk> wrote:

What I take issue with is the en-masse denial that there is a problem with
> the system if it behaves in such a way with no LM + no pruning and/or
> tuning.
>

There is no mass denial taking place.

Regardless of whether or not you tune, the decoder will do its best to find
translations with the highest model score. That is the expected behavior.

What I have tried to tell you, and what other people have tried to tell
you, is that translations with high model scores are not necessarily good
translations.

We all want our models to be such that high model scores correspond to good
translations, and that low model scores correspond with bad translations.
But unfortunately, our models do not innately have this characteristic. We
all know this. We also know a good way to deal with this shortcoming,
namely tuning. Tuning is the process by which we attempt to ensure that
high model scores correspond to high quality translations, and that low
model scores correspond to low quality translations.

If you can design models that naturally correspond with translation quality
without tuning, that's great. If you can do that, you've got a great shot
at winning a Best Paper award at ACL.

In the meantime, you may want to consider an apology for your rude behavior
and unprofessional attitude.

Goodbye.
Lane
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150619/c826fb7f/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 19 Jun 2015 16:41:08 +0000
From: Evgeny Matusov <ematusov@apptek.com>
Subject: [Moses-support] Dependencies in EMS/Experiment.perl
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CY1PR0801MB1595DAAC0A6DF11C0F5FBB5DACA40@CY1PR0801MB1595.namprd08.prod.outlook.com>

Content-Type: text/plain; charset="iso-8859-1"

Hi,


to those of you using Experiment.perl for experiments, maybe you can help me solve the following problem:


I added a step to filter full segment overlap of evaluation and tuning data with the training data. This steps removes all sentences from each CORPUS which are also found in EVALUATION and TUNING sentences. Thus, one of the CORPUS steps depend on EVALUATION and TUNING.


Now, I want to exchange the tuning corpus I am using, picking another one which was already declared in the EVALUATION section. Thus, the filter against which the overlap is checked does not change, and hence the training data does not need to be filtered again, and therefore neither the alignment training nor LM training or anything else should be repeated, just the tuning step should re-start. However, Experiment.perl is not smart enough to realize this. I tried to add "pass-if" or "ignore-if" step on the filter-overlap step that I declared and set a variable to pass it, but this did not help - all steps after it are still executed. Setting TRAINING:config to a valid moses.ini file helps to prevent the alignment training from running, but not the LM training, nor (more importantly), the several cleaning/lowercasing steps that follow the overlap step for each training corpus.


Is there an easy way to block everything below tuning from being repeated, even if the tuning data changes?


Thanks,

Evgeny.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150619/8192611e/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 19 Jun 2015 16:49:26 +0000
From: "Read, James C" <jcread@essex.ac.uk>
Subject: Re: [Moses-support] Major bug found in Moses
To: Lane Schwartz <dowobeha@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, Philipp Koehn
<phi@jhu.edu>
Message-ID:
<DB3PR06MB07131DA35AF788B39942764985A40@DB3PR06MB0713.eurprd06.prod.outlook.com>

Content-Type: text/plain; charset="iso-8859-1"

So, all I did was filter out the less likely phrase pairs and the BLEU score shot up. Was that such a stroke of genius? Was that not blindingly obvious?


Your telling me that redesigning the search algorithm to prefer higher scoring phrase pairs is all we need to do to get a best paper at ACL?


James


________________________________
From: Lane Schwartz <dowobeha@gmail.com>
Sent: Friday, June 19, 2015 7:40 PM
To: Read, James C
Cc: Philipp Koehn; Burger, John D.; moses-support@mit.edu
Subject: Re: [Moses-support] Major bug found in Moses

On Fri, Jun 19, 2015 at 11:28 AM, Read, James C <jcread@essex.ac.uk<mailto:jcread@essex.ac.uk>> wrote:

What I take issue with is the en-masse denial that there is a problem with the system if it behaves in such a way with no LM + no pruning and/or tuning.

There is no mass denial taking place.

Regardless of whether or not you tune, the decoder will do its best to find translations with the highest model score. That is the expected behavior.

What I have tried to tell you, and what other people have tried to tell you, is that translations with high model scores are not necessarily good translations.

We all want our models to be such that high model scores correspond to good translations, and that low model scores correspond with bad translations. But unfortunately, our models do not innately have this characteristic. We all know this. We also know a good way to deal with this shortcoming, namely tuning. Tuning is the process by which we attempt to ensure that high model scores correspond to high quality translations, and that low model scores correspond to low quality translations.

If you can design models that naturally correspond with translation quality without tuning, that's great. If you can do that, you've got a great shot at winning a Best Paper award at ACL.

In the meantime, you may want to consider an apology for your rude behavior and unprofessional attitude.

Goodbye.
Lane

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150619/612e2be0/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 104, Issue 58
**********************************************

Tweet