Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Filtered vs Non-filtered Translation model (Nat Gillin)
2. Sample SGML and XML files used by mteval-13a.pl (Nat Gillin)
3. Are the numbers in the phrase-table.gz probabilities or log
prob? (Nat Gillin)
4. Re: Are the numbers in the phrase-table.gz probabilities or
log prob? (Nat Gillin)
5. merge lader preordering models for Japanese MT (Vito Mandorino)
----------------------------------------------------------------------
Message: 1
Date: Wed, 12 Oct 2016 08:41:18 +0800
From: Nat Gillin <nat.gillin@gmail.com>
Subject: Re: [Moses-support] Filtered vs Non-filtered Translation
model
To: Philipp Koehn <phi@jhu.edu>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAD2EOZihXA7WyagK85Jb460sxn-NeBM_EOGkcrVqbG9kHtxSKQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Phi,
Thanks for the tips on filtering!
Regards,
Nat
On Tue, Oct 11, 2016 at 9:42 PM, Philipp Koehn <phi@jhu.edu> wrote:
> Hi,
>
> filtering was original introduced as a necessity to deal with memory usage
> of the
> translation model. Since then, the language models have become bigger and
> the data structures to store the translation model more compact, so it may
> not be
> a useful step anymore.
>
> There may be a bit of a speed effect, especially during startup, but it is
> mainly
> done for memory size.
>
> -phi
>
> On Mon, Oct 10, 2016 at 11:41 PM, Nat Gillin <nat.gillin@gmail.com> wrote:
>
>> Dear Moses community,
>>
>> I have a question regarding filtered and non-filtered translation models,
>> is it a common methodology to always filter the models prior to testing?
>>
>> Is it right that the filtering is to reduce the phrase-table and
>> reordering table size such that decoding is faster? Given the appropriate
>> beam size and more computing power the same accuracy (BLEU or otherwise)
>> can also be achieved without a filtering?
>>
>> Regards,
>> Nat
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161011/24038ed2/attachment-0001.html
------------------------------
Message: 2
Date: Wed, 12 Oct 2016 08:43:43 +0800
From: Nat Gillin <nat.gillin@gmail.com>
Subject: [Moses-support] Sample SGML and XML files used by
mteval-13a.pl
To: moses-support@mit.edu
Message-ID:
<CAD2EOZj29WYD=+BGjUFNvnkRTw228+aXS9Ao0DFnP9G8jraEdg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Moses community,
Is it right that the mteval-13a.pl is the canonical BLEU/NIST evaluation
used in WMT and by most researches reporting on BLEU?
Are there links to sample SGML and XML files that is used by mteval-13a.pl ?
I understand that there's also Moses BLEU in the C++ code and also the
multi-bleu.pl. Are there other bleu versions out there? Are they the same?
Thanks in advance for the tips!
Regards,
Nat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161011/81f8ef2b/attachment-0001.html
------------------------------
Message: 3
Date: Wed, 12 Oct 2016 16:48:10 +0800
From: Nat Gillin <nat.gillin@gmail.com>
Subject: [Moses-support] Are the numbers in the phrase-table.gz
probabilities or log prob?
To: moses-support@mit.edu
Message-ID:
<CAD2EOZjPp1VEh0fd=1SPaYwb529a1LoWThBzHVsiU4b8O2pwYg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Moses community,
After training the model with train-model.pl, I have got several files from
the model.
Are the numbers in the phrase-table.gz probabilities or log prob? They look
like probabiltiites (ranging from 0-1), but I would like to confirm it. Is
that also the same for the reordering-table.wbe-msd-bidirectional-fe.gz ?
And as for the weights in moses.ini after MERT, are they also raw
probabilities or log probabilities?
Thanks in advance for the information!
Regards,
Nat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161012/1689bd95/attachment-0001.html
------------------------------
Message: 4
Date: Wed, 12 Oct 2016 17:59:37 +0800
From: Nat Gillin <nat.gillin@gmail.com>
Subject: Re: [Moses-support] Are the numbers in the phrase-table.gz
probabilities or log prob?
To: moses-support@mit.edu
Message-ID:
<CAD2EOZg9mXkQy5-uDwr+0CKHZLbfJ2c6UcvAP4cgw-hAppnOUg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Moses community,
One more question on the numbers in the phrase-table and moses.ini.
Assuming that the numbers in the phrase-table are probabilities (not yet
log) If i were to multiply the weights from moses.ini to the phrase-table,
would it be:
log(weight * phrase-tabe-prob)
or
weight * log(phrase-table-prob)
Considering that the decoder later do an argmax, i guess the value doesn't
really matter. But it'll be good to know if its log after multiplying the
weights or weights multiplied by the log.
Regards,
Nat
On Wed, Oct 12, 2016 at 4:48 PM, Nat Gillin <nat.gillin@gmail.com> wrote:
> Dear Moses community,
>
> After training the model with train-model.pl, I have got several files
> from the model.
>
> Are the numbers in the phrase-table.gz probabilities or log prob? They
> look like probabiltiites (ranging from 0-1), but I would like to confirm
> it. Is that also the same for the reordering-table.wbe-msd-bidirectional-fe.gz
> ?
>
> And as for the weights in moses.ini after MERT, are they also raw
> probabilities or log probabilities?
>
> Thanks in advance for the information!
>
> Regards,
> Nat
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161012/e59dce8b/attachment-0001.html
------------------------------
Message: 5
Date: Wed, 12 Oct 2016 12:00:52 +0200
From: Vito Mandorino <vito.mandorino@linguacustodia.com>
Subject: [Moses-support] merge lader preordering models for Japanese
MT
To: moses-support <moses-support@mit.edu>
Message-ID:
<CA+8mSmGtnGvizifH6EFUj0P_uLwR8PgLFJou4roO38ewfWtgOQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear all,
this is not a Moses question strictly speaking. Is there anyone who knows
if it is possible to separately train two Lader preordering models (for
Japanese-English models) and merge them subsequently? I believe that was
possible but I cannot find any information at the documentation page
http://www.phontron.com/lader/
Best regards,
Vito
--
*M**. Vito MANDORINO -- Chief Scientist*
[image: Description : Description : lingua_custodia_final full logo]
*The Translation Trustee*
*1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
*Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89*
*Email :* *vito.mandorino@linguacustodia.com
<massinissa.ahmim@linguacustodia.com>*
*Website :*
*www.linguacustodia.finance <http://www.linguacustodia.com/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161012/c26dfd35/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20161012/c26dfd35/attachment.jpg
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 120, Issue 12
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 120, Issue 12"
Post a Comment