Moses-support Digest, Vol 105, Issue 75

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: difference in bleu scores (Philipp Koehn)
2. Re: Parallelizer multi core (Vincent Nguyen)
3. Re: Parallelizer multi core (Vincent Nguyen)
4. Re: difference in bleu scores (Arefeh Kazemi Najafabadi)


----------------------------------------------------------------------

Message: 1
Date: Fri, 31 Jul 2015 07:58:15 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] difference in bleu scores
To: Arefeh Kazemi Najafabadi <arefeh.kazeminajafabadi@dcu.ie>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDqwGcbY1pcOedx7_ptz6yNOwLKWJU5=+=8H89CzWqpfQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

MERT uses random initialization of starting weights during the
hill-climbing search to optimize weights, so different runs will lead to
different weights.

Having said that, the differences in scores that you observe is very high.
It really should not be much more than 0.1-0.2. You can reduce the
difference by having larger tuning and test sets (thousands of sentences
for each). I also found using kbest MIRA to produce less variance.

-phi


On Fri, Jul 31, 2015 at 6:59 AM, Arefeh Kazemi Najafabadi <
arefeh.kazeminajafabadi@dcu.ie> wrote:

> Hi everyone
>
> I'm testing Moses Hiero on a parallel corpus. To minimize the tuning error,I run MERT three times from flat weights. The bleu score for the first
> tuning run, on the test data set, is 15.5, while for the second and
> third runs it is 16.14 and 16.60. I wonder if this significant difference is normal (1.10 in bleu)? and what's the reason of this difference?
>
> Regards
>
> Arefeh
>
>
> *Email Disclaimer"This e-mail and any files transmitted with it are confidential and are intended solely for use by the addressee. Any unauthorised dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received this e-mail in error, please notify the sender and delete the message. Any views or opinions presented in this e-mail may solely be the views of the author and cannot be relied upon as being those of Dublin City University. E-mail communications such as this cannot be guaranteed to be virus-free, timely, secure or error-free and Dublin City University does not accept liability for any such matters or their consequences. Please consider the environment before printing this e-mail."S?anadh R?omhphoist"T? an r?omhphost seo agus aon chomhad a sheoltar leis faoi r?n agus is lena ?s?id ag an seola? agus sin amh?in ?. T? cosc ioml?n ar scaipeadh, dh?ileadh n? ch?ipe?il neamh?daraithe ar an teachtaireacht seo agus ar !
aon cheangalt?n at? ag dul leis. M? t? an r?omhphost seo faighte agat tr? dhearmad cuir sin in i?l le do thoil don seolt?ir agus scrios an teachtaireacht. D?fh?adfadh s? gurb iad tuairim? an ?dair agus sin amh?in at? in aon tuairim? no dearctha? at? curtha i l?thair sa r?omhphost seo agus n?or ch?ir glacadh leo mar thuairim? n? dhearctha? Ollscoil Chathair Bhaile ?tha Cliath. N? ghlactar leis go bhfuil cumars?id r?omhphoist den s?rt seo saor ? v?reas, in am, sl?n, n? saor ? earr?id agus n? ghlacann Ollscoil Chathair Bhaile ?tha Cliath le dliteanas in aon ch?s den s?rt sin n? as aon iarmhairt a d?eascr?dh astu. Cuimhnigh ar an timpeallacht le do thoil sula gcuireann t? an r?omhphost seo i gcl?."*
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150731/026bf939/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 31 Jul 2015 13:59:03 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: Re: [Moses-support] Parallelizer multi core
To: moses-support <moses-support@mit.edu>
Message-ID: <55BB6307.8080702@neuf.fr>
Content-Type: text/plain; charset="windows-1252"


for inexperienced people like me :)
Add |--decoder-flags="-threads 4"| is key

if EMS config.basic had "-threads all" by default we would gain A LOT of
time.

cheers,

Vincent


Le 29/07/2015 22:05, Vincent Nguyen a ?crit :
> Hi,
>
> I am wondering what tasks of the EMS are really parallelized.
> I activated the script line + 8 cores.
>
> Training / binarizing / Tuning all make only one core to actually work.
>
> Am I correct ?
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150731/3d62b5d4/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 31 Jul 2015 14:13:55 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: Re: [Moses-support] Parallelizer multi core
To: moses-support@mit.edu
Message-ID: <55BB6683.6050102@neuf.fr>
Content-Type: text/plain; charset="windows-1252"


still, comparing apples to apples (on my machine Xeon5670 @ 2.93Ghz)
tuning Europarlv7 fr to en
KenLM took 13 hours
Irstlm it has been running for the past 40 hours and not finished yet .....


Le 31/07/2015 13:59, Vincent Nguyen a ?crit :
>
> for inexperienced people like me :)
> Add |--decoder-flags="-threads 4"| is key
>
> if EMS config.basic had "-threads all" by default we would gain A LOT
> of time.
>
> cheers,
>
> Vincent
>
>
> Le 29/07/2015 22:05, Vincent Nguyen a ?crit :
>> Hi,
>>
>> I am wondering what tasks of the EMS are really parallelized.
>> I activated the script line + 8 cores.
>>
>> Training / binarizing / Tuning all make only one core to actually work.
>>
>> Am I correct ?
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150731/c4d33e7a/attachment-0001.htm

------------------------------

Message: 4
Date: Fri, 31 Jul 2015 16:51:49 +0430
From: Arefeh Kazemi Najafabadi <arefeh.kazeminajafabadi@dcu.ie>
Subject: Re: [Moses-support] difference in bleu scores
To: Philipp Koehn <phi@jhu.edu>, moses-support@mit.edu
Message-ID:
<CAB2b+QXX6dCi3pH998C_59PY=Y5iKS6KGLO0EvpHM_AUz7v_9Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Philipp

Thanks for the reply.
I have 3000 sentences for tune and 1000 sentence for test. Aren't they
enough?

Regards
Arefeh


On 31 July 2015 at 16:28, Philipp Koehn <phi@jhu.edu> wrote:

> Hi,
>
> MERT uses random initialization of starting weights during the
> hill-climbing search to optimize weights, so different runs will lead to
> different weights.
>
> Having said that, the differences in scores that you observe is very high.
> It really should not be much more than 0.1-0.2. You can reduce the
> difference by having larger tuning and test sets (thousands of sentences
> for each). I also found using kbest MIRA to produce less variance.
>
> -phi
>
>
> On Fri, Jul 31, 2015 at 6:59 AM, Arefeh Kazemi Najafabadi <
> arefeh.kazeminajafabadi@dcu.ie> wrote:
>
>> Hi everyone
>>
>> I'm testing Moses Hiero on a parallel corpus. To minimize the tuning error,I run MERT three times from flat weights. The bleu score for the first
>> tuning run, on the test data set, is 15.5, while for the second and
>> third runs it is 16.14 and 16.60. I wonder if this significant difference is normal (1.10 in bleu)? and what's the reason of this difference?
>>
>> Regards
>>
>> Arefeh
>>
>>
>> *Email Disclaimer"This e-mail and any files transmitted with it are confidential and are intended solely for use by the addressee. Any unauthorised dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received this e-mail in error, please notify the sender and delete the message. Any views or opinions presented in this e-mail may solely be the views of the author and cannot be relied upon as being those of Dublin City University. E-mail communications such as this cannot be guaranteed to be virus-free, timely, secure or error-free and Dublin City University does not accept liability for any such matters or their consequences. Please consider the environment before printing this e-mail."S?anadh R?omhphoist"T? an r?omhphost seo agus aon chomhad a sheoltar leis faoi r?n agus is lena ?s?id ag an seola? agus sin amh?in ?. T? cosc ioml?n ar scaipeadh, dh?ileadh n? ch?ipe?il neamh?daraithe ar an teachtaireacht seo agus ar!
aon cheangalt?n at? ag dul leis. M? t? an r?omhphost seo faighte agat tr? dhearmad cuir sin in i?l le do thoil don seolt?ir agus scrios an teachtaireacht. D?fh?adfadh s? gurb iad tuairim? an ?dair agus sin amh?in at? in aon tuairim? no dearctha? at? curtha i l?thair sa r?omhphost seo agus n?or ch?ir glacadh leo mar thuairim? n? dhearctha? Ollscoil Chathair Bhaile ?tha Cliath. N? ghlactar leis go bhfuil cumars?id r?omhphoist den s?rt seo saor ? v?reas, in am, sl?n, n? saor ? earr?id agus n? ghlacann Ollscoil Chathair Bhaile ?tha Cliath le dliteanas in aon ch?s den s?rt sin n? as aon iarmhairt a d?eascr?dh astu. Cuimhnigh ar an timpeallacht le do thoil sula gcuireann t? an r?omhphost seo i gcl?."*
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>

--


*Email Disclaimer"This e-mail and any files transmitted with it are confidential and are intended solely for use by the addressee. Any unauthorised dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received this e-mail in error, please notify the sender and delete the message. Any views or opinions presented in this e-mail may solely be the views of the author and cannot be relied upon as being those of Dublin City University. E-mail communications such as this cannot be guaranteed to be virus-free, timely, secure or error-free and Dublin City University does not accept liability for any such matters or their consequences. Please consider the environment before printing this e-mail."S?anadh R?omhphoist"T? an r?omhphost seo agus aon chomhad a sheoltar leis faoi r?n agus is lena ?s?id ag an seola? agus sin amh?in ?. T? cosc ioml?n ar scaipeadh, dh?ileadh n? ch?ipe?il neamh?daraithe ar an teachtaireacht seo agus ar ao!
n cheangalt?n at? ag dul leis. M? t? an r?omhphost seo faighte agat tr? dhearmad cuir sin in i?l le do thoil don seolt?ir agus scrios an teachtaireacht. D?fh?adfadh s? gurb iad tuairim? an ?dair agus sin amh?in at? in aon tuairim? no dearctha? at? curtha i l?thair sa r?omhphost seo agus n?or ch?ir glacadh leo mar thuairim? n? dhearctha? Ollscoil Chathair Bhaile ?tha Cliath. N? ghlactar leis go bhfuil cumars?id r?omhphoist den s?rt seo saor ? v?reas, in am, sl?n, n? saor ? earr?id agus n? ghlacann Ollscoil Chathair Bhaile ?tha Cliath le dliteanas in aon ch?s den s?rt sin n? as aon iarmhairt a d?eascr?dh astu. Cuimhnigh ar an timpeallacht le do thoil sula gcuireann t? an r?omhphost seo i gcl?."*

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150731/6a101cef/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 105, Issue 75
**********************************************

Tweet