Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: BLEU Score Variance: Which score to use?
(Marcin Junczys-Dowmunt)
2. Re: BLEU Score Variance: Which score to use? (Hokage Sama)
3. Re: BLEU Score Variance: Which score to use?
(Marcin Junczys-Dowmunt)
4. Re: BLEU Score Variance: Which score to use? (Hokage Sama)
5. How to re-run tuning using EMS (Lane Schwartz)
----------------------------------------------------------------------
Message: 1
Date: Mon, 22 Jun 2015 10:22:43 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] BLEU Score Variance: Which score to use?
To: Hokage Sama <nvncbol@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <5587C5D3.7040908@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed
Difficult to tell with that little data. Once you get beyond 100,000
segments (or 50,000 at least) i would say 2000 per dev (for tuning) and
test set, rest for training. With that few segments it's hard to give
you any recommendations since it might just not give meaningful results.
It's currently a toy model, good for learning and playing around with
options. But not good for trying to infer anything from BLEU scores.
On 22.06.2015 10:17, Hokage Sama wrote:
> Yes the language model was built earlier when I first went through the
> manual to build a French-English baseline system. So I just reused it
> for my Samoan-English system.
> Yes for all three runs I used the same training and testing files.
> How can I determine how much parallel data I should set aside for
> tuning and testing? I have only 10,028 segments (198,385 words)
> altogether. At the moment I'm using 259 segments for testing and the
> rest for training.
>
> Thanks,
> Hilton
>
------------------------------
Message: 2
Date: Mon, 22 Jun 2015 03:31:17 -0500
From: Hokage Sama <nvncbol@gmail.com>
Subject: Re: [Moses-support] BLEU Score Variance: Which score to use?
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAD3ogMbi1ytMa2p70k7KEA=4AGbWhy_Qz9scWYxVA2oJR16p_w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Ok thanks. Appreciate your help.
On 22 June 2015 at 03:22, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:
> Difficult to tell with that little data. Once you get beyond 100,000
> segments (or 50,000 at least) i would say 2000 per dev (for tuning) and
> test set, rest for training. With that few segments it's hard to give you
> any recommendations since it might just not give meaningful results. It's
> currently a toy model, good for learning and playing around with options.
> But not good for trying to infer anything from BLEU scores.
>
>
> On 22.06.2015 10:17, Hokage Sama wrote:
>
>> Yes the language model was built earlier when I first went through the
>> manual to build a French-English baseline system. So I just reused it for
>> my Samoan-English system.
>> Yes for all three runs I used the same training and testing files.
>> How can I determine how much parallel data I should set aside for tuning
>> and testing? I have only 10,028 segments (198,385 words) altogether. At the
>> moment I'm using 259 segments for testing and the rest for training.
>>
>> Thanks,
>> Hilton
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150622/915e743d/attachment-0001.htm
------------------------------
Message: 3
Date: Mon, 22 Jun 2015 10:35:09 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] BLEU Score Variance: Which score to use?
To: Hokage Sama <nvncbol@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <5587C8BD.4060507@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed
You're welcome. Take another close look at those varying bleu scores
though. That would make me worry if it happened to me for the same data
and the same weights.
On 22.06.2015 10:31, Hokage Sama wrote:
> Ok thanks. Appreciate your help.
>
> On 22 June 2015 at 03:22, Marcin Junczys-Dowmunt <junczys@amu.edu.pl
> <mailto:junczys@amu.edu.pl>> wrote:
>
> Difficult to tell with that little data. Once you get beyond
> 100,000 segments (or 50,000 at least) i would say 2000 per dev
> (for tuning) and test set, rest for training. With that few
> segments it's hard to give you any recommendations since it might
> just not give meaningful results. It's currently a toy model, good
> for learning and playing around with options. But not good for
> trying to infer anything from BLEU scores.
>
>
> On 22.06.2015 10 <tel:22.06.2015%2010>:17, Hokage Sama wrote:
>
> Yes the language model was built earlier when I first went
> through the manual to build a French-English baseline system.
> So I just reused it for my Samoan-English system.
> Yes for all three runs I used the same training and testing files.
> How can I determine how much parallel data I should set aside
> for tuning and testing? I have only 10,028 segments (198,385
> words) altogether. At the moment I'm using 259 segments for
> testing and the rest for training.
>
> Thanks,
> Hilton
>
>
>
------------------------------
Message: 4
Date: Mon, 22 Jun 2015 03:39:11 -0500
From: Hokage Sama <nvncbol@gmail.com>
Subject: Re: [Moses-support] BLEU Score Variance: Which score to use?
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAD3ogMa-2g11ussFvOV=v9s-AW1G0+4pNX+xjw66GD2bv4Rysw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Ok I will.
On 22 June 2015 at 03:35, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:
> You're welcome. Take another close look at those varying bleu scores
> though. That would make me worry if it happened to me for the same data and
> the same weights.
>
> On 22.06.2015 10:31, Hokage Sama wrote:
>
>> Ok thanks. Appreciate your help.
>>
>> On 22 June 2015 at 03:22, Marcin Junczys-Dowmunt <junczys@amu.edu.pl
>> <mailto:junczys@amu.edu.pl>> wrote:
>>
>> Difficult to tell with that little data. Once you get beyond
>> 100,000 segments (or 50,000 at least) i would say 2000 per dev
>> (for tuning) and test set, rest for training. With that few
>> segments it's hard to give you any recommendations since it might
>> just not give meaningful results. It's currently a toy model, good
>> for learning and playing around with options. But not good for
>> trying to infer anything from BLEU scores.
>>
>>
>> On 22.06.2015 10 <tel:22.06.2015%2010>:17, Hokage Sama wrote:
>>
>> Yes the language model was built earlier when I first went
>> through the manual to build a French-English baseline system.
>> So I just reused it for my Samoan-English system.
>> Yes for all three runs I used the same training and testing files.
>> How can I determine how much parallel data I should set aside
>> for tuning and testing? I have only 10,028 segments (198,385
>> words) altogether. At the moment I'm using 259 segments for
>> testing and the rest for training.
>>
>> Thanks,
>> Hilton
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150622/15bde343/attachment-0001.htm
------------------------------
Message: 5
Date: Mon, 22 Jun 2015 10:15:04 -0500
From: Lane Schwartz <dowobeha@gmail.com>
Subject: [Moses-support] How to re-run tuning using EMS
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZ=n8eGCA6+9q-LKi8syKpjEMNo-M_thB1vFZAVTDntHDg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Given a successful run of EMS, what do I need to do to configure a new run
that re-uses all of the training, but re-runs tuning?
Thanks,
Lane
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150622/f4573ea8/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 104, Issue 74
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 104, Issue 74"
Post a Comment