Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: moses 2.1 vs 3 (Hieu Hoang)
2. Re: tokenizer for German language (Hieu Hoang)
3. Re: moses 2.1 vs 3 (Mohammad Salameh)
4. Re: moses 2.1 vs 3 (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Mon, 28 Mar 2016 17:45:10 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] moses 2.1 vs 3
To: Mohammad Salameh <msalameh83@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjWetokhYZyC4aE4vZRimyZfpOB2Hko+oTkpeVmNz+=9Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Mohammad
Can you make your model files available for download. I'll take a look at
it. It should work the same asy
Hieu Hoang
http://www.hoang.co.uk/hieu
On 28 Mar 2016 5:01 pm, "Mohammad Salameh" <msalameh83@gmail.com> wrote:
> Hi
> I have recently started using Moses 3.0 and I noticed the following
> difference compared to Moses 2.1
> Assuming ttable-limit is set to 20, Moses 3.0 retrieves the 20 translation
> options of each source segment from the phrase table. But before
> calculating the future costs and while it is checking the translation
> options spans, Moses 3.0 only considers the top 8 options for each span.
> Moses 2.1 considers the exact 20 options for each source span.
>
> I have tested this using the same configuration files on both systems.
> Although it never mentions that it pruned anything, but it seems most of
> the translations options are pruned
>
> *Moses 2.1*
> Total translation options: 814
> Total translation options pruned: 0
> translation options spanning from 0 to 0 is 20
> ...
>
> *Moses 3.0*
> Total translation options: 206
> Total translation options pruned: 0
> translation options spanning from 0 to 0 is 8
> ...
>
> Is there a parameter I can set to consider exactly the top 20 without
> discarding any?
>
>
> Regards,
> Salameh
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160328/480aef66/attachment-0001.html
------------------------------
Message: 2
Date: Mon, 28 Mar 2016 18:04:17 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] tokenizer for German language
To: Alapan Kuila <alapan.cse@gmail.com>, moses-support@mit.edu
Message-ID: <56F96411.8010108@gmail.com>
Content-Type: text/plain; charset="windows-1252"
use the tokenizer.perl script with the arg
-l de
On 28/03/2016 15:23, Alapan Kuila wrote:
> Sir,
>
> Please provide some script or program for German tokenizer.
> Is it right to use tokenizer.perl (
> ~/moses/scripts/tokenizer/tokenizer.perl) for German
> tokenization?Please help.
> Thanks and regards,
>
> Alapan
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160328/44e2ee59/attachment-0001.html
------------------------------
Message: 3
Date: Mon, 28 Mar 2016 11:59:29 -0600
From: Mohammad Salameh <msalameh83@gmail.com>
Subject: Re: [Moses-support] moses 2.1 vs 3
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CADbe+DE1HiVvjMO_VtXOtfoWxT_9rVOMAj-wVbESaRhuB0xePA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Hieu
Please find the model files on Google drive.
https://drive.google.com/folderview?id=0B9m7_C7udg78dElpWUpubG1qV28&usp=sharing
I have generated a smaller phrase table, reordering model (filtered based
on tst.en.5) and language model for sharing. I was also able to reproduce
the same behavior.
I am also attaching the log files based on translating tst.en.1 using the
different moses systems.
Thanks,
Salameh
On Mon, Mar 28, 2016 at 10:45 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
> Hi Mohammad
>
> Can you make your model files available for download. I'll take a look at
> it. It should work the same asy
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
> On 28 Mar 2016 5:01 pm, "Mohammad Salameh" <msalameh83@gmail.com> wrote:
>
>> Hi
>> I have recently started using Moses 3.0 and I noticed the following
>> difference compared to Moses 2.1
>> Assuming ttable-limit is set to 20, Moses 3.0 retrieves the 20
>> translation options of each source segment from the phrase table. But
>> before calculating the future costs and while it is checking the
>> translation options spans, Moses 3.0 only considers the top 8 options for
>> each span.
>> Moses 2.1 considers the exact 20 options for each source span.
>>
>> I have tested this using the same configuration files on both systems.
>> Although it never mentions that it pruned anything, but it seems most of
>> the translations options are pruned
>>
>> *Moses 2.1*
>> Total translation options: 814
>> Total translation options pruned: 0
>> translation options spanning from 0 to 0 is 20
>> ...
>>
>> *Moses 3.0*
>> Total translation options: 206
>> Total translation options pruned: 0
>> translation options spanning from 0 to 0 is 8
>> ...
>>
>> Is there a parameter I can set to consider exactly the top 20 without
>> discarding any?
>>
>>
>> Regards,
>> Salameh
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160328/dc69b7cb/attachment-0001.html
------------------------------
Message: 4
Date: Tue, 29 Mar 2016 00:04:36 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] moses 2.1 vs 3
To: Mohammad Salameh <msalameh83@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <56F9B884.5000400@gmail.com>
Content-Type: text/plain; charset="windows-1252"
well spotted, there seemed to have been a mistake which I've just fixed
https://github.com/moses-smt/mosesdecoder/commit/b8bc4a9fb64731b593ba56cce1da1af0bbb6138e
But are you sure the problem was in Moses 3? It seemed the bug was
introduced only in Dec 2015
https://github.com/moses-smt/mosesdecoder/commit/65f4f1f92af3d645b1dfc2f161bbcf47bdce1402
whereas Moses 3 was released in Jan 2015
On 28/03/2016 16:58, Mohammad Salameh wrote:
> Hi
> I have recently started using Moses 3.0 and I noticed the following
> difference compared to Moses 2.1
> Assuming ttable-limit is set to 20, Moses 3.0 retrieves the 20
> translation options of each source segment from the phrase table. But
> before calculating the future costs and while it is checking the
> translation options spans, Moses 3.0 only considers the top 8 options
> for each span.
> Moses 2.1 considers the exact 20 options for each source span.
>
> I have tested this using the same configuration files on both systems.
> Although it never mentions that it pruned anything, but it seems most
> of the translations options are pruned
>
> *Moses 2.1*
> Total translation options: 814
> Total translation options pruned: 0
> translation options spanning from 0 to 0 is 20
> ...
> *
> *
> *Moses 3.0*
> Total translation options: 206
> Total translation options pruned: 0
> translation options spanning from 0 to 0 is 8
> ...
>
> Is there a parameter I can set to consider exactly the top 20 without
> discarding any?
>
>
> Regards,
> Salameh
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160328/26689bb2/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 113, Issue 71
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 113, Issue 71"
Post a Comment