Moses-support Digest, Vol 109, Issue 56

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Pruning phrase-table (Barry Haddow)
2. Re: experiment.perl not working with config.toy (Nicholas Ruiz)
3. Re: Pruning phrase-table (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Tue, 24 Nov 2015 09:12:19 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Pruning phrase-table
To: Sanjanashree Palanivel <sanjanashree@gmail.com>,
moses-support@mit.edu
Message-ID: <565429F3.6020907@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="windows-1252"

Hi

You're better off using the Johnson pruning method
http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc5 . The relent
code is no longer maintained,

cheers - Barry

On 24/11/15 05:42, Sanjanashree Palanivel wrote:
>
> Dear All,
>
> I just tried to prune the phrase table using relent-filter
> insisde mosesdecoder. I used the command as mentioned in the moses
> site (http://www.statmt.org/moses/?n=Advanced.RuleTables). But, I am
> getting the following error "Use of uninitialized value $_[0] in
> substitution (s///) at /usr/share/perl/5.18/File/Basename.pm line 341.
> fileparse(): need a valid pathname at
> /home/sanjana/Documents/SMT/mosesdecoder/contrib/relent-filter/scripts/calcPruningScores.pl
> line 140." . Where am i being worng.
>
>
> --
> Thanks and regards,
>
> Sanjanasri J.P
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151124/310a2ed6/attachment-0001.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151124/310a2ed6/attachment-0001.pl

------------------------------

Message: 2
Date: Tue, 24 Nov 2015 05:06:51 -0500
From: Nicholas Ruiz <nicruiz@fbk.eu>
Subject: Re: [Moses-support] experiment.perl not working with
config.toy
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAKa+0YP+umd6Te5v_T1WbC5nAaav-qb3KRoLWM-t9C7aRfYz=w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks, Hieu. It turned out that I needed to use an older version on mgiza
to avoid the static linking. It wasn't feasible in my environment to
recompile gcc with the proper flags (no root access).

To be fair, your github branch of mgiza does say that it's intended for
Windows users. :)
On Nov 23, 2015 8:19 PM, "Hieu Hoang" <hieuhoang@gmail.com> wrote:

> my training-tools directory contains this:
>
> Hieus:~/workspace/bin/training-tools$ll
> total 2984
> drwxr-xr-x 6 hieu staff 204 22 Apr 2015 ./
> drwxr-xr-x 3 hieu staff 102 22 Apr 2015 ../
> -rwxr-xr-x 1 hieu staff 2070 22 Apr 2015 merge_alignment.py*
> -rwxr-xr-x 1 hieu staff 1096056 22 Apr 2015 mgiza*
> -rwxr-xr-x 1 hieu staff 376012 22 Apr 2015 mkcls*
> -rwxr-xr-x 1 hieu staff 48224 22 Apr 2015 snt2cooc*
> Hieus:~/workspace/bin/training-tools$
>
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
> On 20 November 2015 at 20:09, Nicholas Ruiz <nicruiz@fbk.eu> wrote:
>
>> Hi all,
>>
>> I'm having some issues running config.toy with EMS (experiment.py). I
>> have a vanilla installation of Moses, which compiles successfully, but EMS
>> is giving me a headache with mgiza. I followed the instructions here and
>> here to install Moses and mgiza:
>> http://www.statmt.org/moses/?n=Development.GetStarted
>> http://www.statmt.org/moses/?n=Moses.ExternalTools#ntoc3
>>
>> My moses installation is here:
>> [nruiz@cheetah-h2 training-tools]$ cd
>> /n/w10-nruiz/mosesdecoder/bin/training-tools
>> [nruiz@cheetah-h2 training-tools]$ ls -l
>> total 8
>> -rwxr-xr-x 1 nruiz slt 3291 Nov 20 14:52 merge_alignment.py
>> drwxr-xr-x 2 nruiz slt 4096 Nov 20 14:50 mgizapp
>> [nruiz@cheetah-h2 training-tools]$ ls -l mgizapp/
>> total 4188
>> -rw-r--r-- 1 nruiz slt 3848 Nov 20 14:50 alignment.o
>> -rw-r--r-- 1 nruiz slt 11144 Nov 20 14:50 AlignTables.o
>> -rw-r--r-- 1 nruiz slt 57344 Nov 20 14:50 ATables.o
>> ....
>>
>> When executing experiment.perl, I get an error in mgiza, complaining that
>> mkcls can't be found. I'm enclosing my minor modification of config.toy for
>> troubleshooting.
>>
>> Here's the command that fails:
>>
>> $ /n/w10-nruiz/mosesdecoder/scripts/training/train-model.perl -mgiza
>> -mgiza-cpus 16 -dont-zip -first-step 2 -last-step 2 -external-bin-dir
>> /n/w10-nruiz/mosesdecoder/bin/training-tools -f fr -e en -alignment
>> grow-diag-final-and -max-phrase-length 5 -reordering msd-bidirectional-fe
>> -score-options '--GoodTuring --MinScore 2:0.0001' -parallel -corpus-dir
>> /n/w10-nruiz/mt/zh-en/experiment/toy/training/prepared.3 -giza-e2f
>> /n/w10-nruiz/mt/zh-en/experiment/toy/training/giza.5 -direction 2
>>
>> Use of implicit split to @_ is deprecated at
>> /n/w10-nruiz/mosesdecoder/scripts/training/train-model.perl line 2091.
>>
>> Using SCRIPTS_ROOTDIR: /n/w10-nruiz/mosesdecoder/scripts
>>
>> Using multi-thread GIZA
>>
>> using gzip
>>
>> ERROR: Cannot find mkcls, GIZA++/mgiza, & snt2cooc.out/snt2cooc in
>> /n/w10-nruiz/mosesdecoder/bin/training-tools.
>>
>> You MUST specify the parameter -external-bin-dir at
>> /n/w10-nruiz/mosesdecoder/scripts/training/train-model.perl line 484.
>> Do you know why train-model.perl is having trouble finding the mgiza
>> executables? Thanks for your help!
>>
>> Thanks,
>> Nick
>>
>> z?n? ???u
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151124/68d41051/attachment-0001.html

------------------------------

Message: 3
Date: Tue, 24 Nov 2015 10:10:46 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Pruning phrase-table
To: Sanjanashree Palanivel <sanjanashree@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <565437A6.1030109@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"

Hi

The documentation suggests:

> A good setting is |-l a+e -n 30|

Try this first,

cheers - Barry

On 24/11/15 10:00, Sanjanashree Palanivel wrote:
>
> Dear Barry,
>
> Thank you for ur earnest response. I am using sigtest filter,
> now. In sigtest filter method, how should I tune a parameters -l -n.
>
> On Nov 24, 2015 2:44 PM, "Barry Haddow" <bhaddow@staffmail.ed.ac.uk
> <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>
> Hi
>
> You're better off using the Johnson pruning method
> http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc5 . The
> relent code is no longer maintained,
>
> cheers - Barry
>
> On 24/11/15 05:42, Sanjanashree Palanivel wrote:
>>
>> Dear All,
>>
>> I just tried to prune the phrase table using relent-filter
>> insisde mosesdecoder. I used the command as mentioned in the
>> moses site (http://www.statmt.org/moses/?n=Advanced.RuleTables).
>> But, I am getting the following error "Use of uninitialized value
>> $_[0] in substitution (s///) at
>> /usr/share/perl/5.18/File/Basename.pm line 341.
>> fileparse(): need a valid pathname at
>> /home/sanjana/Documents/SMT/mosesdecoder/contrib/relent-filter/scripts/calcPruningScores.pl
>> line 140." . Where am i being worng.
>>
>>
>> --
>> Thanks and regards,
>>
>> Sanjanasri J.P
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151124/d42a4394/attachment.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151124/d42a4394/attachment.pl

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 109, Issue 56
**********************************************

Tweet