Moses-support Digest, Vol 103, Issue 29

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Reading binarised model (Hamed Soleimani)
2. Re: Reading binarised model (Marcin Junczys-Dowmunt)
3. Re: Problem with qsub when running experiments (Carla Parra)


----------------------------------------------------------------------

Message: 1
Date: Fri, 08 May 2015 05:10:35 +0000
From: Hamed Soleimani <hamed1soleimani@gmail.com>
Subject: [Moses-support] Reading binarised model
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAMH7Ro=Z==5qwFbxGaVYGRLJAwsmvag46Ly61_QWeTWUp4K5ug@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello,
I've been following the Moses manual to create a Baseline System but I am
now stuck at the Testing section. I get this strange message as i tried to
read binarised table . Appreciate any help thanks.

Defined parameters (per moses.ini or switch):
config: /home/hamed1soleimani/working/binarised-model/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/hamed1soleimani/working/binarised-model/phrase-table
input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0
num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0
output-factor=0
path=/home/hamed1soleimani/working/binarised-model/reordering-table
Distortion KENLM lazyken=0 name=LM0 factor=0
path=/home/hamed1soleimani/lm/mizan.blm.en order=3
input-factors: 0
mapping: 0 T 0
threads: 4
weight: LexicalReordering0= 0.00461529 -0.00657824 0.0643977 0.0665969
0.0722116 0.117282 Distortion0= 0.0236595 LM0= 0.103723 WordPenalty0=
-0.337574 PhrasePenalty0= 0.0164844 TranslationModel0= 0.00832724 0.0814982
0.0505655 0.0464867 UnknownWordPenalty0= 1
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/hamed1soleimani/working/binarised-model/phrase-table
FeatureFunction: TranslationModel0 start: 3 end: 6
line=input-factor=0 output-factor=0
Exception: moses/FF/Factory.cpp:333 in void
Moses::FeatureRegistry::Construct(const string&, const string&) threw
UnknownFeatureException because `i == registry_.end()'.
Feature name input-factor=0 is not registered.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150508/51ffdb5a/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 08 May 2015 08:03:54 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Reading binarised model
To: moses-support@mit.edu
Message-ID: <554C51CA.6010501@amu.edu.pl>
Content-Type: text/plain; charset="windows-1252"

Hi,
check if you a have an unwanted newline-character in the line with the
phrase table feature of your ini file. Looks like it.
Best,
Marcin

W dniu 08.05.2015 o 07:10, Hamed Soleimani pisze:
> Hello,
> I've been following the Moses manual to create a Baseline System but
> I am now stuck at the Testing section. I get this strange message as i
> tried to read binarised table . Appreciate any help thanks.
>
> Defined parameters (per moses.ini or switch):
> config: /home/hamed1soleimani/working/binarised-model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/hamed1soleimani/working/binarised-model/phrase-table
> input-factor=0 output-factor=0 LexicalReordering
> name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/hamed1soleimani/working/binarised-model/reordering-table
> Distortion KENLM lazyken=0 name=LM0 factor=0
> path=/home/hamed1soleimani/lm/mizan.blm.en order=3
> input-factors: 0
> mapping: 0 T 0
> threads: 4
> weight: LexicalReordering0= 0.00461529 -0.00657824 0.0643977 0.0665969
> 0.0722116 0.117282 Distortion0= 0.0236595 LM0= 0.103723 WordPenalty0=
> -0.337574 PhrasePenalty0= 0.0164844 TranslationModel0= 0.00832724
> 0.0814982 0.0505655 0.0464867 UnknownWordPenalty0= 1
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/hamed1soleimani/working/binarised-model/phrase-table
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=input-factor=0 output-factor=0
> Exception: moses/FF/Factory.cpp:333 in void
> Moses::FeatureRegistry::Construct(const string&, const string&) threw
> UnknownFeatureException because `i == registry_.end()'.
> Feature name input-factor=0 is not registered.
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150508/95faea48/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 08 May 2015 12:57:07 +0200
From: Carla Parra <carla.parra@hermestrans.com>
Subject: Re: [Moses-support] Problem with qsub when running
experiments
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <9ee2fdfa796d677e86ff2bc0a50481cc@hermestrans.com>
Content-Type: text/plain; charset=UTF-8; format=flowed

Dear Barry,

thanks a lot for your help. In the end we have reinstalled everything in
the machine (even the OS) and it seems to be working (at least now
tuning is ongoing). I still have to run some other tests, but so far the
baseline examples from the website are running as expected, even when I
specify several clusters / threads.

I have no clue of what happened and I hope I don't have to bother you
again with these issues.

Best,
Carla


El 07.05.2015 11:39, Barry Haddow escribi?:
> Hi Carla
>
> I don't think your second error is qsub-related, you need to look at
> filterphrases.err? to see what is going on.
>
> For the EMS errors, I can't really see why it is detecting that you
> have a cluster. As a workaround, I would suggest that you comment out
> the call to detect_if_cluster() (line 44 in current master) in
> experiment.perl and see if that helps,
>
> cheers - Barry
>
> On 06/05/15 14:59, carla.parra@hermestrans.com wrote:
>
>> Hi again,
>>
>> this is the command I use:
>> "~/mosesdecoder/scripts/ems/experiment.perl -config config.toy
>> -exec"
>>
>> I have also tried a modification of the configuration file
>> "config.placeables". This is when the problem started. To make sure
>> that it was not the config file, I tried with an older config.file
>> from another experiment which finished without problems (just
>> changed the path for the experiment). I tried to run an older
>> experiment which worked and also failed this time with the same
>> error.
>>
>> I attach both files, so that you can see them.
>>
>> I don't know if it might help, but I also tried to run the
>> experiment step-by-step and at decoding I also got an error:
>>
>> /home/hermesta/mosesdecoder/scripts/training/mert-moses.pl
>>
> /home/hermesta/Exps/KES_newDev_placeholders/data/dev/KES10.dev.preproc.tok.true.en
>>
> /home/hermesta/Exps/KES_newDev_placeholders/data/dev/KES10.dev.preproc.tok.true.es
>> /home/hermesta/mosesdecoder/bin/moses
>>
> /home/hermesta/Exps/KES_newDev_placeholders/working/train/model/moses.ini
>> --mertdir /home/hermesta/mosesdecoder/bin/
>>
>> Using SCRIPTS_ROOTDIR: /home/hermesta/mosesdecoder/scripts
>> filtering the phrase tables... mi? may? 6 15:58:15 CEST 2015
>> exec:
>>
> /home/hermesta/mosesdecoder/scripts/training/filter-model-given-input.pl
>> ./filtered
>>
> /home/hermesta/Exps/KES_newDev_placeholders/working/train/model/moses.ini
>>
> /home/hermesta/Exps/KES_newDev_placeholders/data/dev/KES10.dev.preproc.tok.true.en
>>
>> Executing:
>>
> /home/hermesta/mosesdecoder/scripts/training/filter-model-given-input.pl
>> ./filtered
>>
> /home/hermesta/Exps/KES_newDev_placeholders/working/train/model/moses.ini
>>
> /home/hermesta/Exps/KES_newDev_placeholders/data/dev/KES10.dev.preproc.tok.true.en
>>> filterphrases.out 2> filterphrases.err
>> Exit code: 1
>> ERROR: Failed to run
>>
> '/home/hermesta/mosesdecoder/scripts/training/filter-model-given-input.pl
>> ./filtered
>>
> /home/hermesta/Exps/KES_newDev_placeholders/working/train/model/moses.ini
>>
> /home/hermesta/Exps/KES_newDev_placeholders/data/dev/KES10.dev.preproc.tok.true.en'.
>> at /home/hermesta/mosesdecoder/scripts/training/mert-moses.pl line
>> 1719.
>>
>> When checking line 1719 of mert-moses.pl, I realized it is also
>> related to qsub.
>>
>> Thank you so much!
>> Carla
>>
>> El 06.05.2015 15:27, Barry Haddow escribi?:
>> Hi Carla
>>
>> Not sure what's going on, and no reason why things should change
>> when
>> you installed asiya. Something else must have changed.
>>
>> Could you post your EMS config file, and the exact command you use
>> to run EMS?
>>
>> cheers - Barry
>>
>> On 06/05/15 13:32, carla.parra@hermestrans.com wrote:
>> Hi Barry,
>>
>> thanks for your prompt reply. If I am not wrong the name of the
>> server is "hermesta-Z10PE-D8-WS" (I have taken it from the machine
>> information, I attach a screenshot). If I should look somewhere else
>> please let me know.
>>
>> Thanks,
>> Carla
>>
>> El 06.05.2015 14:10, Barry Haddow escribi?:
>> Hi Carla
>>
>> What's your server called?
>>
>> There's a hard-coded list of Edinburgh machines in ems, so I'm
>> wondering if it collides with one of them,
>>
>> cheers - Barry
>>
>> On 06/05/15 12:56, carla.parra@hermestrans.com wrote:
>> Hi everyone,
>>
>> First of all, thanks for reading and hopefully giving me some
>> useful
>> pointer. I am running several SMT experiments on an Ubuntu machine.
>> It
>> is a multicore machine, but I have commented out the options for
>> running
>> experiments on multicore machines in the config.file.
>>
>> Up to last week, I was able to run experiments without problems.
>> However, since yesterday I get the error:
>>
>> "Can't exec "qsub": No existe el archivo o el directorio at
>> /home/hermesta/mosesdecoder/scripts/ems/experiment.perl line 1291."
>>
>>
>> Does anyone know what could be going on? The only thing I did was
>> installing asiya to assess MT output. My guess is that somehow
>> MOSES
>> detects that it is a multicore machine and tries to parallelize
>> jobs.
>> However, I don't understand why this was not happening last week,
>> for
>> instance. I have also tried to update MOSES by running "git pull"
>> and I
>> also reinstalled MOSES hoping this would fix the problem. My
>> background
>> is linguistics, and thus I am a bit lost now.
>>
>> Thank you very much,
>>
>> Carla Parra Escart?n
>> Marie Curie ER - EXPERT ITN
>> Hermes Traducciones
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support [1]
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]
>
>
>
> Links:
> ------
> [1] http://mailman.mit.edu/mailman/listinfo/moses-support
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.

--
Carla Parra Escart?n
Marie Curie Experienced Researcher - EXPERT ITN
http://expert-itn.eu/
Hermes Traducciones


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 103, Issue 29
**********************************************

Tweet