Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Generation step for factorized translation (Marco Damonte)
2. Problem in translation (fatma elzahraa Eltaher)
3. Re: Problem in translation (Barry Haddow)
4. Re: EMS help (Vincent Nguyen)
----------------------------------------------------------------------
Message: 1
Date: Tue, 28 Jul 2015 16:46:04 +0000
From: Marco Damonte <mdtux89@gmail.com>
Subject: [Moses-support] Generation step for factorized translation
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAD2JQQOCpokYgWX4S2kFouMEqehkk08nsPrwg5y6LyxFm9rZmQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
I'm using EMS and factorized translation. For instance, I have this setting:
input-factors = word
output-factors = word brown50
alignment-factors = "word -> word"
translation-factors = "word -> word+brown50"
generation-factors = "brown50"
reordering-factors = "word -> word"
decoding-steps = "t0"
that is, I have brown clusters as an output factor.
Does someone can point me the proper way to add a generation step to create
the surface word using the cluster factor?
I tried:
generation-factors = "brown50 -> word"
but the experiment crashed durinh tuning
Thanks in advance
Marco
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150728/36d221f9/attachment-0001.htm
------------------------------
Message: 2
Date: Tue, 28 Jul 2015 20:13:52 +0200
From: fatma elzahraa Eltaher <fatmaeltaher@gmail.com>
Subject: [Moses-support] Problem in translation
To: moses-support@mit.edu
Message-ID:
<CAOW1BbSQjBzpPJj2CKOOd0stpuq+4dfY2jMYL1NWCXtwZALWKw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear All,
I try to build a Model but I get an attached error file . is this mean that
there are a problem in model . Because I test it by word in training data
but the output was wrong.
kindly find the attached file.
thank you,
Fatma El-Zahraa El -Taher
Teaching Assistant at Computer & System department
Faculty of Engineering, Azhar University
Email : fatmaeltaher@gmail.com
mobile: +201141600434
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150728/c867b40f/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: filterphrases.err
Type: application/octet-stream
Size: 1257 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150728/c867b40f/attachment-0001.obj
------------------------------
Message: 3
Date: Tue, 28 Jul 2015 20:47:00 +0100
From: Barry Haddow <bhaddow@inf.ed.ac.uk>
Subject: Re: [Moses-support] Problem in translation
To: fatma elzahraa Eltaher <fatmaeltaher@gmail.com>,
moses-support@mit.edu
Message-ID: <55B7DC34.5040309@inf.ed.ac.uk>
Content-Type: text/plain; charset="iso-8859-1"
Hi Fatma
I don't see any error in the file. What do you mean "the output was
wrong." ?
cheers - Barry
On 28/07/15 19:13, fatma elzahraa Eltaher wrote:
> Dear All,
>
> I try to build a Model but I get an attached error file . is this mean
> that there are a problem in model . Because I test it by word in
> training data but the output was wrong.
>
> kindly find the attached file.
>
> thank you,
>
>
>
> Fatma El-Zahraa El -Taher
>
> Teaching Assistant at Computer & System department
>
> Faculty of Engineering, Azhar University
>
> Email : fatmaeltaher@gmail.com <mailto:fatmaeltaher@gmail.com>
> mobile: +201141600434
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150728/ded8e724/attachment-0001.htm
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150728/ded8e724/attachment-0001.bat
------------------------------
Message: 4
Date: Tue, 28 Jul 2015 22:36:47 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: Re: [Moses-support] EMS help
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>, moses-support
<moses-support@mit.edu>
Message-ID: <55B7E7DF.8040605@neuf.fr>
Content-Type: text/plain; charset="windows-1252"
Many thanks it seems to have fixed the issue, maybe worth to fix it on
github.
Le 28/07/2015 15:44, Barry Haddow a ?crit :
> Hi Vincent
>
>
> I think the quotes are getting stripped off further down the pipeline.
> You could work around by changing to the compact phrase table. Or try
> editing binarize-model.perl to change
>
> safesystem("$RealBin/filter-model-given-input.pl $targetdir
> $input_config /dev/null $hierarchical -nofilter -Binarizer
> $binarizer") || die "binarising failed";
>
> to
>
> safesystem("$RealBin/filter-model-given-input.pl $targetdir
> $input_config /dev/null $hierarchical -nofilter -Binarizer
> \"$binarizer\"") || die "binarising failed";
>
> Note the escaped quotes around the $binarizer.
>
> cheers - Barry
>
> On 28/07/15 14:09, Vincent Nguyen wrote:
>> same error:
>>
>> #!/bin/bash
>> PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"
>> cd /home/moses/working
>> echo 'starting at '`date`' on '`hostname`
>> mkdir -p /home/moses/working/training
>> mkdir -p /home/moses/working/model
>> /home/moses/mosesdecoder/scripts/training/binarize-model.perl
>> /home/moses/working/model/moses.ini.5
>> /home/moses/working/model/moses.bin.ini.6 -Binarizer
>> "/home/moses/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2"
>>
>> echo 'finished at '`date`
>> touch /home/moses/working/steps/6/TRAINING_binarize-config.6.DONE
>>
>>
>>
>>
>> Le 28/07/2015 14:47, Barry Haddow a ?crit :
>>> Hi Vincent
>>>
>>> It could be a bug. Could you edit
>>> mosesdecoder/scripts/ems/experiment.meta and change the line:
>>>
>>> template: $binarize-all IN OUT -Binarizer $ttable-binarizer
>>>
>>> to
>>>
>>> template: $binarize-all IN OUT -Binarizer "$ttable-binarizer"
>>>
>>> Note that I have added quotes. Then you'll have to delete the most
>>> recent run, and re-run experiment.perl. If it works, fine. If it
>>> doesn't, could you post the steps/6/TRAINING_binarize-config.6
>>> script (hopefully I got the name right - you may need to change the
>>> number)
>>>
>>> cheers - Barry
>>>
>>>
>>> On 28/07/15 13:11, Vincent Nguyen wrote:
>>>> I know but this is what I have in my config.basic now:
>>>> # conversion of rule table into binary on-disk format
>>>> ttable-binarizer = "$moses-bin-dir/CreateOnDiskPt 1 1 4 100 2"
>>>> binarize-all = $moses-script-dir/training/binarize-model.perl
>>>>
>>>> I don't where else I can add the 5 arguments or if I need to
>>>> reference ttable-binarizer somewhere
>>>>
>>>>
>>>> Le 28/07/2015 13:49, Barry Haddow a ?crit :
>>>>> Hi Vincent
>>>>>
>>>>> If you look at the error log, you will see:
>>>>>
>>>>>> Usage: /home/moses/mosesdecoder/bin/CreateOnDiskPt
>>>>>> numSourceFactors numTargetFactors numScores tableLimit
>>>>>> sortScoreIndex inputPath outputPath
>>>>> You are missing the first 5 arguments to CreateOnDiskPt, as given
>>>>> in config.basic.
>>>>>
>>>>> cheers - Barry
>>>>>
>>>>> On 28/07/15 12:37, Vincent Nguyen wrote:
>>>>>> I don't know why but the binarize crashes see below ....
>>>>>>
>>>>>>>
>>>>>>>> in my working directory I have 2 subdir,
>>>>>>>> "tuning" with inside moses.filtered.ini.5 moses.ini.5
>>>>>>>> moses.tuned.ini.5
>>>>>>>> and
>>>>>>>> "model" with inside moses.ini.5 (apparently this one does not
>>>>>>>> have the
>>>>>>>> tuned weights)
>>>>>>>>
>>>>>>>> those in the tuning subdir : the "tuned" one moses.tuned.ini.5
>>>>>>>> generated
>>>>>>>> after the moses.ini.5 seems to point on phrase-table.5.gz not
>>>>>>>> binarized
>>>>>>>> and the moses.5.ini seem to point on the binarized within
>>>>>>>> tuning/filtered.5/...
>>>>>>>> unclear to me on which one I should use.
>>>>>>> If you run EMS, there will be a filtered ini file inside the
>>>>>>> evaluation directory which can be used to translate the test set
>>>>>>> using the tuned weights. However this model is filtered for the
>>>>>>> test set, so you cannot use it on other sentences.
>>>>>>>
>>>>>>> If you want the full model binarised, then you should add:
>>>>>>>
>>>>>>> binarize-all = $moses-script-dir/training/binarize-model.perl
>>>>>>>
>>>>>>> to the [GENERAL] section of the EMS config and rerun EMS. In
>>>>>>> this case the moses.tuned.ini in tuning can be used to translate
>>>>>>> any sentences.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Executing:
>>>>>> /home/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>>> /home/moses/working/model/moses.bin.ini.6.tables
>>>>>> /home/moses/working/model/moses.ini.5 /dev/null -nofilter
>>>>>> -Binarizer /home/moses/mosesdecoder/bin/CreateOnDiskPt
>>>>>> Executing: mkdir -p /home/moses/working/model/moses.bin.ini.6.tables
>>>>>> Stripping XML...
>>>>>> Executing:
>>>>>> /home/moses/mosesdecoder/scripts/training/../generic/strip-xml.perl
>>>>>> < /dev/null >
>>>>>> /home/moses/working/model/moses.bin.ini.6.tables/input.34384
>>>>>> pt:PhraseDictionaryMemory name=TranslationModel0 num-features=4
>>>>>> path=/home/moses/working/model/phrase-table.5 input-factor=0
>>>>>> output-factor=0
>>>>>> Considering factor 0
>>>>>> ro:LexicalReordering name=LexicalReordering0 num-features=6
>>>>>> type=wbe-msd-bidirectional-fe-allff input-factor=0
>>>>>> output-factor=0
>>>>>> path=/home/moses/working/model/reordering-table.5.wbe-msd-bidirectional-fe.gz
>>>>>>
>>>>>> Considering factor 0
>>>>>> Filtering files...
>>>>>> filtering /home/moses/working/model/phrase-table.5 ->
>>>>>> /home/moses/working/model/moses.bin.ini.6.tables/phrase-table.0-0.1.1...
>>>>>>
>>>>>> Executing: ln -s /home/moses/working/model/phrase-table.5.gz
>>>>>> /home/moses/working/model/moses.bin.ini.6.tables/phrase-table.0-0.1.1.gz
>>>>>>
>>>>>> binarizing...
>>>>>> Executing: /home/moses/mosesdecoder/bin/CreateOnDiskPt
>>>>>> /home/moses/working/model/moses.bin.ini.6.tables/phrase-table.0-0.1.1.gz
>>>>>> /home/moses/working/model/moses.bin.ini.6.tables/phrase-table.0-0.1.1.bin
>>>>>>
>>>>>> Usage: /home/moses/mosesdecoder/bin/CreateOnDiskPt
>>>>>> numSourceFactors numTargetFactors numScores tableLimit
>>>>>> sortScoreIndex inputPath outputPath
>>>>>> Exit code: 1
>>>>>> Can't binarize at
>>>>>> /home/moses/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>>> line 417.
>>>>>> Exit code: 1
>>>>>> binarising failed at
>>>>>> /home/moses/mosesdecoder/scripts/training/binarize-model.perl
>>>>>> line 43.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150728/537e36f7/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 105, Issue 62
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 105, Issue 62"
Post a Comment