Moses-support Digest, Vol 104, Issue 10

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: placeholder error with compact phrase-table
(Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Mon, 08 Jun 2015 18:01:44 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] placeholder error with compact
phrase-table
To: Vito Mandorino <vito.mandorino@linguacustodia.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <5575BC68.4080605@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed

OK, please send me the link on my e-mail address:
junczys@amu.edu.pl

I will try to take a look.

On 08.06.2015 14:39, Vito Mandorino wrote:
> Hi,
>
> sorry for late reply. I can by now share the phrase-table (262
> Mb) with you Marcin, thank you for proposing. If you agree I can send
> you a link for downloading it .
>
> The command which I used to compactify the phrase-table is
>
> /home/Moses/mosesdecoder/bin/processPhraseTableMin -in
> /home/vito/CommGest_fren/v4_bis/enfr/CARTOUCHES/cart_nb_2_1.ph/train/model/phrase-table.sorted
> <http://cart_nb_2_1.ph/train/model/phrase-table.sorted> -out
> /home/vito/CommGest_fren/v4_bis/enfr/MODELS/model_nb_l06_mu33_21_11_leclBis_.ph/mert-work/debug_placeholder/essaiForMarcin_3/phT_all.comp.1
> -nscores 4 -threads 24
>
>
> and the decoding command causing the error message
>
> Placeholder should be aligned to 1, and only 1, word
>
>
> is
>
> echo "usd <ne translation="@num@" entity="717">@num@</ne> mn worth
> of stocks" | /mosesdecoder/bin/moses -threads all -mp
> -placeholder-factor 1 -xml-input exclusive -f moses.ini
>
>
> Finally, the moses.ini file is
>
> #########################
> ### MOSES CONFIG FILE ###
> #########################
>
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
>
> [distortion-limit]
> 6
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/vito/CommGest_fren/v4_bis/enfr/MODELS/model_nb_l06_mu33_21_11_leclBis_.ph/mert-work/debug_placeholder/essaiForMarcin_3/phT_all.comp.1 input-factor=0
> output-factor=0 table-limit=20
> LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/vito/CommGest_fren/v4_bis/enfr/CARTOUCHES/cart_nb_2_1.ph/binarised-model/reordering-table
> <http://cart_nb_2_1.ph/binarised-model/reordering-table>
> Distortion
> KENLM lazyken=0 name=LM0 factor=0
> path=/home/vito/CommGest_fren/v4_bis/enfr/corpus/nb/corpus.lm.commGest_3T_3C_pros.ph.blm.fr.mm
> <http://corpus.lm.commGest_3T_3C_pros.ph.blm.fr.mm> order=5
>
> # dense weights for feature functions
> [weight]
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2
> LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3
> Distortion0= 0.3
> LM0= 0.5
>
>
>
> In my tests, the decoding yielded the error (on the sentence above)
> about half of the time that I run the processPhraseTableMin command,
> and there were no problems with plain or onDisk phrase-tables.
>
> Thank you and best regards,
>
> Vito Mandorino
>
> 2015-05-29 1:45 GMT+02:00 Marcin Junczys-Dowmunt <junczys@amu.edu.pl
> <mailto:junczys@amu.edu.pl>>:
>
> Hi,
> Oops, missed that post, should set-up some filter based on
> "compact". Looks like another alignment-based error in my pt. I
> never used place holders before, so I never came across this. Can
> you somehow share stuff so I can reproduce this?
> Best,
> Marcin
>
> W dniu 29.05.2015 o 01:40, Hieu Hoang pisze:
>>
>> The placeholders need word alignment info to work. What is the
>> exact command u used to binarise? Are you sure the text pt had
>> alignment info?
>>
>> On 27 May 2015 12:35, "Vito Mandorino"
>> <vito.mandorino@linguacustodia.com
>> <mailto:vito.mandorino@linguacustodia.com>> wrote:
>>
>> Dear all,
>>
>> I'm getting some troubles when using placeholders together
>> with compact phrase table.
>> If I decode the segment
>>
>> usd <ne translation="@num@" entity="717">@num@</ne> mn
>> worth of stocks
>>
>>
>> with compact phrase table I get the error
>>
>> Line 0: Search took 0.108 seconds
>> terminate called after throwing an instance of
>> 'util::Exception'
>> what(): moses/IOWrapper.cpp:273 in std::map<long
>> unsigned int, const Moses::Factor*>
>> Moses::IOWrapper::GetPlaceholders(const
>> Moses::Hypothesis&, Moses::FactorType) threw
>> util::Exception because `targetPos.size() != 1'.
>> Placeholder should be aligned to 1, and only 1, word
>>
>>
>>
>> four times out of 13 different compactifications of the
>> phrase table (the remaining 9 work fine). If I decode with
>> non-compact phrase-table everything works fine. What could be
>> the reason of the error? Is it possible that this is due to
>> small random information losses when creating the compact tables?
>>
>> Thank you and best regards,
>>
>> Vito Mandorino
>>
>> --
>> *M**. Vito MANDORINO -- Chief Scientist*
>>
>> Description : Description : lingua_custodia_final full logo
>>
>> */The Translation Trustee/*
>>
>> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>>
>> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
>> <tel:%2B33%206%2084%2065%2068%2089>*
>>
>> *Email :****vito.mandorino@linguacustodia.com
>> <mailto:massinissa.ahmim@linguacustodia.com>***
>>
>> *Website :****www.linguacustodia.com
>> <http://www.linguacustodia.com/> -
>> www.thetranslationtrustee.com
>> <http://www.thetranslationtrustee.com/>*
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> *M**. Vito MANDORINO -- Chief Scientist*
>
> Description : Description : lingua_custodia_final full logo
>
> */The Translation Trustee/*
>
> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89*
>
> *Email :****vito.mandorino@linguacustodia.com
> <mailto:massinissa.ahmim@linguacustodia.com>***
>
> *Website :****www.linguacustodia.com
> <http://www.linguacustodia.com/> - www.thetranslationtrustee.com
> <http://www.thetranslationtrustee.com/>*
>



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 104, Issue 10
**********************************************

Tweet