Home » Uncategories » Moses-support Digest, Vol 104, Issue 9

Moses-support Digest, Vol 104, Issue 9

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: kbmira error (Rico Sennrich)
2. Re: placeholder error with compact phrase-table (Vito Mandorino)


----------------------------------------------------------------------

Message: 1
Date: Mon, 8 Jun 2015 09:38:29 +0000 (UTC)
From: Rico Sennrich <rico.sennrich@gmx.ch>
Subject: Re: [Moses-support] kbmira error
To: moses-support@mit.edu
Message-ID: <loom.20150608T113552-324@post.gmane.org>
Content-Type: text/plain; charset=utf-8

Hieu Hoang <hieuhoang@...> writes:

>
>
>
> Hi All
> Does anyone know why I get this error? ?? # $MOSES_DIR/bin/kbmira?
--dense-init run3.dense --sparse-init run3.sparse-weights? --ffile
run1.features.dat --ffile run2.features.dat --ffile run3.features.dat
--scfile run1.scores.dat --scfile run2.scores.dat --scfile run3.scores.dat
-o mert.out....Found 401 initial sparse features.......Error: Feature file 2
ended prematurely


Hi Hieu,

grep FEATURES_TXT_BEGIN run1.features.dat | wc -l
751
FEATURES_TXT_BEGIN run2.features.dat | wc -l
751
grep FEATURES_TXT_BEGIN run3.features.dat | wc -l
750

is it possible that run3.features.dat got corrupted somehow? Sentence 147
seems to be missing.

best wishes,
Rico



------------------------------

Message: 2
Date: Mon, 8 Jun 2015 14:39:36 +0200
From: Vito Mandorino <vito.mandorino@linguacustodia.com>
Subject: Re: [Moses-support] placeholder error with compact
phrase-table
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CA+8mSmE51DNzZ8Sk-WxKZsLifx_tx+0gdGKRkRY4o2uz8849RQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

sorry for late reply. I can by now share the phrase-table (262 Mb) with you
Marcin, thank you for proposing. If you agree I can send you a link for
downloading it .

The command which I used to compactify the phrase-table is

/home/Moses/mosesdecoder/bin/processPhraseTableMin -in
/home/vito/CommGest_fren/v4_bis/enfr/CARTOUCHES/
cart_nb_2_1.ph/train/model/phrase-table.sorted -out
/home/vito/CommGest_fren/v4_bis/enfr/MODELS/model_nb_l06_mu33_21_11_leclBis_.ph/mert-work/debug_placeholder/essaiForMarcin_3/phT_all.comp.1
-nscores 4 -threads 24


and the decoding command causing the error message

Placeholder should be aligned to 1, and only 1, word


is

echo "usd <ne translation="@num@" entity="717">@num@</ne> mn worth of
stocks" | /mosesdecoder/bin/moses -threads all -mp -placeholder-factor 1
-xml-input exclusive -f moses.ini


Finally, the moses.ini file is

#########################
### MOSES CONFIG FILE ###
#########################

# input factors
[input-factors]
0

# mapping steps
[mapping]
0 T 0

[distortion-limit]
6

# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/vito/CommGest_fren/v4_bis/enfr/MODELS/model_nb_l06_mu33_21_11_leclBis_.ph/mert-work/debug_placeholder/essaiForMarcin_3/phT_all.comp.1
input-factor=0
output-factor=0 table-limit=20
LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/vito/CommGest_fren/v4_bis/enfr/CARTOUCHES/
cart_nb_2_1.ph/binarised-model/reordering-table
Distortion
KENLM lazyken=0 name=LM0 factor=0
path=/home/vito/CommGest_fren/v4_bis/enfr/corpus/nb/
corpus.lm.commGest_3T_3C_pros.ph.blm.fr.mm order=5

# dense weights for feature functions
[weight]
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3
Distortion0= 0.3
LM0= 0.5



In my tests, the decoding yielded the error (on the sentence above) about
half of the time that I run the processPhraseTableMin command, and there
were no problems with plain or onDisk phrase-tables.

Thank you and best regards,

Vito Mandorino

2015-05-29 1:45 GMT+02:00 Marcin Junczys-Dowmunt <junczys@amu.edu.pl>:

> Hi,
> Oops, missed that post, should set-up some filter based on "compact".
> Looks like another alignment-based error in my pt. I never used place
> holders before, so I never came across this. Can you somehow share stuff so
> I can reproduce this?
> Best,
> Marcin
>
> W dniu 29.05.2015 o 01:40, Hieu Hoang pisze:
>
> The placeholders need word alignment info to work. What is the exact
> command u used to binarise? Are you sure the text pt had alignment info?
> On 27 May 2015 12:35, "Vito Mandorino" <vito.mandorino@linguacustodia.com>
> wrote:
>
>> Dear all,
>>
>> I'm getting some troubles when using placeholders together with compact
>> phrase table.
>> If I decode the segment
>>
>> usd <ne translation="@num@" entity="717">@num@</ne> mn worth of stocks
>>
>>
>> with compact phrase table I get the error
>>
>> Line 0: Search took 0.108 seconds
>> terminate called after throwing an instance of 'util::Exception'
>> what(): moses/IOWrapper.cpp:273 in std::map<long unsigned int, const
>> Moses::Factor*> Moses::IOWrapper::GetPlaceholders(const Moses::Hypothesis&,
>> Moses::FactorType) threw util::Exception because `targetPos.size() != 1'.
>> Placeholder should be aligned to 1, and only 1, word
>>
>>
>>
>> four times out of 13 different compactifications of the phrase table (the
>> remaining 9 work fine). If I decode with non-compact phrase-table
>> everything works fine. What could be the reason of the error? Is it
>> possible that this is due to small random information losses when creating
>> the compact tables?
>>
>> Thank you and best regards,
>>
>> Vito Mandorino
>>
>> --
>> *M**. Vito MANDORINO -- Chief Scientist*
>>
>>
>> [image: Description : Description : lingua_custodia_final full logo]
>>
>> *The Translation Trustee*
>>
>> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>>
>> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
>> <%2B33%206%2084%2065%2068%2089>*
>>
>> *Email :* *vito.mandorino@linguacustodia.com
>> <massinissa.ahmim@linguacustodia.com>*
>>
>> *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
>> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>*
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
*M**. Vito MANDORINO -- Chief Scientist*


[image: Description : Description : lingua_custodia_final full logo]

*The Translation Trustee*

*1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*

*Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89*

*Email :* *vito.mandorino@linguacustodia.com
<massinissa.ahmim@linguacustodia.com>*

*Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150608/d2242bf4/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150608/d2242bf4/attachment.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150608/d2242bf4/attachment-0001.jpg

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 104, Issue 9
*********************************************

0 Response to "Moses-support Digest, Vol 104, Issue 9"

Post a Comment

Subscribe to: Post Comments (Atom)