Moses-support Digest, Vol 122, Issue 8

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: too few factors error in mert (Sa?o Kuntaric)
2. Re: too few factors error in mert (Hasan Sait ARSLAN)
3. Re: too few factors error in mert (Matthias Huck)
4. Re: Regarding Decoding Time (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 6 Dec 2016 11:15:42 +0100
From: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Subject: Re: [Moses-support] too few factors error in mert
To: Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CANsquDr4tr3GN5Nwqx6Y2kbRrAvNrTWx406n0Qj=J+RdCk-TZQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Please see my reply to another thread below. I believe you need your source
part of the tuning set factored as well.

On 30/06/2016 21:44, Sa?o Kuntaric wrote:

Hi all,

I would like to ask one more question. When you say that my reference only
has the surface form, are you talking about the "tuning corpus", which in
the case of my command

~/mosesdecoder/scripts/training/mert-moses.pl ~/working/IT_corpus/TMX/txt/
factored_corpus/singles/tuning_corpus.tagged.clean.en
~/working/IT_corpus/TMX/txt/factored_corpus/singles/tuning
_corpus.tagged.clean.sl ~/mosesdecoder/bin/moses
~/working/IT_corpus/TMX/txt/factored_corpus/singles/test/model/moses.ini
--mertdir ~/mosesdecoder/bin/ --decoder-flags="-threads all"

are tuning_corpus.tagged.clean.en and tuning_corpus.tagged.clean.sl? Can
tuning be done with files that only contains surface forms?

it's usual that the reference tuning data does not have factors, even if
there are factors in the phrase table. After all, you don't care if the
output surface form is correct but the other factors are wrong.

Will the results be compatible with tuning done with a factored tuning
corpus?

yes

Best regards,

Sa?o

2016-12-06 10:18 GMT+01:00 Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>:

> Hi,
>
> I have a factored dataset. It involves 4 factors, factor1|factor2|factor3|factor4.
> I have trained my model with such a dataset.
>
> Now when I want to tune my model, I encounter with the following error:
>
>
>
>
> *Exception: moses/Word.cpp:159 in void
> Moses::Word::CreateFromString(Moses::FactorDirection, const
> std::vector<long unsigned int>&, const StringPiece&, bool, bool) threw
> util::Exception because `!isNonTerminal && i < factorOrder.size()'.Too few
> factors in string '-|-|Punc|Punc*
> The details of the error is in mert.txt file, which is attached to this
> e-mail.
>
> Thanks,
>
> Kind Regards,
> Hasan Sait Arslan
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
lp,

Sa?o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161206/e8ac5bfb/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 6 Dec 2016 12:27:00 +0200
From: Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
Subject: Re: [Moses-support] too few factors error in mert
To: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAJj7HD03cdkrHjaUAnS6sPyzFWJuk6e6JN7jQxpAxJjjFLiqzg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Yes, but my dev set is also factored. It is at the same format with
training set. There is no difference at all.

2016-12-06 12:15 GMT+02:00 Sa?o Kuntaric <saso.kuntaric@gmail.com>:

> Please see my reply to another thread below. I believe you need your
> source part of the tuning set factored as well.
>
> On 30/06/2016 21:44, Sa?o Kuntaric wrote:
>
> Hi all,
>
> I would like to ask one more question. When you say that my reference only
> has the surface form, are you talking about the "tuning corpus", which in
> the case of my command
>
> ~/mosesdecoder/scripts/training/mert-moses.pl
> ~/working/IT_corpus/TMX/txt/factored_corpus/singles/tuning_corpus.tagged.clean.en
> ~/working/IT_corpus/TMX/txt/factored_corpus/singles/tuning_
> corpus.tagged.clean.sl ~/mosesdecoder/bin/moses
> ~/working/IT_corpus/TMX/txt/factored_corpus/singles/test/model/moses.ini
> --mertdir ~/mosesdecoder/bin/ --decoder-flags="-threads all"
>
> are tuning_corpus.tagged.clean.en and tuning_corpus.tagged.clean.sl? Can
> tuning be done with files that only contains surface forms?
>
> it's usual that the reference tuning data does not have factors, even if
> there are factors in the phrase table. After all, you don't care if the
> output surface form is correct but the other factors are wrong.
>
> Will the results be compatible with tuning done with a factored tuning
> corpus?
>
> yes
>
> Best regards,
>
> Sa?o
>
> 2016-12-06 10:18 GMT+01:00 Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
> :
>
>> Hi,
>>
>> I have a factored dataset. It involves 4 factors,
>> factor1|factor2|factor3|factor4. I have trained my model with such a
>> dataset.
>>
>> Now when I want to tune my model, I encounter with the following error:
>>
>>
>>
>>
>> *Exception: moses/Word.cpp:159 in void
>> Moses::Word::CreateFromString(Moses::FactorDirection, const
>> std::vector<long unsigned int>&, const StringPiece&, bool, bool) threw
>> util::Exception because `!isNonTerminal && i < factorOrder.size()'.Too few
>> factors in string '-|-|Punc|Punc*
>> The details of the error is in mert.txt file, which is attached to this
>> e-mail.
>>
>> Thanks,
>>
>> Kind Regards,
>> Hasan Sait Arslan
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> lp,
>
> Sa?o
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161206/5eef7f9c/attachment-0001.html

------------------------------

Message: 3
Date: Tue, 06 Dec 2016 11:27:40 +0100
From: Matthias Huck <mhuck@cis.lmu.de>
Subject: Re: [Moses-support] too few factors error in mert
To: Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <1481020060.14951.48.camel@cis.lmu.de>
Content-Type: text/plain; charset="UTF-8"

Hi,

Maybe your moses.ini lets the decoder expect five input factors, wherea
s there are only four present in the data?

I see this in your log file:

input-factors: 0 1 2 3 4

Cheers,
Matthias


On Tue, 2016-12-06 at 11:18 +0200, Hasan Sait ARSLAN wrote:
> Hi,
>
> I have a factored dataset. It involves 4 factors,
> factor1|factor2|factor3|factor4. I have trained my model with such a
> dataset.
>
> Now when I want to tune my model, I encounter with the following error:
>
>
>
>
> *Exception: moses/Word.cpp:159 in void
> Moses::Word::CreateFromString(Moses::FactorDirection, const
> std::vector&, const StringPiece&, bool, bool) threw
> util::Exception because `!isNonTerminal && i < factorOrder.size()'.Too few
> factors in string '-|-|Punc|Punc*
> The details of the error is in mert.txt file, which is attached to this
> e-mail.
>
> Thanks,
>
> Kind Regards,
> Hasan Sait Arslan
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 4
Date: Tue, 6 Dec 2016 10:55:44 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Regarding Decoding Time
To: Shubham Khandelwal <skhlnmiit@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbiKJzw3+1TN5KzuQR2s9EtehV2VhyitzRHx=H=pXABnRw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

have a look at this page
http://www.statmt.org/moses/?n=Moses.Optimize

Hieu Hoang
http://www.hoang.co.uk/hieu

On 5 December 2016 at 14:15, Shubham Khandelwal <skhlnmiit@gmail.com> wrote:

> Hello,
>
> I have created one translation model (German to English) and converted it
> to Compact format.
> Now when I run the command:
> ~/mosesdecoder/bin/moses2 -f moses.ini
> Then, It loads the model very quickly but when I pass the German Text
> (around 4-5 lines) for the translation then it translates the text in 10
> minutes.
> Can you please tell me that how can I reduce this time for the translation
> ?
>
> Thanking You.
>
> Regards,
> Shubham
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161206/d72e35fc/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 122, Issue 8
*********************************************

Tweet