Moses-support Digest, Vol 83, Issue 24

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Reg:LM issues in factored model with EMS (karan sharma)
2. Fwd: Reg:LM issues in factored model with EMS (karan sharma)
3. Re: Reg:LM issues in factored model with EMS (Philipp Koehn)
4. Re: Reg: Tuning issues in running factored model (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Fri, 13 Sep 2013 01:57:47 +0530
From: karan sharma <karan.sharma.bond@gmail.com>
Subject: [Moses-support] Reg:LM issues in factored model with EMS
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAFc-37QLwb8ViEFLLCCLSA1tCvdqdRVOw=pzs8UMd488ZMPKKA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hey,
I am using EMS for factored model.I am getting this error "step
LM:toy:train crashed".

STDERR file:
one of required KneserNey count-of-counts is zero
error in discount estimator for order 1

Can anyone help with it?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130913/421876a9/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 13 Sep 2013 02:57:57 +0530
From: karan sharma <karan.sharma.bond@gmail.com>
Subject: [Moses-support] Fwd: Reg:LM issues in factored model with EMS
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAFc-37SZhJH0gq+SBGWPmHJPBTujCa=w14vYCgK7u4_0wytnmw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hey,
I am using EMS for factored model.I am getting this error "step
LM:toy:train crashed".

STDERR file:
one of required KneserNey count-of-counts is zero
error in discount estimator for order 1

Can anyone help with it?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130913/96909b66/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 13 Sep 2013 12:11:17 +0200
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Reg:LM issues in factored model with EMS
To: karan sharma <karan.sharma.bond@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAFADDC6xsQGcRE0mgxM4HfD-Zh9dwj9VCzrQddbHSf-1rLPvg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi,

this error is not that unusual when you train an n-gram model over part of
speech tags. These tags are all very frequent, which then causes Kneser-Ney
smoothing to fail.

You can address this by not using Kneser Ney smoothing for such models.
This can be done in EMS by adding the line

settings = "-interpolate -unk"

to the [LM:...] section where you define the n-gram model over tags.

-phi



On Thu, Sep 12, 2013 at 10:27 PM, karan sharma
<karan.sharma.bond@gmail.com>wrote:

>
> Hey,
> I am using EMS for factored model.I am getting this error "step
> LM:toy:train crashed".
>
> STDERR file:
> one of required KneserNey count-of-counts is zero
> error in discount estimator for order 1
>
> Can anyone help with it?
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130913/7e9dba63/attachment-0001.htm

------------------------------

Message: 4
Date: Fri, 13 Sep 2013 17:57:33 +0200
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Reg: Tuning issues in running factored
model
To: karan sharma <karan.sharma.bond@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDS-Djk1E_MqscTJBDWcqAHJq_9VQSe2BWtrvx0Qra5vQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi,

it seems that you did not compile Moses with SRILM support.
Absent of using KenLM instead, you can compile SRILM in
when adding the --with-srilm switch in the call to bjam.

-phi


On Wed, Sep 11, 2013 at 11:54 PM, karan sharma
<karan.sharma.bond@gmail.com>wrote:

> Hi,
> I am using factored based model in moses.
> My factors are word+POS both on source and target side.
> I am getting an error in tuning
>
> Error is :
> Exception: moses/FF/Factory.cpp:173 in void
> Moses::FeatureRegistry::Construct(const string&, const string&) threw
> UnknownFeatureException because `i == registry_.end()'.
>
> Feature name SRILM is not registered.
> Exit code: 1
> Failed to run moses with the config
> /home/working_dir/tuning/moses.filtered.ini
>
> Can someone please help to resolve it.
>
> Regards
>
>
> On Thu, Sep 12, 2013 at 3:21 AM, karan sharma <karan.sharma.bond@gmail.com
> > wrote:
>
>> Hi,
>> I am using factored based model in moses.
>> My factors are word+POS both on source and target side.
>> I am getting an error in tuning
>>
>> Error is :
>> Feature name SRILM is not registered.
>> Exit code: 1
>> Failed to run moses with the config
>> /home/working_dir/tuning/moses.filtered.ini
>>
>> Can someone please help to resolve it.
>>
>> Regards
>>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130913/523d7f98/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 83, Issue 24
*********************************************

Tweet