Moses-support Digest, Vol 127, Issue 19

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Error in language model training. (rmogla)
2. Re: Error in language model training. (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Mon, 15 May 2017 17:14:26 +0530
From: rmogla <rmogla01@gmail.com>
Subject: [Moses-support] Error in language model training.
To: moses-support@mit.edu
Message-ID:
<CAJUS1291Q5ZfiDQrfQ6A31ZrDcB8=1YqbgXP8gOCzswFxtDQJg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello,

While using KenLM for language model training for Hindi I got this error.


mosesdecoder/lm/builder/adjust_counts.cc:61 in void
lm::builder::{anonymous}::StatCollector::CalculateDiscounts(const
lm::builder::DiscountConfig&) threw BadDiscountException because
`discounts_[i].amount[j] < 0.0 || discounts_[i].amount[j] > j'.

ERROR: 1-gram discount out of range for adjusted count 2: -0.0625
Aborted (core dumped)


Please suggest how to remove this error.

Thanking in advance.
Best Regards
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170515/a8ad688d/attachment-0001.html

------------------------------

Message: 2
Date: Mon, 15 May 2017 11:19:41 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Error in language model training.
To: rmogla <rmogla01@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDCjzOkstbh4+O+6wU5YEmz8kSBvb+rVwBFF=i=dZXqu7Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

this is ac common error when the data is "odd", such as
heavily duplicated segments, reduced vocabulary, etc.

To avoid the error, just add the switch "--discount_fallback"
when invoking lmplz.

-phi

On Mon, May 15, 2017 at 7:44 AM, rmogla <rmogla01@gmail.com> wrote:

> Hello,
>
> While using KenLM for language model training for Hindi I got this error.
>
>
> mosesdecoder/lm/builder/adjust_counts.cc:61 in void
> lm::builder::{anonymous}::StatCollector::CalculateDiscounts(const
> lm::builder::DiscountConfig&) threw BadDiscountException because
> `discounts_[i].amount[j] < 0.0 || discounts_[i].amount[j] > j'.
>
> ERROR: 1-gram discount out of range for adjusted count 2: -0.0625
> Aborted (core dumped)
>
>
> Please suggest how to remove this error.
>
> Thanking in advance.
> Best Regards
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170515/333789e5/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 127, Issue 19
**********************************************

Tweet