Moses-support Digest, Vol 166, Issue 5

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Failed to get the language model by KenLM (Chen, Y.)
2. Re: Failed to get the language model by KenLM (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 5 Aug 2020 21:08:04 +0200
From: "Chen, Y." <y.w.chen.1@student.rug.nl>
Subject: [Moses-support] Failed to get the language model by KenLM
To: moses-support@mit.edu
Message-ID:
<CAKzNZFTC_gu_DRLAuAP0tJLQ_OujE8iZ7=Ez609MhLODUMQcbg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

To whom it may concern,

I used Moses to train the language model by KenLM, however, I got the
following error and I could not get the final model. Could you help me with
this? Thank you in advance!

The command line I entered:
bzcat 40m_training_data.bz2 | python preprocess.py |
./mosesdecoder/bin/lmplz -o 3 > 40m_training_data.arpa

Error message I got at the first step of Counting and sorting n-grams (out
of 5 steps):
terminate called after throwing an instance of 'util::FDException'
what(): util/file.cc:220 in void util::WriteOrThrow(int, const void*,
std::size_t) threw FDException because `ret < 1'.
No space left on device in /tmp/lmWazsTd (deleted) while writing 732919964
bytes
Aborted

Best,
Yu-Wen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20200805/549afa26/attachment-0001.html

------------------------------

Message: 2
Date: Fri, 7 Aug 2020 19:04:06 -0700
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Failed to get the language model by KenLM
To: "Chen, Y." <y.w.chen.1@student.rug.nl>, moses-support@mit.edu
Message-ID: <169b5dcf-0cbd-8f64-f177-c8f52150a2f9@gmail.com>
Content-Type: text/plain; charset="utf-8"

use lmplz -T [path]

and set path to somewhere with lots of disk space

On Wed Aug 5,20 12:08 PM, Chen, Y. wrote:
> To whom it may concern,
>
> I used Moses to train the language model by KenLM, however,?I got the
> following?error and I could not get?the final model. Could you help me
> with this? Thank you in advance!
>
> The command line I entered:
> bzcat 40m_training_data.bz2 | python preprocess.py |
> ./mosesdecoder/bin/lmplz -o 3 > 40m_training_data.arpa
> Error message I got at the first step of Counting and sorting n-grams
> (out of 5 steps):
> terminate called after throwing an instance of 'util::FDException'
> ? what(): ?util/file.cc:220 in void util::WriteOrThrow(int, const
> void*, std::size_t) threw FDException because `ret < 1'.
> No space left on device in /tmp/lmWazsTd (deleted) while writing
> 732919964 bytes
> Aborted
>
> Best,
> ?Yu-Wen
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
http://statmt.org/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20200807/2088b2c0/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 166, Issue 5
*********************************************

Tweet