Moses-support Digest, Vol 166, Issue 12

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Failed to get the language model by KenLM (Chen, Y.)
2. Re: Failed to get the language model by KenLM (Kenneth Heafield)


----------------------------------------------------------------------

Message: 1
Date: Sat, 8 Aug 2020 13:09:05 +0200
From: "Chen, Y." <y.w.chen.1@student.rug.nl>
Subject: Re: [Moses-support] Failed to get the language model by KenLM
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<CAKzNZFQmNU3WkDAV14Q55nVndC_LX7fWtCzrmh=kshFGJWVmKA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Hieu,

Thank you for your help! I solved this problem and built the language
model. But when I tried to binarize the arpa file with 'build_binary'
provided by KenLM, it showed the following error:

./bin/build_binary: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found
(required by ./bin/build_binary)
./bin/build_binary: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not
found (required by ./bin/build_binary)
./bin/build_binary: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not
found (required by ./bin/build_binary)

Can you help me with this?

Thank you very much!

Best,
Yu-Wen

On Sat, Aug 8, 2020 at 4:04 AM Hieu Hoang <hieuhoang@gmail.com> wrote:

> use lmplz -T [path]
>
> and set path to somewhere with lots of disk space
> On Wed Aug 5,20 12:08 PM, Chen, Y. wrote:
>
> To whom it may concern,
>
> I used Moses to train the language model by KenLM, however, I got the
> following error and I could not get the final model. Could you help me with
> this? Thank you in advance!
>
> The command line I entered:
> bzcat 40m_training_data.bz2 | python preprocess.py |
> ./mosesdecoder/bin/lmplz -o 3 > 40m_training_data.arpa
> Error message I got at the first step of Counting and sorting n-grams (out
> of 5 steps):
> terminate called after throwing an instance of 'util::FDException'
> what(): util/file.cc:220 in void util::WriteOrThrow(int, const void*,
> std::size_t) threw FDException because `ret < 1'.
> No space left on device in /tmp/lmWazsTd (deleted) while writing 732919964
> bytes
> Aborted
>
> Best,
> Yu-Wen
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> Hieu Hoanghttp://statmt.org/hieu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20200808/b07a73ad/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 25 Aug 2020 20:38:36 +0100
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Failed to get the language model by KenLM
To: <moses-support@mit.edu>
Message-ID: <f3c770a3-6bc7-2f1e-7345-7ca52d1f8277@kheafield.com>
Content-Type: text/plain; charset=utf-8

It appears you are trying to run on a machine with very different
libraries from the machine you compiled on. Don't do that. Compile on
the same machine.

On 8/8/20 12:09 PM, Chen, Y. wrote:
> Dear Hieu,?
>
> Thank you for your help! I solved this?problem and built the language
> model. But when I tried to binarize the arpa file with 'build_binary'
> provided by KenLM, it showed the following?error:?
>
> ./bin/build_binary: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not
> found (required by ./bin/build_binary)
> ./bin/build_binary: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not
> found (required by ./bin/build_binary)
> ./bin/build_binary: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not
> found (required by ./bin/build_binary)
>
> Can you help me with this??
>
> Thank you very?much!
>
> Best,?
> Yu-Wen
>
> On Sat, Aug 8, 2020 at 4:04 AM Hieu Hoang <hieuhoang@gmail.com
> <mailto:hieuhoang@gmail.com>> wrote:
>
> use lmplz -T [path]
>
> and set path to somewhere with lots of disk space
>
> On Wed Aug 5,20 12:08 PM, Chen, Y. wrote:
>> To whom it may concern,?
>>
>> I used Moses to train the language model by KenLM, however,?I got
>> the following?error and I could not get?the final model. Could you
>> help me with this? Thank you in advance!?
>>
>> The command line I entered:?
>> bzcat 40m_training_data.bz2 | python preprocess.py |
>> ./mosesdecoder/bin/lmplz -o 3 > 40m_training_data.arpa
>> Error message I got at the first step of Counting and sorting
>> n-grams?(out of 5 steps):
>> terminate called after throwing an instance of 'util::FDException'
>> ? what(): ?util/file.cc:220 in void util::WriteOrThrow(int, const
>> void*, std::size_t) threw FDException because `ret < 1'.
>> No space left on device in /tmp/lmWazsTd (deleted) while writing
>> 732919964 bytes
>> Aborted
>>
>> Best,
>> ?Yu-Wen
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> Hieu Hoang
> http://statmt.org/hieu
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 166, Issue 12
**********************************************

Tweet