Moses-support Digest, Vol 113, Issue 17

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: RNNLM Integration? (Marcin Junczys-Dowmunt)
2. Error in Running the Language Model Training of the KenLM
(BIRENDRA CHAUHAN SINGH)
3. Re: Error in Running the Language Model Training of the KenLM
(Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Mon, 7 Mar 2016 21:49:46 +0000
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] RNNLM Integration?
To: moses-support@mit.edu
Message-ID: <56DDF77A.3000806@amu.edu.pl>
Content-Type: text/plain; charset="windows-1252"

Something like this maybe?

http://www.statmt.org/wmt15/pdf/WMT34.pdf

W dniu 07.03.2016 o 21:41, Lane Schwartz pisze:
> Philipp,
>
> Are you aware of any published work examining the importance of
> hypothesis recombination in terms of time/space/quality tradeoffs?
>
> Lane
>
>
> On Mon, Mar 7, 2016 at 3:19 PM, Philipp Koehn <phi@jhu.edu
> <mailto:phi@jhu.edu>> wrote:
>
> Hi,
>
> integrating this into the decoder will break all hypothesis
> recombination, so it may be better (and definitely easier) to use
> the RNNLM to rerank n-best lists.
>
> -phi
>
> On Mon, Mar 7, 2016 at 3:46 PM, Jake Ballinger
> <ballingerj@allegheny.edu <mailto:ballingerj@allegheny.edu>> wrote:
>
> Hello everyone,
>
> Has anyone used an RNNLM language model instead of one of the
> recommended language models? I was specifically looking at the
> RNNLM toolkit provided by Tomas Mikolov at http://rnnlm.org/.
>
> Thank you!
>
> --
> Jake Ballinger
> Major: Computer Science
> Minors: Chinese, French, Spanish, & Math
> 443-974-6184 <tel:443-974-6184>
> ballingerj@allegheny.edu <mailto:ballingerj@allegheny.edu>
> Box 582
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> When a place gets crowded enough to require ID's, social collapse is not
> far away. It is time to go elsewhere. The best thing about space travel
> is that it made it possible to go elsewhere.
> -- R.A. Heinlein, "Time Enough For Love"
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160307/9e7e4334/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 8 Mar 2016 03:24:36 +0530
From: BIRENDRA CHAUHAN SINGH <birendrachauhan3@gmail.com>
Subject: [Moses-support] Error in Running the Language Model Training
of the KenLM
To: moses-support@mit.edu
Message-ID:
<CAKogDtz8eBMuCv7C3tkkLUwjjChHB-ZWSxmtVcp4rvhYaUEtnQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

on running this:

mkdir ~/lm
cd ~/lm
~/mosesdecoder/bin/lmplz -o 3
<~/corpus/news-commentary-v8.fr-en.true.en >
news-commentary-v8.fr-en.arpa.en

Error:


bhupendra@berry:~/lm$ ~/mosesdecoder/bin/lmplz -o 3
<~/corpus/news-commentary-v8.fr-en.true.en >
news-commentary-v8.fr-en.arpa.en
=== 1/5 Counting and sorting n-grams ===
Reading /home/bhupendra/corpus/news-commentary-v8.fr-en.true.en
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
util/scoped.cc:20 in void* util::{anonymous}::InspectAddr(void*,
std::size_t, const char*) threw MallocException because `!addr &&
requested'.
Cannot allocate memory for 2826111760 bytes in malloc
Aborted (core dumped)
bhupendra@berry:~/lm$ ~/mosesdecoder/bin/lmplz -o 3
<~/corpus/news-commentary-v8.fr-en.true.en >
news-commentary-v8.fr-en.arpa.en
=== 1/5 Counting and sorting n-grams ===
Reading /home/bhupendra/corpus/news-commentary-v8.fr-en.true.en
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
util/scoped.cc:20 in void* util::{anonymous}::InspectAddr(void*,
std::size_t, const char*) threw MallocException because `!addr &&
requested'.
Cannot allocate memory for 2826111760 bytes in malloc
Aborted (core dumped)

Can you please help me out.I am haivng 4GB Ram now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160307/78c4851b/attachment-0001.html

------------------------------

Message: 3
Date: Mon, 7 Mar 2016 21:56:42 +0000
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Error in Running the Language Model
Training of the KenLM
To: moses-support@mit.edu
Message-ID: <56DDF91A.9090909@amu.edu.pl>
Content-Type: text/plain; charset="windows-1252"

Try -S 50% or even smaller. But 4GB is awfully small.

W dniu 07.03.2016 o 21:54, BIRENDRA CHAUHAN SINGH pisze:
> on running this:
> mkdir ~/lm
> cd ~/lm
> ~/mosesdecoder/bin/lmplz -o 3 <~/corpus/news-commentary-v8.fr-en.true.en > news-commentary-v8.fr-en.arpa.en
> Error:
> bhupendra@berry:~/lm$ ~/mosesdecoder/bin/lmplz -o 3
> <~/corpus/news-commentary-v8.fr-en.true.en >
> news-commentary-v8.fr-en.arpa.en === 1/5 Counting and sorting n-grams
> === Reading /home/bhupendra/corpus/news-commentary-v8.fr-en.true.en
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> ****************************************************************************************************
> util/scoped.cc:20 in void* util::{anonymous}::InspectAddr(void*,
> std::size_t, const char*) threw MallocException because `!addr &&
> requested'. Cannot allocate memory for 2826111760 bytes in malloc
> Aborted (core dumped) bhupendra@berry:~/lm$ ~/mosesdecoder/bin/lmplz
> -o 3 <~/corpus/news-commentary-v8.fr-en.true.en >
> news-commentary-v8.fr-en.arpa.en === 1/5 Counting and sorting n-grams
> === Reading /home/bhupendra/corpus/news-commentary-v8.fr-en.true.en
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> ****************************************************************************************************
> util/scoped.cc:20 in void* util::{anonymous}::InspectAddr(void*,
> std::size_t, const char*) threw MallocException because `!addr &&
> requested'. Cannot allocate memory for 2826111760 bytes in malloc
> Aborted (core dumped)
> Can you please help me out.I am haivng 4GB Ram now.
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160307/ef96fd49/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 113, Issue 17
**********************************************

Tweet