Moses-support Digest, Vol 105, Issue 28

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Strange behavior of build-lm.sh (Hieu Hoang)
2. Re: Strange behavior of build-lm.sh (Hamed Soleimani)
3. Re: Strange behavior of build-lm.sh (Hieu Hoang)
4. Moses build failed (Manuela Sanguinetti)


----------------------------------------------------------------------

Message: 1
Date: Tue, 14 Jul 2015 20:49:39 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Strange behavior of build-lm.sh
To: Hamed Soleimani <hamed1soleimani@gmail.com>,
"moses-support@mit.edu" <moses-support@mit.edu>,
"user-irstlm@list.fbk.eu" <user-irstlm@list.fbk.eu>
Message-ID: <55A53DA3.3050500@gmail.com>
Content-Type: text/plain; charset="windows-1252"

you may have to create the temp directory
~/NLP/lm/v1.1/tmp
yourself before you run the script


On 12/07/2015 18:21, Hamed Soleimani wrote:
> Hi,
> Already it was working well but now every time i try it it gives me
> these:
>
> ~/irstlm/bin/build-lm.sh -i ~/NLP/lm/v1.1/mizan.final.sb.en -t
> ~/NLP/lm/v1.1/tmp -p -s improved-kneser-ney -o
> ~/NLP/lm/v1.1/mizan.final.lm.en
> Temporary directory /home/hamed1soleimani/NLP/lm/v1.1/tmp does not exist
> creating /home/hamed1soleimani/NLP/lm/v1.1/tmp
> Extracting dictionary from training corpus
> Splitting dictionary into 3 lists
> Extracting n-gram statistics for each word list
> Important: dictionary must be ordered according to order of appearance
> of words in data
> used to generate n-gram blocks, so that sub language model blocks
> results ordered too
> Extracting n-gram statistics for dict.*
> $bin/ngt -i="$inpfile" -n=$order -gooout=y -o="$gzip -c >
> $tmpdir/ngram.${sdict}.gz" -fd="$tmpdir/$sdict" $dictionary
> -iknstat="$tmpdir/ikn.stat.$sdict" >> $logfile 2>&1
> Estimating language models for each word list
> ls: cannot access /home/hamed1soleimani/NLP/lm/v1.1/tmp/dict.*: No
> such file or directory
> Merging language models into
> /home/hamed1soleimani/NLP/lm/v1.1/mizan.final.lm.en
> Cleaning temporary directory /home/hamed1soleimani/NLP/lm/v1.1/tmp
> Removing temporary directory /home/hamed1soleimani/NLP/lm/v1.1/tmp
>
> Best Regards,
> Hamed
>
> --
>
> Hamed Soleimani;
> Member of Council; http://www.theinternetofthings.eu/hamed-soleimani
> Researcher at AISP research lab at University of Isfahan
> Instructor at Isfahan Mathematics House
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150714/dd0e083c/attachment-0001.htm

------------------------------

Message: 2
Date: Tue, 14 Jul 2015 18:44:05 +0000
From: Hamed Soleimani <hamed1soleimani@gmail.com>
Subject: Re: [Moses-support] Strange behavior of build-lm.sh
To: Hieu Hoang <hieuhoang@gmail.com>, "moses-support@mit.edu"
<moses-support@mit.edu>, "user-irstlm@list.fbk.eu"
<user-irstlm@list.fbk.eu>
Message-ID:
<CAMH7Rom1_01SQ9M_hC_04EcP7DUasLa3d0kxnpj_zGMML-vN7Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

I have tested it before but It didn't work.

On Tue, Jul 14, 2015, 21:19 Hieu Hoang <hieuhoang@gmail.com> wrote:

> you may have to create the temp directory
> ~/NLP/lm/v1.1/tmp
> yourself before you run the script
>
>
>
> On 12/07/2015 18:21, Hamed Soleimani wrote:
>
> Hi,
> Already it was working well but now every time i try it it gives me these:
>
> ~/irstlm/bin/build-lm.sh -i ~/NLP/lm/v1.1/mizan.final.sb.en -t
> ~/NLP/lm/v1.1/tmp -p -s improved-kneser-ney -o
> ~/NLP/lm/v1.1/mizan.final.lm.en
> Temporary directory /home/hamed1soleimani/NLP/lm/v1.1/tmp does not exist
> creating /home/hamed1soleimani/NLP/lm/v1.1/tmp
> Extracting dictionary from training corpus
> Splitting dictionary into 3 lists
> Extracting n-gram statistics for each word list
> Important: dictionary must be ordered according to order of appearance of
> words in data
> used to generate n-gram blocks, so that sub language model blocks results
> ordered too
> Extracting n-gram statistics for dict.*
> $bin/ngt -i="$inpfile" -n=$order -gooout=y -o="$gzip -c >
> $tmpdir/ngram.${sdict}.gz" -fd="$tmpdir/$sdict" $dictionary
> -iknstat="$tmpdir/ikn.stat.$sdict" >> $logfile 2>&1
> Estimating language models for each word list
> ls: cannot access /home/hamed1soleimani/NLP/lm/v1.1/tmp/dict.*: No such
> file or directory
> Merging language models into
> /home/hamed1soleimani/NLP/lm/v1.1/mizan.final.lm.en
> Cleaning temporary directory /home/hamed1soleimani/NLP/lm/v1.1/tmp
> Removing temporary directory /home/hamed1soleimani/NLP/lm/v1.1/tmp
>
> Best Regards,
> Hamed
>
> --
>
> Hamed Soleimani;
> Member of Council; http://www.theinternetofthings.eu/hamed-soleimani
> Researcher at AISP research lab at University of Isfahan
> Instructor at Isfahan Mathematics House
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoang
> Researcher
> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>
> --

Hamed Soleimani;
Member of Council; http://www.theinternetofthings.eu/hamed-soleimani
Researcher at AISP research lab at University of Isfahan
Instructor at Isfahan Mathematics House
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150714/3f2a6699/attachment-0001.htm

------------------------------

Message: 3
Date: Tue, 14 Jul 2015 23:46:56 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Strange behavior of build-lm.sh
To: Hamed Soleimani <hamed1soleimani@gmail.com>,
"moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <55A56730.50604@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

hmm. I'm not quite sure what's wrong with it.

If all you want to do it create a language model and you don't care how
it's created, you might want to use kenlm instead. The program you need
is called lmplz, described here
https://kheafield.com/code/kenlm/estimation/
kenlm is included with moses so you don't have to download and compile
it separately

On 14/07/2015 22:44, Hamed Soleimani wrote:
> m blocks, so that sub language model blocks results ordered too
> Extracting n-gram statistics for di

--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu



------------------------------

Message: 4
Date: Wed, 15 Jul 2015 13:19:15 +0200
From: Manuela Sanguinetti <msanguin@di.unito.it>
Subject: [Moses-support] Moses build failed
To: moses-support@mit.edu
Message-ID: <55A641B3.90300@di.unito.it>
Content-Type: text/plain; charset="iso-8859-15"

Hi,

I'm experiencing some problems in compiling Moses on Windows using
Cygwin. I manually installed Boost (v. 1.58), but when I run the command

~/mosesdecoder
$ ./bjam --with-boost=~/boost_1_58_0/ -j4

the build fails.
I'm attaching the 'build.log.gz' file, as suggested in the error
message, hoping that someone could help me figuring out what the problem
is and how I can fix it.


Thank you in advance for any hint you may provide,
MS
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.log.gz
Type: application/x-gzip
Size: 222 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150715/102dbdf7/attachment.bin

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 105, Issue 28
**********************************************

Tweet