Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: max phrase length (Jorg Tiedemann)
2. Re: Moses compile help!!!! (Hieu Hoang)
3. Re: max phrase length (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Tue, 10 Feb 2015 13:19:39 +0100
From: Jorg Tiedemann <tiedeman@gmail.com>
Subject: Re: [Moses-support] max phrase length
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <8CE3E559-A6E4-4136-9518-A2F3D45C4967@gmail.com>
Content-Type: text/plain; charset="windows-1252"
Just to make sure that I understand the consequences for decoding correctly. Is it true that this only becomes a problem if the jump length in the lattice is larger than the max-phrase-length flag, isn't it?
J?rg
On Feb 6, 2015, at 5:03 PM, Hieu Hoang wrote:
> the decoder creates a list of all possible paths through the lattice up the the max-phrase-length before it does the pt lookups
>
> So if the lattice is large and the max-phrase-length is high, it will consume a lot of memory.
>
> in the old version of moses (v1), it only built the paths as it does the lookup and only keep the path if it finds rules for it. This is more memory efficient but the code was too convoluted
>
> On 06/02/15 12:43, Jorg Tiedemann wrote:
>> Hi,
>>
>> I have a question about the -max-phrase-length flag. I have lattice input that collects a lot of memory and eventually crashes when memory runs out (over 90GB). Now I realized that setting max-phrase-length helps even when setting it to length 7 which is the maximum length in my extracted phrase table. I do't really see why this is the case but maybe I misunderstand the flag and what it does. Is it not related to the max length of translation options coming from the phrase table?
>> Thanks for clarifying this to me.
>>
>> cheers,
>> J?rg
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> Hieu Hoang
> Research Associate (until March 2015)
> ** searching for interesting commercial MT position **
> University of Edinburgh
> http://www.hoang.co.uk/hieu
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150210/5ba45e6c/attachment-0001.htm
------------------------------
Message: 2
Date: Tue, 10 Feb 2015 13:20:26 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses compile help!!!!
To: James Johnson <jamesjohnson1097@gmail.com>, moses-support@mit.edu
Message-ID: <54DA059A.4030509@gmail.com>
Content-Type: text/plain; charset="windows-1252"
if this is your first time, please follow the instructions here:
http://www.statmt.org/moses/?n=Development.GetStarted
Specifically,
1. install boost, or compile it yourself.
2. Don't use SRILM
3. irstlm version has changed. There is a message at the top of your
moses.log
!!! You are linking with the IRSTLM library; be sure the
release is >= 5.80.07 !!!
Or even easier, use KenLM instead of IRSTLM. It comes with moses
so you don't need to download it separately
On 10/02/15 08:55, James Johnson wrote:
>
>
>
>
> Hi ,
>
> I am using _*ubuntu-14.04*_ machine and _boost version_
> *boost_1_55_0* and _irstlm-version_ *irstlm-5.80.06* and
> _srilm-version_*srilm-1.7.1* and the latest downloaded mosesdecoder
> Every time i try compiling moses , i am facing the same issue again
> and again . I have no clue or idea how to solve this . I have attached
> two log files , the first one would be bulid.log.gz file and the
> second is log of how it is executed . Please help me in solving this
> issue
>
>
>
>
>
>
>
> --
> @jamesjohnson
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150210/86f64d14/attachment-0001.htm
------------------------------
Message: 3
Date: Tue, 10 Feb 2015 13:25:29 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] max phrase length
To: Jorg Tiedemann <tiedeman@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhchkYEMyreUg_CZc1thcoa80L8zQ0oBORwMZdaXD5kXQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
For the memory issue, it's a problem if the max-phrase-length is too high.
The decoder will generate a list of all possible sub-paths up to
length max-phrase-length.
If there are many possible paths, then it consumes lots of memory.
Another issue is distortion limits. I'm not sure how distortion calculated
with lattice input.
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu
On 10 February 2015 at 12:19, Jorg Tiedemann <tiedeman@gmail.com> wrote:
>
> Just to make sure that I understand the consequences for decoding
> correctly. Is it true that this only becomes a problem if the jump length
> in the lattice is larger than the max-phrase-length flag, isn't it?
>
> J?rg
>
>
>
>
> On Feb 6, 2015, at 5:03 PM, Hieu Hoang wrote:
>
> the decoder creates a list of all possible paths through the lattice up
> the the max-phrase-length before it does the pt lookups
>
> So if the lattice is large and the max-phrase-length is high, it will
> consume a lot of memory.
>
> in the old version of moses (v1), it only built the paths as it does the
> lookup and only keep the path if it finds rules for it. This is more memory
> efficient but the code was too convoluted
>
> On 06/02/15 12:43, Jorg Tiedemann wrote:
>
> Hi,
>
> I have a question about the -max-phrase-length flag. I have lattice
> input that collects a lot of memory and eventually crashes when memory runs
> out (over 90GB). Now I realized that setting max-phrase-length helps even
> when setting it to length 7 which is the maximum length in my extracted
> phrase table. I do't really see why this is the case but maybe I
> misunderstand the flag and what it does. Is it not related to the max
> length of translation options coming from the phrase table?
> Thanks for clarifying this to me.
>
> cheers,
> J?rg
>
>
>
>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoang
> Research Associate (until March 2015)
> ** searching for interesting commercial MT position **
> University of Edinburghhttp://www.hoang.co.uk/hieu
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150210/e8e9cd10/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 100, Issue 37
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 100, Issue 37"
Post a Comment