Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Moses Compilation error (Adel Khalifa)
2. Re: Support Moses and GPU on cloud (liling tan)
3. Eliminating load times for MOSES phrase table, language model
and reordering model (RR)
4. Re: Eliminating load times for MOSES phrase table, language
model and reordering model (Dingyuan Wang)
----------------------------------------------------------------------
Message: 1
Date: Tue, 4 Apr 2017 18:47:55 +0200
From: Adel Khalifa <adelkhalifa9@gmail.com>
Subject: [Moses-support] Moses Compilation error
To: Philipp Koehn <phi@jhu.edu>
Cc: moses-support@mit.edu
Message-ID:
<CAL3Vmkc+rGpsvnav_WX4wtNBkc3Q5Hb81FwCkD2MNcWEN2ey1g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hello,
When compiling moses I face an issue
./bjam --with-boost=/my_install_to/boost_1_63_0/
--with-srilm=/my_install_to/srilm --with-irstlm=/my_install_to/irstlm
--with-giza=/my_install_to/tools -j2
fatal: No names found, cannot describe anything.
ln: failed to create symbolic link ?CreateProbingPT2?: File exists
warning: No toolsets are configured.
warning: Configuring default toolset "gcc".
warning: If the default is wrong, your build may not work correctly.
warning: Use the "toolset=xxxxx" option to override our guess.
warning: For more configuration options, please consult
warning:
http://boost.org/boost-build2/doc/html/bbv2/advanced/configuration.html
NOT BUILDING MOSES SERVER!
Then I got this issue many time before stop
...failed gcc.link
"moses-cmd/bin/gcc-5.2.1/release/link-static/threading-multi/moses...
gcc.link
moses-cmd/bin/gcc-5.2.1/release/link-static/threading-multi/lmbrgrid"
/usr/bin/ld: cannot find -lirstlm
/usr/bin/ld: cannot find -loolm
/usr/bin/ld: cannot find -ldstruct
/usr/bin/ld: cannot find -lmisc
/usr/bin/ld: cannot find -lflm
collect2: error: ld returned 1 exit status
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170404/a3240920/attachment-0001.html
------------------------------
Message: 2
Date: Wed, 5 Apr 2017 16:42:52 +0800
From: liling tan <alvations@gmail.com>
Subject: Re: [Moses-support] Support Moses and GPU on cloud
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAKzPaJLjyx8aZ8_AiHuHs8dKSrLkGq0DwxPhEMABDtuaqJmvHA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Marcin and Moses community,
Thanks for the tips!
Yeah, g2.8xlarge is painfully expensive... Training on separate instances
sounds more reasonable. Now, I've to explain to the devs why I need 2
instances ;P
Regards,
Liling
On Tue, Apr 4, 2017 at 4:25 PM, liling tan <alvations@gmail.com> wrote:
> Dear Marcin and Moses community,
>
> Are you running on g2.8xlarge on AWS?
>
> I think I went to the cheap g2.2xlarge and 15GB RAM is a little too low
> for MGIZA++ , taking forever... I think I've got to recreate a new larger
> instance.
>
> Regards,
> Liling
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170405/286f2229/attachment-0001.html
------------------------------
Message: 3
Date: Wed, 5 Apr 2017 13:33:13 +0200
From: RR <rmijic@gmail.com>
Subject: [Moses-support] Eliminating load times for MOSES phrase
table, language model and reordering model
To: moses-support@mit.edu
Message-ID:
<CAHstjPiVwZyoqK9gYJpWf2ZxY0CuN2yZ-8eXtCdn+HN19dBWpw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
I am currently working on a system to explore MOSES decoding parameter
space - running MOSES in decoding mode with one parameter set, getting a
BLEU score and rerunning with different parameters.
More than 50% of my time is being spent repeatedly "loading" the phrase
table, language model and reordering model. For example,
$ ./example.fr | .../moses -f ./run11.moses.ini -threads 31 >
./trans_baseline
gives: "Start loading text phrase table. Moses format : [37.296] seconds."
37 seconds just to load a 590MB phrase table, even if it is on RAMDISK.
htop indicates that for those 37 seconds, a single core is 100% utilized.
My understanding is that some kind of optimized data structure (a hash
table of some kind?) is being created in those 37 seconds, which is then
lost and recreated when I re-run MOSES with a different value of a decoding
parameter.
If I want to eliminate this loading time, what is the best way forward? Is
there a phrase table format that avoids a significant loading time (i.e. a
500MB phrase table loads in < 1 sec)? Should I try to find a way to run a
single MOSES server but with different decoding parameters? I have a system
with 130GB RAM and I am using a small phrase table that has been filtered
for my decoding set.
Thanks in advance,
Roko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170405/f43edfb8/attachment-0001.html
------------------------------
Message: 4
Date: Wed, 5 Apr 2017 19:42:54 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: Re: [Moses-support] Eliminating load times for MOSES phrase
table, language model and reordering model
To: RR <rmijic@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAFt8H77jmnLq23EF5hGppW1wLLbF=JTjw4mvO17+OSCxg6qv7g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Try Compact Table? http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc3
2017?4?5? ??7:33?"RR" <rmijic@gmail.com>???
> Hi,
>
> I am currently working on a system to explore MOSES decoding parameter
> space - running MOSES in decoding mode with one parameter set, getting a
> BLEU score and rerunning with different parameters.
>
> More than 50% of my time is being spent repeatedly "loading" the phrase
> table, language model and reordering model. For example,
>
> $ ./example.fr | .../moses -f ./run11.moses.ini -threads 31 >
> ./trans_baseline
>
> gives: "Start loading text phrase table. Moses format : [37.296] seconds."
>
> 37 seconds just to load a 590MB phrase table, even if it is on RAMDISK.
> htop indicates that for those 37 seconds, a single core is 100% utilized.
>
> My understanding is that some kind of optimized data structure (a hash
> table of some kind?) is being created in those 37 seconds, which is then
> lost and recreated when I re-run MOSES with a different value of a decoding
> parameter.
>
> If I want to eliminate this loading time, what is the best way forward? Is
> there a phrase table format that avoids a significant loading time (i.e. a
> 500MB phrase table loads in < 1 sec)? Should I try to find a way to run a
> single MOSES server but with different decoding parameters? I have a system
> with 130GB RAM and I am using a small phrase table that has been filtered
> for my decoding set.
>
> Thanks in advance,
>
> Roko
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170405/3814782c/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 126, Issue 5
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 126, Issue 5"
Post a Comment