Moses-support Digest, Vol 135, Issue 10

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Error on Moses (Redentor Periabras)
2. Pointer to previous research (Tom Hoar)
3. Moses 4 binaries for macOS (Tom Hoar)
4. Re: Moses 4 binaries for macOS (Kenneth Heafield)


----------------------------------------------------------------------

Message: 1
Date: Fri, 12 Jan 2018 10:48:57 +0000
From: Redentor Periabras <redperiabras@gmail.com>
Subject: Re: [Moses-support] Error on Moses
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<CAOsE3uteLPiVX9VWgf1dXQjmmbdzX8npbQEMxvrQu=Wy469wnA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Im using Ubuntu 17.10, thank you for your response.

On Thu, Jan 11, 2018 at 5:27 PM Hieu Hoang <hieuhoang@gmail.com> wrote:

> You can probably ignore that error.
>
> There seems to be a problem with a unit test but that may be spurious.
> What OS/distribution are you running?
>
> On 11/01/18 08:15, Redentor Periabras wrote:
>
> Hi, I got this error as I run:
>
> *`./bjam --with-boost=~/Projects/mosesdecoder/boost_1_64_0 -j8`*
>
> Your response is very much appreciated. Thank you!
>
>
> --
> *Redentor Periabras*
> BS Computer Science
> Department of Computer Science
> College of Computer and Information Sciences
> Polytechnic University of the Philippines, Manila
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoanghttp://moses-smt.org/
>
>

--
*Redentor Periabras*
BS Computer Science
Department of Computer Science
College of Computer and Information Sciences
Polytechnic University of the Philippines, Manila
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180112/bb4413d3/attachment-0001.html

------------------------------

Message: 2
Date: Fri, 12 Jan 2018 21:23:52 +0700
From: Tom Hoar <tahoar@slate.rocks>
Subject: [Moses-support] Pointer to previous research
To: moses-support@mit.edu
Message-ID: <C1A868D9-0AEF-46FD-98A6-45911450A196@slate.rocks>
Content-Type: text/plain; charset=us-ascii

Can anyone point me to previous research that used sub-sentence parallel aligned corpus for the translation model and target language paragraphs as the language model corpus?




------------------------------

Message: 3
Date: Fri, 12 Jan 2018 22:08:51 +0700
From: Tom Hoar <tahoar@slate.rocks>
Subject: [Moses-support] Moses 4 binaries for macOS
To: moses-support@mit.edu
Message-ID: <7e644bcc-db3e-5936-9f6c-6ff4a2c77d39@slate.rocks>
Content-Type: text/plain; charset="utf-8"

I'm testing the Release 4.0 (5th Oct, 2017) binaries for macOS
(macOS.zip) on a Mac High Sierra (10.13) virtual machine from
TechsViewer
(https://techsviewer.com/install-macos-high-sierra-virtualbox-windows/).
I'm using the homebrew version of GNU sort/split utilities. The host
machine is an 8 virtual core i7 4 Ghz with 32GB RAM and 500GB SSD. The
virtual guest is configured with 24 GB RAM and six cores.

I've found two problems and I hope someone can help.

1. train-model.perl works all of the way through. The sort/split
utilities work fine. The problem is mgiza in step 2 took almost 3
hours to complete with a small 15,000 segment parallel corpus. Step
2 using the same corpus on the native Windows host takes less than
10 minutes. That step with the Ubuntu 16.04 binaries running on
Windows Subsystem for Linux (WSL) takes 9 1/2 minutes.

Observing the Mac's Activity Monitor during step 2 shows that mgiza
was running 6 threads, but the total CPU load was only 115%. All the
other macOS binaries ran at a comparable speed to the other
platforms' binaries. for example, when the moses binary runs with 6
threads, the monitor shows it's consuming nearly 550%.

Last year, we cross-compiled our own macOS binaries on Linux host
using circa Moses 3 source. We experienced the same slow performance
with mgiza binary on an old Mac notebook hardware. At the time, we
wrote delayed work because we thought slow performance was due to
the cross-compile settings and/or the slow hardware. After slow
results with the Moses4 mgiza binary, I re-cross-compiled the mgiza
binary on our system and it was also dead slow on the 10.13 guest
taking almost 3 hours.

These tests were run on a very fast host machine with a virtual
guest. Using Hieu's binaries or mine seems to run against those
assumptions that hardware/cross-compiler are totally at fault. Has
anyone else experienced this kind of slow performance?

2. The lmplz binary fails on this host with this message. Was this not
compiled with static links for a reason? Where can I find the
necessary libraries?
$ ./lmplz
dyld: Library not loaded:
/User/hieu/workspace/xmlrpc-c/xmlrpc-c-1.39.07/lib/libxmlrpc_xmltok.3.39.dylib
? Referenced from: /opt/macOS/bin/./lmplz
? Reason: image not found
Abort trap: 6


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180112/61d42eef/attachment-0001.html

------------------------------

Message: 4
Date: Fri, 12 Jan 2018 15:26:14 +0000
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Moses 4 binaries for macOS
To: moses-support@mit.edu
Message-ID: <1e789669-7158-e110-6fa9-0b5a12dc3b56@kheafield.com>
Content-Type: text/plain; charset=utf-8

Hi Tom.

lmplz doesn't need libxmlrpc_xmltok. Looks like a case of
over-aggressive dependencies, resulting in a binary that needs a library
it doesn't use.

One could install xmlrpc-c (a third-party library used by Moses server)
in the same path. Or I guess substitute lmplz from the kenlm repo which
won't have this dependency and which should compile fine on OS X.

Kenneth

On 01/12/2018 03:08 PM, Tom Hoar wrote:
> I'm testing the Release 4.0 (5th Oct, 2017) binaries for macOS
> (macOS.zip) on a Mac High Sierra (10.13) virtual machine from
> TechsViewer
> (https://techsviewer.com/install-macos-high-sierra-virtualbox-windows/).
> I'm using the homebrew version of GNU sort/split utilities. The host
> machine is an 8 virtual core i7 4 Ghz with 32GB RAM and 500GB SSD. The
> virtual guest is configured with 24 GB RAM and six cores.
>
> I've found two problems and I hope someone can help.
>
> 1. train-model.perl works all of the way through. The sort/split
> utilities work fine. The problem is mgiza in step 2 took almost 3
> hours to complete with a small 15,000 segment parallel corpus. Step
> 2 using the same corpus on the native Windows host takes less than
> 10 minutes. That step with the Ubuntu 16.04 binaries running on
> Windows Subsystem for Linux (WSL) takes 9 1/2 minutes.
>
> Observing the Mac's Activity Monitor during step 2 shows that mgiza
> was running 6 threads, but the total CPU load was only 115%. All the
> other macOS binaries ran at a comparable speed to the other
> platforms' binaries. for example, when the moses binary runs with 6
> threads, the monitor shows it's consuming nearly 550%.
>
> Last year, we cross-compiled our own macOS binaries on Linux host
> using circa Moses 3 source. We experienced the same slow performance
> with mgiza binary on an old Mac notebook hardware. At the time, we
> wrote delayed work because we thought slow performance was due to
> the cross-compile settings and/or the slow hardware. After slow
> results with the Moses4 mgiza binary, I re-cross-compiled the mgiza
> binary on our system and it was also dead slow on the 10.13 guest
> taking almost 3 hours.
>
> These tests were run on a very fast host machine with a virtual
> guest. Using Hieu's binaries or mine seems to run against those
> assumptions that hardware/cross-compiler are totally at fault. Has
> anyone else experienced this kind of slow performance?
>
> 2. The lmplz binary fails on this host with this message. Was this not
> compiled with static links for a reason? Where can I find the
> necessary libraries?
> $ ./lmplz
> dyld: Library not loaded:
> /User/hieu/workspace/xmlrpc-c/xmlrpc-c-1.39.07/lib/libxmlrpc_xmltok.3.39.dylib
> ? Referenced from: /opt/macOS/bin/./lmplz
> ? Reason: image not found
> Abort trap: 6
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 135, Issue 10
**********************************************

Tweet