Moses-support Digest, Vol 97, Issue 18

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Load LD_LIBRARY_PATH in ems with -cluster (sge) (Hieu Hoang)
2. Re: Load LD_LIBRARY_PATH in ems with -cluster (sge)
(Raymond W. M. Ng)
3. Re: Load LD_LIBRARY_PATH in ems with -cluster (sge) (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Thu, 13 Nov 2014 13:36:12 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Load LD_LIBRARY_PATH in ems with -cluster
(sge)
To: "Raymond W. M. Ng" <wm.ng@sheffield.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjPBwiK5CotrWSO9hd7MLcDbs4iBhY2qHa1pCphtm+w5A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

personally, I would try to ensure that moses was statically linked to the
boost libraries so it won't go looking for the libboost*.so at runtime.

I do this by building my own boost library and linking to it, rather than
the system boost. The instructions are here
http://www.statmt.org/moses/?n=Development.GetStarted

On 13 November 2014 13:23, Raymond W. M. Ng <wm.ng@sheffield.ac.uk> wrote:

> Dear moses,
>
> I have the following error in training my translation model with ems on
> SGE.
>
> /home/users/moses/v-2014-08-15/mosesdecoder/bin/build_binary: error while
> loading shared libraries: libboost_thread-mt.so.1.56.0: cannot open shared
> object file: No such file or directory
>
> In the offline mode, I can run this without problem by first sourcing an
> environment file which contains something like
> setenv LD_LIBRARY_PATH /export/sw/boost/v1.56.0/x86_64/lib
>
> So, I think the LD_LIBRARY_PATH environment was not loaded in the sge
> parallel scripts,
>
> grep 'ENV' /home/moses/mosesdecoder/scripts/generic/qsub-wrapper.pl
>
> returns nothing.
>
> Is there something to modify in the code or did I miss any setting so the
> environment variable is not loaded?
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141113/8f4b9764/attachment-0001.htm

------------------------------

Message: 2
Date: Thu, 13 Nov 2014 14:39:45 +0000
From: "Raymond W. M. Ng" <wm.ng@sheffield.ac.uk>
Subject: Re: [Moses-support] Load LD_LIBRARY_PATH in ems with -cluster
(sge)
To: undisclosed-recipients:;
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CANkx1N7s3=7Sr-gHavPqBnvmQU-0zFaB0hq7g7KVgkF6rYmx0w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks Hieu,

I did link boost (and sirlm, irstlm) when i built moses.

./bjam --with-boost=/export/sw/std/boost/v1.56.0/x86_64
--with-srilm=/export/sw/spl/srilm/v1.6.0/srilm
--with-irstlm=/export/sw/spl/irstlm/v5.80.03/x86_64 -j8

But I think at run time I still need to load the path.
setenv LD_LIBRARY_PATH /export/sw/std/boost/v1.56.0/x86_64/lib

It won't cause an issue if everything is run locally, because I could
source the environment just once. Maybe this problem is, again, specific to
computing environment

cheers
raymond


On 13 November 2014 13:36, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:

> personally, I would try to ensure that moses was statically linked to the
> boost libraries so it won't go looking for the libboost*.so at runtime.
>
> I do this by building my own boost library and linking to it, rather than
> the system boost. The instructions are here
> http://www.statmt.org/moses/?n=Development.GetStarted
>
> On 13 November 2014 13:23, Raymond W. M. Ng <wm.ng@sheffield.ac.uk> wrote:
>
>> Dear moses,
>>
>> I have the following error in training my translation model with ems on
>> SGE.
>>
>> /home/users/moses/v-2014-08-15/mosesdecoder/bin/build_binary: error while
>> loading shared libraries: libboost_thread-mt.so.1.56.0: cannot open shared
>> object file: No such file or directory
>>
>> In the offline mode, I can run this without problem by first sourcing an
>> environment file which contains something like
>> setenv LD_LIBRARY_PATH /export/sw/boost/v1.56.0/x86_64/lib
>>
>> So, I think the LD_LIBRARY_PATH environment was not loaded in the sge
>> parallel scripts,
>>
>> grep 'ENV' /home/moses/mosesdecoder/scripts/generic/qsub-wrapper.pl
>>
>> returns nothing.
>>
>> Is there something to modify in the code or did I miss any setting so the
>> environment variable is not loaded?
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141113/021bbf10/attachment-0001.htm

------------------------------

Message: 3
Date: Thu, 13 Nov 2014 14:44:55 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Load LD_LIBRARY_PATH in ems with -cluster
(sge)
To: "Raymond W. M. Ng" <wm.ng@sheffield.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbip0015w5H9k5OKLHDER+mH9GwUbvkb_HgMM7Q4U+Sxiw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

you don't need to set LD_LIBRARY_PATH if the boost lib statically linked.
This is how I always use it, I never set this variable.

You need to make sure there's no .so file in the folder
/export/sw/std/boost/v1.56.0/x86_64/lib
/export/sw/std/boost/v1.56.0/x86_64/lib64


On 13 November 2014 14:39, Raymond W. M. Ng <wm.ng@sheffield.ac.uk> wrote:

> Thanks Hieu,
>
> I did link boost (and sirlm, irstlm) when i built moses.
>
> ./bjam --with-boost=/export/sw/std/boost/v1.56.0/x86_64
> --with-srilm=/export/sw/spl/srilm/v1.6.0/srilm
> --with-irstlm=/export/sw/spl/irstlm/v5.80.03/x86_64 -j8
>
> But I think at run time I still need to load the path.
> setenv LD_LIBRARY_PATH /export/sw/std/boost/v1.56.0/x86_64/lib
>
> It won't cause an issue if everything is run locally, because I could
> source the environment just once. Maybe this problem is, again, specific to
> computing environment
>
> cheers
> raymond
>
>
> On 13 November 2014 13:36, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
>> personally, I would try to ensure that moses was statically linked to the
>> boost libraries so it won't go looking for the libboost*.so at runtime.
>>
>> I do this by building my own boost library and linking to it, rather than
>> the system boost. The instructions are here
>> http://www.statmt.org/moses/?n=Development.GetStarted
>>
>> On 13 November 2014 13:23, Raymond W. M. Ng <wm.ng@sheffield.ac.uk>
>> wrote:
>>
>>> Dear moses,
>>>
>>> I have the following error in training my translation model with ems on
>>> SGE.
>>>
>>> /home/users/moses/v-2014-08-15/mosesdecoder/bin/build_binary: error
>>> while loading shared libraries: libboost_thread-mt.so.1.56.0: cannot open
>>> shared object file: No such file or directory
>>>
>>> In the offline mode, I can run this without problem by first sourcing an
>>> environment file which contains something like
>>> setenv LD_LIBRARY_PATH /export/sw/boost/v1.56.0/x86_64/lib
>>>
>>> So, I think the LD_LIBRARY_PATH environment was not loaded in the sge
>>> parallel scripts,
>>>
>>> grep 'ENV' /home/moses/mosesdecoder/scripts/generic/qsub-wrapper.pl
>>>
>>> returns nothing.
>>>
>>> Is there something to modify in the code or did I miss any setting so
>>> the environment variable is not loaded?
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141113/67143e91/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 97, Issue 18
*********************************************

Tweet