Moses-support Digest, Vol 90, Issue 13

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Moses on Mac OSX (Hieu Hoang)
2. Re: yum-installed Boost environment variables for MGIZA++
(Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Fri, 4 Apr 2014 11:09:01 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: [Moses-support] Moses on Mac OSX
To: Tom Dean <thomas.linus.dean@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID:
<CAEKMkbjJkue=sHdpTvxWwTJWnj-s6gPt=H9ck26wUgDGJS76gg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

If you're trying to compile the Moses (from github) on OSX, you may run
into the following problems:
1. mert program doesn't compile. Fixed by excluding the new
MeteorScorer, which depends on gcc-only functions

https://github.com/moses-smt/mosesdecoder/commit/ee03e0f0e3f113b8de5d6e0e91022394ffef7263
2. doesn't compile with boost 1.55. I've no idea why, but it compiles
with boost 1.54 ok

--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140404/9cf908b2/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 4 Apr 2014 11:17:41 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] yum-installed Boost environment variables
for MGIZA++
To: Gideon <gidi8ster@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhv9hE6+Xo5BQzXybNZCqL-wm3wEP-OVcp2Vc-kyJeHmA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

On 4 April 2014 09:09, Gideon <gidi8ster@gmail.com> wrote:

> Dear Hieu Hoang,
>
> I have now tried to compile Boost manually instead. There are still
> errors, and now Moses compilation also fails. (I assumed that I should
> re-compile Moses since it uses Boost). These were my steps:
>
> cd boost_1_55_0
> ./bootstrap
>
> Following your advice:
>
> ./b2 -j4 -a --prefix=$PWD --libdir=$PWD/lib64 --layout=tagged link=static
> threading=multi,single install || echo FAILURE
> (-a to re-compile)
>
> There is one message written to standard error:
> link.jam: No such file or directory (it does not exist anywhere in the
> downloaded version)
>
> The rest is in standard output. I have attached the two files as a tarball.
>
> Here is a typical warning:
>
> ./boost/concept/detail/general.hpp:71:20: warning: typedef
> 'boost_concept_check653' locally defined but not used
> [-Wunused-local-typedefs]
> BOOST_PP_CAT(boost_concept_check,__LINE__)
>
> And an error:
>
> libs/iostreams/src/bzip2.cpp:20:56: fatal error: bzlib.h: No such file or
> directory
> #include "bzlib.h" // Julian Seward's "bzip.h" header.
>
I looks like the bz2 library and header file isn't installed on your
computer. See here for how you can install it on your distribution
http://www.statmt.org/moses/?n=Development.GetStarted
It's likely this prevent boost_iostream lib being created. The lib file is
usually called
libboost_iostreams*

Boost has to be ok before you can start on trying to install mgiza or Moses


> ^
> compilation terminated.
>
> Eventually:
> ...failed updating 2 targets...
> ...skipped 6 targets...
> ...updated 10795 targets...
>
> Using your command, "ERROR" is written to output. I don't know if it is
> fatal (i.e. if I can still use Moses and MGIZA++).
>
> For Moses, I assume the following is correct:
>
> export BOOST_ROOT=/home/kotzegj/tools/boost_1_55_0
> export BOOST_BUILD_PATH=/home/kotzegj/tools/boost_1_55_0/boost
>
> I noticed that now, BOOST_ROOT does have "include", which means that
> somehow the installation with yum is different.
>
> I ran bjam like this:
>
> ./bjam --with-irstlm=/usr/local/irstlm
> --with-boost=/home/kotzegj/tools/boost_1_55_0 -j4 -a
>
> Here is a typical error:
>
> /usr/bin/ld: cannot find -lboost_iostreams
>
> collect2: error: ld returned 1 exit status
> ...failed gcc.link
> moses-chart-cmd/bin/gcc-4.8.2/release/debug-symbols-on/link-static/threading-multi/moses_chart...
> gcc.compile.c++
> moses-cmd/bin/gcc-4.8.2/release/debug-symbols-on/link-static/threading-multi/mbr.o
>
> I cannot find any file containing boost_iostreams under my Boost path,
> however:
>
> [kotzegj@lt-2433323 boost_1_55_0]$ fnd "iostreams"
> ./boost/iostreams
> ./doc/html/boost_asio/example/cpp03/iostreams
> ./libs/asio/example/cpp03/iostreams
> ./libs/iostreams
> ./bin.v2/libs/iostreams
> ./include/boost/iostreams
>
> Perhaps this means that boost_iostreams wasn't built or something? I can
> attach build.log.gz (of Moses) in a following email.
>
> Thank you for your time.
>
> Gideon
>
>
>
> ---
> gidi8ster@gmail.com
> www.gideonkotze.nl
> +27 78 739 8923 (Mobile)
>
>
> On Thu, Apr 3, 2014 at 6:43 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
>> I'm not too sure. Maybe you should try to compile boost yourself.
>>
>> The instructions are here
>> http://www.statmt.org/moses/?n=Development.GetStarted
>>
>>
>> On 3 April 2014 16:23, Gideon <gidi8ster@gmail.com> wrote:
>>
>>> I have done what you asked, and removed all occurrences of "-mt" in
>>> compile.sh. However:
>>>
>>> /usr/bin/ld: cannot find -lboost_system
>>> /usr/bin/ld: cannot find -lboost_thread
>>>
>>> /usr/bin/ld: cannot find -lpthread
>>> /usr/bin/ld: cannot find -lstdc++
>>> /usr/bin/ld: cannot find -lm
>>> /usr/bin/ld: cannot find -lc
>>> collect2: error: ld returned 1 exit status
>>>
>>> Two other things I noticed:
>>>
>>> - The script expects a $BOOST_ROOT/include, although there exists no
>>> include directory in /home/kotzegj/tools/boost_1_54_0
>>> - Even if I change $BOOST_ROOT to, say, "/usr" (which does contain
>>> "include" which on its own contains "boost"), the error output is exactly
>>> the same.
>>>
>>> I have run "locate boost" and attached the output in case you wanted to
>>> look at it. Perhaps you can tell from it if I have a faulty installation?
>>>
>>> Thanks again.
>>>
>>> Gideon
>>>
>>>
>>>
>>> ---
>>> gidi8ster@gmail.com
>>> www.gideonkotze.nl
>>> +27 78 739 8923 (Mobile)
>>>
>>>
>>> On Thu, Apr 3, 2014 at 4:59 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>>>
>>>> change
>>>> -lboost_*-mt
>>>> to
>>>> -lboost_*
>>>>
>>>> i'm not sure why you're linking to stdc++, m, c.
>>>>
>>>> Check there's not typo in your compile.sh
>>>>
>>>> On 3 April 2014 15:52, Gideon <gidi8ster@gmail.com> wrote:
>>>>
>>>>> Thank you, I have now managed with SVN, it turns out that I had to
>>>>> change the configuration for my (university) proxy settings. However, there
>>>>> are still compilation errors it seems. I have warnings such as this:
>>>>>
>>>>> /home/kotzegj/tools/mgiza/mgizapp-code/mgizapp/src/mkcls/KategProblem.cpp:446:9:
>>>>> warning: deprecated conversion from string constant to 'char*'
>>>>> [-Wwrite-strings]
>>>>> in="other ";
>>>>>
>>>>> And perhaps more importantly, some files that are not found:
>>>>>
>>>>> /usr/bin/ld: cannot find -lboost_system-mt
>>>>> /usr/bin/ld: cannot find -lboost_thread-mt
>>>>> /usr/bin/ld: cannot find -lpthread
>>>>> /usr/bin/ld: cannot find -lstdc++
>>>>> /usr/bin/ld: cannot find -lm
>>>>> /usr/bin/ld: cannot find -lc
>>>>>
>>>>> which would indicate that there are still some problems with (finding)
>>>>> Boost.
>>>>>
>>>>> And still no mgiza binary.
>>>>>
>>>>> Some context (fnd is an alias for "find . -name"):
>>>>>
>>>>> [kotzegj@lt-2433323 usr]$ fnd "*boost_thread*"
>>>>> ./lib64/libboost_thread.so
>>>>> ./lib64/libboost_thread.so.1.54.0
>>>>>
>>>>> [kotzegj@lt-2433323 usr]$ fnd "*boost_system*"
>>>>> ./lib64/libboost_system.so.1.54.0
>>>>> ./lib64/libboost_system.so
>>>>>
>>>>> As mentioned, I installed Boost using yum, and these are my current
>>>>> variables in compile.sh:
>>>>>
>>>>> SRC_DIR=/home/kotzegj/tools/mgiza/mgizapp-code/mgizapp/src
>>>>> BOOST_ROOT=/home/kotzegj/tools/boost_1_54_0
>>>>> BOOST_LIBRARYDIR=/usr/lib64
>>>>>
>>>>> BOOST_ROOT is pointing to the uncompiled unbuilt Boost - perhaps it
>>>>> should point somewhere else?
>>>>>
>>>>> Thank you for your time.
>>>>>
>>>>> Best,
>>>>>
>>>>> Gideon Kotz?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---
>>>>> gidi8ster@gmail.com
>>>>> www.gideonkotze.nl
>>>>>
>>>>>
>>>>> On Thu, Apr 3, 2014 at 3:05 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk>wrote:
>>>>>
>>>>>> You should use the version in svn, rather than a packaged version.
>>>>>> mgiza isn't really looked after by anyone anymore, bug fixes go straight
>>>>>> into the svn and nowhere else.
>>>>>>
>>>>>> I've just pushed a fix for a minor OSX compile error
>>>>>> https://sourceforge.net/p/mgizapp/code/48/
>>>>>>
>>>>>> Try to download it via svn. If it doesn't work, i will consider
>>>>>> moving it to github.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2 April 2014 14:34, Gideon <gidi8ster@gmail.com> wrote:
>>>>>>
>>>>>>> Dear Hieu Hoang
>>>>>>>
>>>>>>> Thank you for you reply. I made some changes:
>>>>>>>
>>>>>>> - uncommented out the top variables and commented out the Mac ones
>>>>>>> - set BOOST_ROOT to /home/kotzegj/tools/boost_1_54_0 and
>>>>>>> BOOST_LIBRARYDIR to /usr/lib64
>>>>>>> - set SRC_DIR to /home/kotzegj/tools/mgiza/mgizapp/src
>>>>>>>
>>>>>>> and ran ./compile.sh.
>>>>>>>
>>>>>>> Two issues remain so far:
>>>>>>>
>>>>>>> 1) I can't find any MGIZA++/mgiza++/etc. binary (find . -name). Is
>>>>>>> there a way to test if MGIZA++ has installed correctly? I did not notice
>>>>>>> any other error messages, apart from the files the script tries to remove
>>>>>>> at the beginning, which do not exist.
>>>>>>>
>>>>>> it should be called
>>>>>> manual-compile/mgiza
>>>>>> If not, there was a compile error
>>>>>>
>>>>>>>
>>>>>>> 2) When compiling, this warning pops up all the time (you are
>>>>>>> probably aware of it)
>>>>>>>
>>>>>>> /usr/include/c++/4.8.2/backward/backward_warning.h:32:2: warning:
>>>>>>> #warning This file includes at least one deprecated or antiquated header
>>>>>>> which may be removed without further notice at a future date. Please use a
>>>>>>> non-deprecated interface with equivalent functionality instead. For a
>>>>>>> listing of replacement headers and interfaces, consult the file
>>>>>>> backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp]
>>>>>>>
>>>>>>> So I assume this is OK for now but there needs to be an update in
>>>>>>> the future?
>>>>>>>
>>>>>> i see it. Be my guest if you wanna update it
>>>>>>
>>>>>>>
>>>>>>> and (just to test)
>>>>>>>
>>>>>>> 2) [kotzegj@lt-2433323 manual-compile]$ ./mkcls
>>>>>>> ERROR: can not open file train.
>>>>>>> Error: Could not read the file 'train'.
>>>>>>>
>>>>>>> But perhaps this is the default?
>>>>>>>
>>>>>> I don't know. Try running it with valid arguments, eg
>>>>>>
>>>>>> http://www.statmt.org/moses/RELEASE-2.1/models/cs-en/steps/1/TRAINING_prepare-data.1.STDERR
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> 3) svn checkout http://svn.code.sf.net/p/mgizapp/code/trunkmgizapp-code
>>>>>>> --> did not work (timeout). I downloaded 0.7.3 from here instead:
>>>>>>> http://www.kyloo.net/software/doku.php/mgiza:overview
>>>>>>>
>>>>>>> svn: E000110: Unable to connect to a repository at URL '
>>>>>>> http://svn.code.sf.net/p/mgizapp/code/trunk'
>>>>>>> svn: E000110: Error running context: Connection timed out
>>>>>>>
>>>>>>> Thank you for your time.
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> Gideon
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---
>>>>>>> gidi8ster@gmail.com
>>>>>>> www.gideonkotze.nl
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 2, 2014 at 2:38 PM, Hieu Hoang <hieuhoang@gmail.com>wrote:
>>>>>>>
>>>>>>>> the cmake build system on mgiza is difficult to use.
>>>>>>>>
>>>>>>>> I create an alternative compile script for mgiza. If you've
>>>>>>>> download mgiza via svn, you should find my script in
>>>>>>>> mgizapp/manual-compile/compile.sh
>>>>>>>> Look at it, change the paths for your own needs
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/04/2014 11:34, Gideon wrote:
>>>>>>>>
>>>>>>>> Dear Moses support team
>>>>>>>>
>>>>>>>> I was wondering if MGIZA++ has been test-compiled on a Fedora
>>>>>>>> Linux system where Boost has been installed using yum, as I'm encountering
>>>>>>>> some problems. I'm working on a Fedora 20 system with x86_64 architecture.
>>>>>>>> So far I have done the following:
>>>>>>>>
>>>>>>>> yum install boost.x86_64
>>>>>>>> yum install boost-devel.x86_64 (version 1.54 was installed)
>>>>>>>> yum install gcc
>>>>>>>> yum install gcc-c++
>>>>>>>> yum install gperftools
>>>>>>>> Downloaded and installed IRSTLM
>>>>>>>> ./bjam --with-irstlm=/home/kotzegj/tools/irstlm-5.80.03 -j4
>>>>>>>> --with-boost=/usr/lib64
>>>>>>>> export BOOST_ROOT=/usr/lib64
>>>>>>>> export
>>>>>>>> BOOST_BUILD_PATH=/home/kotzegj/tools/mosesdecoder/jam-files/boost-build
>>>>>>>> ./bjam --with-irstlm=/usr/local/irstlm -j4
>>>>>>>>
>>>>>>>> I have tested Moses with sample-models and it seems OK.
>>>>>>>>
>>>>>>>> However, MGIZA++ does not find Boost on its own:
>>>>>>>>
>>>>>>>> -- Could NOT find Boost
>>>>>>>> CMake Error at CMakeLists.txt:59 (MESSAGE):
>>>>>>>> Boost not found, please set the BOOST_ROOT and BOOST_LIBRARYDIR
>>>>>>>> environment
>>>>>>>> variables
>>>>>>>>
>>>>>>>> I have no success with setting either of these variables to:
>>>>>>>>
>>>>>>>> - /usr/lib64
>>>>>>>> - /usr/include
>>>>>>>> - /usr/include/boost
>>>>>>>> - /home/kotzegj/tools/mosesdecoder/jam-files/boost-build
>>>>>>>> etc.
>>>>>>>>
>>>>>>>> or if I download Boost manually, set the decompressed directory
>>>>>>>> as BOOST_ROOT, and symlink lib64 as a child and set that as
>>>>>>>> BOOST_LIBRARYDIR. (I've tried all kinds of stuff.)
>>>>>>>>
>>>>>>>> I have the feeling (after some Googling) that the yum
>>>>>>>> installation does not result in the file structure that cmake is looking
>>>>>>>> for, and that instead I should try a manual installation. I hope that I'm
>>>>>>>> wrong?
>>>>>>>>
>>>>>>>> Thank you for your time.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> Gideon Kotz?
>>>>>>>>
>>>>>>>> ---
>>>>>>>> gidi8ster@gmail.com
>>>>>>>> www.gideonkotze.nl
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Moses-support mailing list
>>>>>>>> Moses-support@mit.edu
>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Hieu Hoang
>>>>>> Research Associate
>>>>>> University of Edinburgh
>>>>>> http://www.hoang.co.uk/hieu
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Hieu Hoang
>>>> Research Associate
>>>> University of Edinburgh
>>>> http://www.hoang.co.uk/hieu
>>>>
>>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140404/9dc4d474/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 90, Issue 13
*********************************************

Tweet