Home » Uncategories » Moses-support Digest, Vol 90, Issue 2

Moses-support Digest, Vol 90, Issue 2

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. yum-installed Boost environment variables for MGIZA++ (Gideon)
2. Re: yum-installed Boost environment variables for MGIZA++
(Hieu Hoang)
3. Re: <type> in train-model.perl's -lm parameter (Hieu Hoang)
4. Re: factored training data question (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 2 Apr 2014 12:34:09 +0200
From: Gideon <gidi8ster@gmail.com>
Subject: [Moses-support] yum-installed Boost environment variables for
MGIZA++
To: moses-support@mit.edu
Message-ID:
<CALP7mv=_sfVjRj=kKy5fw_+caMRpmF53T=beWpBT+fwrtXeE5w@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Dear Moses support team

I was wondering if MGIZA++ has been test-compiled on a Fedora Linux system
where Boost has been installed using yum, as I'm encountering some
problems. I'm working on a Fedora 20 system with x86_64 architecture. So
far I have done the following:

yum install boost.x86_64
yum install boost-devel.x86_64 (version 1.54 was installed)
yum install gcc
yum install gcc-c++
yum install gperftools
Downloaded and installed IRSTLM
./bjam --with-irstlm=/home/kotzegj/tools/irstlm-5.80.03 -j4
--with-boost=/usr/lib64
export BOOST_ROOT=/usr/lib64
export
BOOST_BUILD_PATH=/home/kotzegj/tools/mosesdecoder/jam-files/boost-build
./bjam --with-irstlm=/usr/local/irstlm -j4

I have tested Moses with sample-models and it seems OK.

However, MGIZA++ does not find Boost on its own:

-- Could NOT find Boost
CMake Error at CMakeLists.txt:59 (MESSAGE):
Boost not found, please set the BOOST_ROOT and BOOST_LIBRARYDIR
environment
variables

I have no success with setting either of these variables to:

- /usr/lib64
- /usr/include
- /usr/include/boost
- /home/kotzegj/tools/mosesdecoder/jam-files/boost-build
etc.

or if I download Boost manually, set the decompressed directory as
BOOST_ROOT, and symlink lib64 as a child and set that as BOOST_LIBRARYDIR.
(I've tried all kinds of stuff.)

I have the feeling (after some Googling) that the yum installation does not
result in the file structure that cmake is looking for, and that instead I
should try a manual installation. I hope that I'm wrong?

Thank you for your time.

Best regards,

Gideon Kotz?

---
gidi8ster@gmail.com
www.gideonkotze.nl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140402/87794346/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 02 Apr 2014 13:38:35 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] yum-installed Boost environment variables
for MGIZA++
To: moses-support@mit.edu
Message-ID: <533C04CB.9040901@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

the cmake build system on mgiza is difficult to use.

I create an alternative compile script for mgiza. If you've download
mgiza via svn, you should find my script in
mgizapp/manual-compile/compile.sh
Look at it, change the paths for your own needs

On 02/04/2014 11:34, Gideon wrote:
> Dear Moses support team
>
> I was wondering if MGIZA++ has been test-compiled on a Fedora Linux
> system where Boost has been installed using yum, as I'm encountering
> some problems. I'm working on a Fedora 20 system with x86_64
> architecture. So far I have done the following:
>
> yum install boost.x86_64
> yum install boost-devel.x86_64 (version 1.54 was installed)
> yum install gcc
> yum install gcc-c++
> yum install gperftools
> Downloaded and installed IRSTLM
> ./bjam --with-irstlm=/home/kotzegj/tools/irstlm-5.80.03 -j4
> --with-boost=/usr/lib64
> export BOOST_ROOT=/usr/lib64
> export
> BOOST_BUILD_PATH=/home/kotzegj/tools/mosesdecoder/jam-files/boost-build
> ./bjam --with-irstlm=/usr/local/irstlm -j4
>
> I have tested Moses with sample-models and it seems OK.
>
> However, MGIZA++ does not find Boost on its own:
>
> -- Could NOT find Boost
> CMake Error at CMakeLists.txt:59 (MESSAGE):
> Boost not found, please set the BOOST_ROOT and BOOST_LIBRARYDIR
> environment
> variables
>
> I have no success with setting either of these variables to:
>
> - /usr/lib64
> - /usr/include
> - /usr/include/boost
> - /home/kotzegj/tools/mosesdecoder/jam-files/boost-build
> etc.
>
> or if I download Boost manually, set the decompressed directory as
> BOOST_ROOT, and symlink lib64 as a child and set that as
> BOOST_LIBRARYDIR. (I've tried all kinds of stuff.)
>
> I have the feeling (after some Googling) that the yum installation
> does not result in the file structure that cmake is looking for, and
> that instead I should try a manual installation. I hope that I'm wrong?
>
> Thank you for your time.
>
> Best regards,
>
> Gideon Kotz?
>
> ---
> gidi8ster@gmail.com <mailto:gidi8ster@gmail.com>
> www.gideonkotze.nl <http://www.gideonkotze.nl/>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140402/70557e01/attachment-0001.htm

------------------------------

Message: 3
Date: Wed, 02 Apr 2014 14:11:08 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] <type> in train-model.perl's -lm
parameter
To: moses-support@mit.edu
Message-ID: <533C0C6C.5080101@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

There used to be a enum type in the decoder code, but the format change
in the moses.ini has made it obsolete.

there's only now passing reference for it in the code, eg.
train-model.perl line 2063-2066

$type_name = "SRILM" if $type == 0;
$type_name = "IRSTLM" if $type == 1;
$type_name = "KENLM lazyken=0" if $type == 8;
$type_name = "KENLM lazyken=1" if $type == 9;

We should try to make this more user friendly in future, but it requires
changes to other training scripts


On 29/03/2014 18:10, Benjamin K?rner wrote:
>
> Dear all,
>
> I somehow remember that the third : separated numeric in the --lm
> parameter of train-model.perl specifies the LM type. See
>
> -lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:*8*
>
> So 8 for KenLM.
>
> Is there any source for that?
>
> Thanks,
>
> Benjamin
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140402/73f3eadf/attachment-0001.htm

------------------------------

Message: 4
Date: Wed, 02 Apr 2014 14:14:29 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] factored training data question
To: Viktor Pless <viktor.pless@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <533C0D35.1090209@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

the POS taggers will probably produce
POS1 POS2 POS3 POS4 ....
which corresponding to the sentence
word1 word2 word3 word4

to get the format
word1|POS1 word2|POS2 word3|POS3 word4|POS4

Use the script
scripts/training/combine_factors.pl

On 28/03/2014 22:40, Viktor Pless wrote:
> Hi everyone,
> How can I produce factored training data like below?
>
> "You will have to provide training data in the format
> word0factor0|word0factor1|word0factor2
> word1factor0|word1factor1|word1factor2 ..."
>
> Could you please tell me the what program produces this format?
> Neither MXPOST nor treetagger does it for me.
>
> Thank you in advance.
> Viktor
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140402/a57d8cd9/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 90, Issue 2
********************************************

0 Response to "Moses-support Digest, Vol 90, Issue 2"

Post a Comment

Subscribe to: Post Comments (Atom)