Home » Uncategories » Moses-support Digest, Vol 85, Issue 14

Moses-support Digest, Vol 85, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses installation terminated with IRSTLM and RANDLM
(Read, James C)
2. Re: gappy phrases (Matthias Huck)
3. alignment info (Jorg Tiedemann)
4. Re: Moses installation terminated with IRSTLM and RANDLM
(Guchun Zhang)
5. Re: gappy phrases (Read, James C)


----------------------------------------------------------------------

Message: 1
Date: Tue, 5 Nov 2013 19:14:26 +0000
From: "Read, James C" <jcread@essex.ac.uk>
Subject: Re: [Moses-support] Moses installation terminated with IRSTLM
and RANDLM
To: Guchun Zhang <gzhang@alphacrc.com>, "moses-support@MIT.EDU"
<moses-support@MIT.EDU>
Message-ID:
<F00840E41983C645928E21E3C35F4EB1012CF353B4@mbx1-node2.essex.ac.uk>
Content-Type: text/plain; charset="iso-8859-1"

I just wrestled with thing kind of thing. Did you install IRSTLM and RANDLM to your home directory? You need to set compile options --with-randlm and --with-irstlm to whereever you installed them to. If you did ./configure --prefix=$HOME then this will be your home directory.

James



________________________________
From: moses-support-bounces@mit.edu [moses-support-bounces@mit.edu] on behalf of Guchun Zhang [gzhang@alphacrc.com]
Sent: 05 November 2013 17:44
To: moses-support@MIT.EDU
Subject: [Moses-support] Moses installation terminated with IRSTLM and RANDLM

Hi,

Trying to install a new Moses and link it with a few LMs. But I got the following errors, regarding IRSTLM and RANDLM. The directories for search are incorrect, but I don't know where to change them. Or I am expected to create those directories and copy the files into there?

"g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -pthread -DMAX_NUM_FACTORS=4 -DMPI_ENABLE -DNDEBUG -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -I"." -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/irstlm/include" -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/irstlm/include/irstlm" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"util/double-conversion" -c -o "moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/IRST.o" "moses/LM/IRST.cpp"

moses/LM/IRST.cpp:25:24: fatal error: dictionary.h: No such file or directory
compilation terminated.
...failed gcc.compile.c++ moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/IRST.o...
gcc.compile.c++ moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/Rand.o

"g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g -pthread -DMAX_NUM_FACTORS=4 -DMPI_ENABLE -DNDEBUG -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -I"." -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/randlm/include" -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/randlm/include/RandLM" -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi" -I"util/double-conversion" -c -o "moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/Rand.o" "moses/LM/Rand.cpp"

moses/LM/Rand.cpp:38:20: fatal error: RandLM.h: No such file or directory

The build log is attached. Many thanks for the help.

Cheers,
Guchun



------------------------------

Message: 2
Date: Tue, 05 Nov 2013 19:35:37 +0000
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] gappy phrases
To: moses-support@mit.edu
Message-ID: <1383680137.20373.116.camel@portedgar>
Content-Type: text/plain; charset="UTF-8"

Hi James,

I tried Phrasal Beta2 and Beta3 a couple of months ago. Both worked for
me with some minor hassle. You should follow the instructions from
http://www-nlp.stanford.edu/wiki/Software/Phrasal#Phrasal in order to
set up your environment.

I'm also sure that the Phrasal developers are able to assist you with
the installation and usage of their toolkit in case you're running into
trouble.

Cheers,
Matthias


On Tue, 2013-11-05 at 18:52 +0000, Read, James C wrote:
> I just downloaded and installed (I think) a version called Beta4. I
> followed the instructions on the quick start and got to running the
> align script which then bombs out because the VERSION variable isn't
> set to what it should be. Setting that to something sensible it now
> looks like the Berkeley aligner is aligning the suggested data from
> WMT-08.
>
> I note that the instructions guide you to download an old version of
> corenlp and don't give any compilation instructions. Just wget and tar
> xzf and that's it. I'm not convinced that this means corenlp is
> actually 'installed' or that phrasal is using it in any way. Please
> correct me if you know better.
>
> thanks,
> James



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 3
Date: Tue, 5 Nov 2013 20:38:11 +0100
From: Jorg Tiedemann <tiedeman@gmail.com>
Subject: [Moses-support] alignment info
To: "moses-support@MIT.EDU" <moses-support@mit.edu>
Message-ID: <8B96B546-57CE-4934-8BF4-4FA30D4CD143@gmail.com>
Content-Type: text/plain; charset=windows-1252

Hi again,

I have another question about the alignment info option. My output is different from what I expect according to
http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc9

I get only something like
?. ||| 0-0 1-1 2-2 3-3

How do I get both phrase and word alignment in my output?
(like described here: http://article.gmane.org/gmane.comp.nlp.moses.user/3776/match=alignment+info)

Thanks!
J?rg





------------------------------

Message: 4
Date: Tue, 5 Nov 2013 21:02:32 +0000
From: Guchun Zhang <gzhang@alphacrc.com>
Subject: Re: [Moses-support] Moses installation terminated with IRSTLM
and RANDLM
To: "Read, James C" <jcread@essex.ac.uk>
Cc: "moses-support@MIT.EDU" <moses-support@mit.edu>
Message-ID:
<CA+cfSVLsA0ka=6jz=4m71DYp_B6Y7VAhp9TBg8W=UEkggbWdMA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks, James for the quick reply.

I probably should have read READMEs more carefully. It's the prefix.

Cheers,
Guchun


On 5 November 2013 19:14, Read, James C <jcread@essex.ac.uk> wrote:

> I just wrestled with thing kind of thing. Did you install IRSTLM and
> RANDLM to your home directory? You need to set compile options
> --with-randlm and --with-irstlm to whereever you installed them to. If you
> did ./configure --prefix=$HOME then this will be your home directory.
>
> James
>
>
>
> ________________________________
> From: moses-support-bounces@mit.edu [moses-support-bounces@mit.edu] on
> behalf of Guchun Zhang [gzhang@alphacrc.com]
> Sent: 05 November 2013 17:44
> To: moses-support@MIT.EDU
> Subject: [Moses-support] Moses installation terminated with IRSTLM and
> RANDLM
>
> Hi,
>
> Trying to install a new Moses and link it with a few LMs. But I got the
> following errors, regarding IRSTLM and RANDLM. The directories for search
> are incorrect, but I don't know where to change them. Or I am expected to
> create those directories and copy the files into there?
>
> "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -g
> -pthread -DMAX_NUM_FACTORS=4 -DMPI_ENABLE -DNDEBUG -DTRACE_ENABLE=1
> -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -I"."
> -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/irstlm/include"
> -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/irstlm/include/irstlm"
> -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi"
> -I"util/double-conversion" -c -o
> "moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/IRST.o"
> "moses/LM/IRST.cpp"
>
> moses/LM/IRST.cpp:25:24: fatal error: dictionary.h: No such file or
> directory
> compilation terminated.
> ...failed gcc.compile.c++
> moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/IRST.o...
> gcc.compile.c++
> moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/Rand.o
>
> "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall
> -g -pthread -DMAX_NUM_FACTORS=4 -DMPI_ENABLE -DNDEBUG -DTRACE_ENABLE=1
> -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES -I"."
> -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/randlm/include"
> -I"/home/alpha_mt/mt-grid/tools/mt/moses/external-tools/randlm/include/RandLM"
> -I"/usr/lib/openmpi/include" -I"/usr/lib/openmpi/include/openmpi"
> -I"util/double-conversion" -c -o
> "moses/LM/bin/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/Rand.o"
> "moses/LM/Rand.cpp"
>
> moses/LM/Rand.cpp:38:20: fatal error: RandLM.h: No such file or directory
>
> The build log is attached. Many thanks for the help.
>
> Cheers,
> Guchun
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131105/710a0e1d/attachment-0001.htm

------------------------------

Message: 5
Date: Tue, 5 Nov 2013 21:54:42 +0000
From: "Read, James C" <jcread@essex.ac.uk>
Subject: Re: [Moses-support] gappy phrases
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, Daniel Cer
<daniel.cer@gmail.com>
Message-ID:
<F00840E41983C645928E21E3C35F4EB1012CF3541E@mbx1-node2.essex.ac.uk>
Content-Type: text/plain; charset="iso-8859-1"

Could you tell me a little bit more? When you say basic decoder what do you mean? What components does it have? Would I need to first train my translation models in the usual way? Will phrase tables generated with the Moses training script be in the right format for it?

What I really need is to perhaps implement my own translation model. I want to experiment with different units of translation and observe whether the translation model ranks better translations higher in isolation. For this reason I need to eliminate other variables like the effects of reordering models, language models and beam searches.

James

________________________________
From: hieuhoang@gmail.com [hieuhoang@gmail.com] on behalf of Hieu Hoang [Hieu.Hoang@ed.ac.uk]
Sent: 05 November 2013 12:12
To: Read, James C
Cc: Daniel Cer; Kenneth Heafield; moses-support@mit.edu
Subject: Re: [Moses-support] gappy phrases

If you're gonna throw most of Moses away, you may consider using and extending the basic decoder I've just added to github
https://github.com/moses-smt/mosesdecoder/tree/master/contrib/basic-decoder

Or write your own. You're welcome to add it to moses too


On 5 November 2013 07:29, Read, James C <jcread@essex.ac.uk<mailto:jcread@essex.ac.uk>> wrote:
Interesting.

This isn't going to be an issue for the kind of experiments I will be running, though. I won't be using a language model or a reordering model or a beam search during decoding (I know, not much of Moses left). My main usage of 'Moses' at the moment is training translation models so I can run experiments with 'units of translation' in isolation from other variables in the system (language model, reordering model, beam search). I would really like to be able to run some experiments also with discontinuous phrases both source side and target side.

Any idea what kind of changes I would need to make to the training process to be able to learn these kind of transformations? I suppose I'm also going to need to modify the operation of the translation model to get these working as well.

thanks,
James

________________________________
From: moses-support-bounces@mit.edu<mailto:moses-support-bounces@mit.edu> [moses-support-bounces@mit.edu<mailto:moses-support-bounces@mit.edu>] on behalf of Daniel Cer [daniel.cer@gmail.com<mailto:daniel.cer@gmail.com>]
Sent: 04 November 2013 19:43
To: Kenneth Heafield
Cc: moses-support@mit.edu<mailto:moses-support@mit.edu>
Subject: Re: [Moses-support] gappy phrases

Hi everyone,

Ken and I just spoke about this.

Here's a quick synopsis of our semi-recent experience at Stanford with discontinuous/gappy phrases:

* Source side gaps are effectively free and don't really degrade decoding time.
* Target side gaps are fine for smaller beam sizes (e.g., < 200).
* When using large beams, our current implementation slows down dramatically. For example, with a stack size of 500, I think it was sometimes taking over an hour to translate some sentences.

While discontinuous phrases can moderately increase the BLEU score, but we get a bigger increase by just using very large beam sizes.

Dan



On Sun, Nov 3, 2013 at 8:34 PM, Kenneth Heafield <moses@kheafield.com<mailto:moses@kheafield.com><mailto:moses@kheafield.com<mailto:moses@kheafield.com>>> wrote:
Hi,

I'll throw in the anecdote that gappy phrases are currently not in use
at Stanford. My predecessor told me that it took a lot longer and only
improved BLEU slightly on Chinese-English. But it's also possible that
something didn't get passed down correctly from Michel to my predecessor
to me. . .

Kenneth

On 11/03/13 14:18, Read, James C wrote:
> My understanding is that they used a similar approach as the grammar extraction to extract the gappy phrases. Would it be a massive undertaking to get Moses to support this?
>
> James
> ________________________________________
> From: Barry Haddow [bhaddow@staffmail.ed.ac.uk<mailto:bhaddow@staffmail.ed.ac.uk><mailto:bhaddow@staffmail.ed.ac.uk<mailto:bhaddow@staffmail.ed.ac.uk>>]
> Sent: 30 October 2013 09:26
> To: Read, James C
> Cc: moses-support@mit.edu<mailto:moses-support@mit.edu><mailto:moses-support@mit.edu<mailto:moses-support@mit.edu>>
> Subject: Re: [Moses-support] gappy phrases
>
> No, but it does support hiero and syntax models.
>
> On 29/10/13 22:23, Read, James C wrote:
>> Hi,
>>
>> does anybody know if Moses supports gappy phrases http://www-nlp.stanford.edu/pubs/naacl10-discontinuous_phrases.pdf
>>
>> James
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu<mailto:Moses-support@mit.edu><mailto:Moses-support@mit.edu<mailto:Moses-support@mit.edu>>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu<mailto:Moses-support@mit.edu><mailto:Moses-support@mit.edu<mailto:Moses-support@mit.edu>>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu<mailto:Moses-support@mit.edu><mailto:Moses-support@mit.edu<mailto:Moses-support@mit.edu>>
http://mailman.mit.edu/mailman/listinfo/moses-support


_______________________________________________
Moses-support mailing list
Moses-support@mit.edu<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu




------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 85, Issue 14
*********************************************

0 Response to "Moses-support Digest, Vol 85, Issue 14"

Post a Comment

Subscribe to: Post Comments (Atom)