Moses-support Digest, Vol 104, Issue 6

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Decoder can't read binarised version of lexical reordering
table in official Moses virtual machine (Davood Mohammadifar)
2. Re: Fail to build and compile moses (Jeroen Vermeulen)
3. Re: Renaming aux.m4m (Jeroen Vermeulen)
4. Re: Fail to build and compile moses (Tom Hoar)
5. Error while training the translation system (Harshit Gupta)
6. Can't read reordering table (Hamed Soleimani)


----------------------------------------------------------------------

Message: 1
Date: Tue, 2 Jun 2015 16:39:21 +0000
From: Davood Mohammadifar <davood_mf@hotmail.com>
Subject: [Moses-support] Decoder can't read binarised version of
lexical reordering table in official Moses virtual machine
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <SNT150-W52017254FA30E552E3A8598CB50@phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"

Hello everyone

I was trying to buid baseline system in the Ubuntu 14.04 32-bit (Virtual Mahine that i downloaded from Moses web-page). It has moses v3.0. I imported VM using VMware player.

I performed steps for buiding baseline system until the end of binarising phrase table and reordering table successfully. After that, i changed moses.ini config file based on instructions in manual:

1. Change PhraseDictionaryMemory to PhraseDictionaryCompact
2. Set the path of the PhraseDictionary feature to point to /home/hieu/working/binarised-model/phrase-table
3. Set the path of the LexicalReordering feature to point to /home/hieu/working/binarised-model/reordering-table

After executing the following command, some messages related to loading config file and performing some operation are shown.
~/workspace/github/mosesdecoder/bin/moses -f ~/working/mert-work/moses.ini

But, i faced the folowing error:
Can't read /home/hieu/working/binarised-model/reordering-table

I found similar problem here:
https://www.mail-archive.com/moses-support%40mit.edu/msg12027.html

Solution for that problem was in the next messages. In fact, solution was deleting /bin/ folder and recompiling moses:

./bjam -a --with-boost=/home/hieu/workspace/boost/boost_1_55_0 --with-cmph=/home/hieu/workspace/cmph-2.0 --with-irstlm=/home/hieu/workspace/irstlm -j8

(path was correctet based on my VM)

But in virtual machine, cmph-2.0 folder is in github folder. So i guess that Moses has compiled with it.
So, Which solution would you recommend? Should i recomplile Moses?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150602/321e3d20/attachment-0001.htm

------------------------------

Message: 2
Date: Tue, 02 Jun 2015 23:54:51 +0700
From: Jeroen Vermeulen <jtv@precisiontranslationtools.com>
Subject: Re: [Moses-support] Fail to build and compile moses
To: Davood Mohammadifar <davood_mf@hotmail.com>,
"moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <556DDFDB.4080005@precisiontranslationtools.com>
Content-Type: text/plain; charset=windows-1252

On 02/06/15 22:55, Davood Mohammadifar wrote:
> my compiler: GCC 4.7.2
> I wondered that I was seeing the folder gcc-4.6 in the messages.

Could it be that your compiler was installed incorrectly? Maybe it's
confused about multiple versions.


> My system is not strong:
>
> Memory: 2GB
> Cores: 2 (intel Pentium Dual core)
>
> Should i compile with ./bjam -j2 instead of j8?

Yes, definitely try -j2.


Jeroen


------------------------------

Message: 3
Date: Wed, 03 Jun 2015 00:04:31 +0700
From: Jeroen Vermeulen <jtv@precisiontranslationtools.com>
Subject: Re: [Moses-support] Renaming aux.m4m
To: ugermann@inf.ed.ac.uk
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <556DE21F.6040405@precisiontranslationtools.com>
Content-Type: text/plain; charset=utf-8

On 02/06/15 20:44, Ulrich Germann wrote:
> Nope. Hardly anyone (if anyone except me) is using it anyway, and as far
> as I'm concerned, it's on its way to a separate repo anyway (so that I
> can clone it into an experimental directory without pulling in the whole
> moses saussage. aux.m4m has no external user exposure, so it should be
> safe to rename.

Excellent, thanks. I just renamed it.


Jeroen



------------------------------

Message: 4
Date: Wed, 03 Jun 2015 10:45:48 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Fail to build and compile moses
To: moses-support <moses-support@mit.edu>
Message-ID: <556E786C.80908@precisiontranslationtools.com>
Content-Type: text/plain; charset=utf-8; format=flowed

May I ask why you are trying to use inadequate hardware for Moses? You
are unlikely to achieve a stable system with your hardware and 32-bit
compiled version of Moses, regardless of the size of your swap file.

This is the bare minimum configuration we consider tolerable as a
testing and learning environments, but it performs unacceptably slow for
most production tasks.

CPU: QuadCore, 2.6 GHz, single-threading
RAM: 4 GB
Hard Disk: 1 x 500 Gigabyte

This is a capable baseline configuration that performs acceptably well
with light production tasks when translating one or two language pairs
per week and training/tuning one new translation model monthly.

CPU: Quad core, 2.6 GHz, dual-threading
RAM: 8 GB
Hard Disk: 1 Terabyte


Date: Tue, 2 Jun 2015 15:55:29 +0000
From: Davood Mohammadifar <davood_mf@hotmail.com>
Subject: Re: [Moses-support] Fail to build and compile moses
To: Jeroen Vermeulen <jtv@precisiontranslationtools.com>,
"moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <SNT150-W76CE3A168AF3523F0E81328CB50@phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"

my compiler: GCC 4.7.2
I wondered that I was seeing the folder gcc-4.6 in the messages.

My system is not strong:

Memory: 2GB
Cores: 2 (intel Pentium Dual core)

Should i compile with ./bjam -j2 instead of j8?



------------------------------

Message: 5
Date: Wed, 3 Jun 2015 18:32:02 +0530
From: Harshit Gupta <harshitgupta165@gmail.com>
Subject: [Moses-support] Error while training the translation system
To: moses-support@mit.edu
Message-ID:
<CAHgj_vsXsphMBrtPVDV3gpU0OK=AYjcibV9jbHr50x1JG-y3zg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

This is the command given in moses baseline tutorial for training the corpora.

mkdir ~/working
cd ~/working
nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir train \
-corpus ~/corpus/news-commentary-v8.fr-en.clean \
-f fr -e en -alignment grow-diag-final-and -reordering msd-bidirectional-fe \
-lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8 \
-external-bin-dir ~/mosesdecoder/tools >& training.out &
I followed all the steps for preparing the corpora, so I have followed
the same directory structure.

However, when I am running this command, I am not getting the model
folder and moses.ini file in ~/working/train/model .

On terminal, it just displays a number after I execute this command.
Also, the training.out file generated in ~/working/train

contains most of the lines as ERROR : parameter does not exist and
unrecognized attribute.

Please help me through it.


--
Harshit Gupta
Third Year Undergraduate
Electrical Engineering
IIT Madras
P* Please don't print this e-mail unless you really need to*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150603/81a96867/attachment-0001.htm

------------------------------

Message: 6
Date: Wed, 03 Jun 2015 14:21:39 +0000
From: Hamed Soleimani <hamed1soleimani@gmail.com>
Subject: [Moses-support] Can't read reordering table
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAMH7RokEPFsvDG=M_JBwnx4xzKdFt0qcL6aP+i5hF4qnrXSP3A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

My baseline translation systems was working great, but yesterday as i
updated and compiled mosesdecoder source code again and switched from giza
to mgiza I couldn't translate any sentence anymore. I lookup recent commits
in github but i didn't find anything related to my problem.

This is result of decoder:

[hamed1soleimani@hamed1soleimani ~]$ mosesdecoder/bin/moses -f
working/v1.2/binarised-model/moses.ini
Defined parameters (per moses.ini or switch):
config: working/v1.2/binarised-model/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/hamed1soleimani/working/v1.2/binarised-model/phrase-table
input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0
num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0
output-factor=0
path=/home/hamed1soleimani/working/v1.2/binarised-model/reordering-table
Distortion KENLM lazyken=0 name=LM0 factor=0
path=/home/hamed1soleimani/lm/v1.2/mizan.final.blm.en order=3
input-factors: 0
mapping: 0 T 0
threads: 4
weight: LexicalReordering0= 0.0605259 0.0381224 0.183614 0.0646507
0.0559396 0.0463035 Distortion0= 0.0353544 LM0= 0.0781512 WordPenalty0=
-0.17106 PhrasePenalty0= 0.0882376 TranslationModel0= 0.0128144 0.0528043
0.0662476 0.0461745 UnknownWordPenalty0= 1
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/hamed1soleimani/working/v1.2/binarised-model/phrase-table
input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/hamed1soleimani/working/v1.2/binarised-model/reordering-table
FeatureFunction: LexicalReordering0 start: 7 end: 12
Initializing Lexical Reordering Feature..
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM lazyken=0 name=LM0 factor=0
path=/home/hamed1soleimani/lm/v1.2/mizan.final.blm.en order=3
FeatureFunction: LM0 start: 14 end: 14
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Can't read
/home/hamed1soleimani/working/v1.2/binarised-model/reordering-table



my moses.ini file is:

# MERT optimized configuration
# decoder /home/hamed1soleimani/mosesdecoder/bin/moses
# BLEU 0.0653349 on dev
/home/hamed1soleimani/corpus/v1.2/mizan.sample.tun.true.fa
# We were before running iteration 9
# finished Wed Jun 3 16:08:23 IRDT 2015
### MOSES CONFIG FILE ###
#########################

# input factors
[input-factors]
0

# mapping steps
[mapping]
0 T 0

[distortion-limit]
6

# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/hamed1soleimani/working/v1.2/binarised-model/phrase-table
input-factor=0 output-factor=0
LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/hamed1soleimani/working/v1.2/binarised-model/reordering-table
Distortion
KENLM lazyken=0 name=LM0 factor=0
path=/home/hamed1soleimani/lm/v1.2/mizan.final.blm.en order=3

# dense weights for feature functions

[threads]
4
[weight]

LexicalReordering0= 0.0605259 0.0381224 0.183614 0.0646507 0.0559396
0.0463035
Distortion0= 0.0353544
LM0= 0.0781512
WordPenalty0= -0.17106
PhrasePenalty0= 0.0882376
TranslationModel0= 0.0128144 0.0528043 0.0662476 0.0461745
UnknownWordPenalty0= 1

Best regards,
Hamed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150603/3ef919fd/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 104, Issue 6
*********************************************

Tweet