Moses-support Digest, Vol 89, Issue 69

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Implementation of pre-reordering for German and other
languages (Maxim Khalilov)
2. Mosesserver fails with "core dumped" (Ertu?rul YILMAZ (B?LGEM))
3. cube pruning for phrase-based decoding (Michael me)
4. Call for papers (CogALex-IV) workshop co-located with Coling
(zock)


----------------------------------------------------------------------

Message: 1
Date: Tue, 25 Mar 2014 18:07:37 +0100
From: Maxim Khalilov <maxkhalilov@gmail.com>
Subject: [Moses-support] Implementation of pre-reordering for German
and other languages
To: moses-support@mit.edu, mt-list@eamt.org
Message-ID:
<CAPKeiSBo0HX5Df2QL+ATnO48JWbLK6-uOCoOZnTgZzMnK5bESg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Moses community,

I am looking for a ready to use or easily customizable implementation of
pre-reordering algorithms for Moses? In particular, I'm interested in
language pairs with German as a source language and a variety of languages
as target, so probably the best solution is syntax based.

As a starting point, I would consider the algorithm described in (Collins
et al., 2005), but I don't know if there is an implementation available and
which parser it relies on.

Thanks for your help beforehand,
Maxim Khalilov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140325/9fe85c16/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 26 Mar 2014 12:11:37 +0200 (EET)
From: Ertu?rul YILMAZ (B?LGEM) <yilmaz.ertugrul@tubitak.gov.tr>
Subject: [Moses-support] Mosesserver fails with "core dumped"
To: moses-support@mit.edu
Message-ID:
<350942122.104328991.1395828697387.JavaMail.zimbra@tubitak.gov.tr>
Content-Type: text/plain; charset="utf-8"

Hi All,

I am working on running a phrase-based MT system with mosesserver resulting in the attached failure.
I was able to run the same system with binarized phrase and reordering tables, but it fails to run them in memory.

Did any of you run into a similar problem or have any ideas on the resolution for this? Attaching my moses.ini also.

Thanks,

ertugrul@pala01$ /usr/local/smt/moses_2013_11/bin/mosesserver -threads 8 -dl 5 -minimum-bayes-risk -v 0 -f moses.tuned.ini.5.mod
/usr/local/smt/moses_2013_11/bin
line=UnknownWordPenalty
line=WordPenalty
line=PhrasePenalty
line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/media/pala12/burak/aren/ems2/model/phrase-table-sigtest-filter.5.gz input-factor=0 output-factor=0
line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/media/pala12/burak/aren/ems2/model/reordering-table-sigtest-filter.5.wbe-msd-bidirectional-fe.gz
Initializing LexicalReordering..
line=Distortion
line=KENLM lazyken=0 name=LM0 factor=0 path=/media/pala12/burak/aren/ems2/lm/nist12.binlm.1 order=4
line=KENLM lazyken=0 name=LM1 factor=0 path=/media/pala12/burak/aren/ems2/lm/isi.binlm.1 order=4
line=KENLM lazyken=0 name=LM2 factor=0 path=/media/pala12/burak/aren/ems2/lm/un.binlm.1 order=4
line=KENLM lazyken=0 name=LM3 factor=0 path=/media/pala06/ilknur/pala05/gigaword_lm/gigaword5gr.tcafteraps3.irstlm.bin order=5
Loading table into memory...done.
Start loading text SCFG phrase table. Moses format : [233] seconds
Check nextPos != string::npos failed in moses/Phrase.cpp:214
Aborted (core dumped)


----------------------------------------------------------
Ertu?rul YILMAZ
Senior Researcher

Speech and Natural Language Processing Group
T?B?TAK B?LGEM
41470 GEBZE / KOCAEL?, TURKEY
T +90 262 648 1000 - 2216
F +90 262 648 1100
www.bilgem.tubitak.gov.tr
yilmaz.ertugrul@tubitak.gov.tr


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140326/a96ffd41/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.tuned.ini.5.mod
Type: audio/x-mod
Size: 1362 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140326/a96ffd41/attachment-0001.bin

------------------------------

Message: 3
Date: Wed, 26 Mar 2014 11:11:14 +0000
From: Michael me <therestlesstimer@gmail.com>
Subject: [Moses-support] cube pruning for phrase-based decoding
To: moses-support@mit.edu
Message-ID:
<CAKxaUbxM7Or9B-jSnXQ4t-Uoe5xctBa_NU9StVDoznx5JgSWiw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi All,

looking into the code, it appears to me that with cube pruning, the PB
decoder processes the stacks one by one until all stacks are processed,
which is the same procedure without cube pruning. The difference is that
you put all hypotheses through a priority queue before you drop them into
stacks further down.

But looking at this paper (Figure 5):

http://acl.ldc.upenn.edu/P/P07/P07-1019.pdf

it seems that with cube pruning a PB decoder will process multiple stacks
together. e.g. if we are currently at stack #5, then all hypotheses from
previous 4 stacks are expanded into stack #5 through a priority queue? Is
this how cube pruning for the PB decoder is implemented in Moses, or is it
not?

Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140326/e19822f1/attachment-0001.htm

------------------------------

Message: 4
Date: Tue, 25 Mar 2014 22:31:38 +0800
From: zock <zock@free.fr>
Subject: [Moses-support] Call for papers (CogALex-IV) workshop
co-located with Coling
To: corpora@uib.no, moses-support@mit.edu
Message-ID: <5331934A.3090601@free.fr>
Content-Type: text/plain; charset="us-ascii"

An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140325/27cf1ddf/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 89, Issue 69
*********************************************

Tweet