Moses-support Digest, Vol 108, Issue 29

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. how to find merge_alignment.py at train-model perl (???)
2. Re: how to find merge_alignment.py at train-model perl
(Hieu Hoang)
3. Re: Issue with alignment (gang tang)


----------------------------------------------------------------------

Message: 1
Date: Fri, 9 Oct 2015 12:40:07 +0300
From: ??? <fountain1128@gmail.com>
Subject: [Moses-support] how to find merge_alignment.py at train-model
perl
To: moses-support@mit.edu
Message-ID: <4D6FB46E-59D7-45B3-9AED-C8E67E854A5A@gmail.com>
Content-Type: text/plain; charset="gb2312"

Hi all,

I?ve done the corpus preparation and language model training with my own data.
However, the training of translation system doesn?t go very well. The error massage says "Cannot find /opt/mosesdecoder/tools/merge_alignment.py at /opt/mosesdecoder/scripts/training/train-model.perl line 384."

But they are just at the right place as the following text shows.

I?m using Mgiza instead of the default one on the website.
Both of the Mosesdecoder and MGIZA are installed at the admin?s directory.

fangting@phon:/opt/mosesdecoder$ ls
bin scripts tools
fangting@phon:/opt/mosesdecoder$ cd tools/
fangting@phon:/opt/mosesdecoder/tools$ ls
d4norm merge_alignment.py run.sh sntpostproc.py
force-align-moses-old.sh mgiza snt2cooc symal
force-align-moses.sh mkcls snt2cooc.pl
giza2bal.pl plain2snt snt2coocrmp
hmmnorm plain2snt-hasvcb.py snt2plain

fangting@phon:~/working$ nohup nice $MOSES/scripts/training/train-model.perl -root-dir train -corpus ~/es-fi/OpenSubtitles2013.es-fi.clean -f es -e fi -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:$HOME/lm/OpenSubtitles2013.es-fi.blm.fi:8 -mgiza -mgiza-cpus 8 -external-bin-dir $MOSES/tools >& training.out &
[1] 22362
fangting@phon:~/working$ ls
training.out
[1]+ Exit 255 nohup nice $MOSES/scripts/training/train-model.perl -root-dir train -corpus ~/es-fi/OpenSubtitles2013.es-fi.clean -f es -e fi -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:$HOME/lm/OpenSubtitles2013.es-fi.blm.fi:8 -mgiza -mgiza-cpus 8 -external-bin-dir $MOSES/tools &> training.out
fangting@phon:~/working$ cat training.out
nohup: ignoring input
Using SCRIPTS_ROOTDIR: /opt/mosesdecoder/scripts
Using multi-thread GIZA
ERROR: Cannot find /opt/mosesdecoder/tools/merge_alignment.py at /opt/mosesdecoder/scripts/training/train-model.perl line 384.

Best,
Fangting
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151009/4a95898b/attachment-0001.html

------------------------------

Message: 2
Date: Fri, 9 Oct 2015 10:49:37 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] how to find merge_alignment.py at
train-model perl
To: ??? <fountain1128@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbifCM3e3dh3idMcU6ZDg+9OSJJFvZLgeU2EURy5BbotUA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

strange. Is the file executable?

Hieu Hoang
http://www.hoang.co.uk/hieu

On 9 October 2015 at 10:40, ??? <fountain1128@gmail.com> wrote:

> Hi all,
>
> I?ve done the corpus preparation and language model training with my own
> data.
> However, the training of translation system doesn?t go very well. The
> error massage says "Cannot find
> /opt/mosesdecoder/tools/merge_alignment.py at
> /opt/mosesdecoder/scripts/training/train-model.perl line 384."
>
> But they are just at the right place as the following text shows.
>
> I?m using Mgiza instead of the default one on the website.
> Both of the Mosesdecoder and MGIZA are installed at the admin?s directory.
>
> fangting@phon:/opt/mosesdecoder$ ls
> *bin* *scripts* *tools*
> fangting@phon:/opt/mosesdecoder$ cd tools/
> fangting@phon:/opt/mosesdecoder/tools$ ls
> *d4norm* *merge_alignment.py* *run.sh*
> *sntpostproc.py*
> *force-align-moses-old.sh* *mgiza* *snt2cooc* *symal*
> *force-align-moses.sh* *mkcls* *snt2cooc.pl
> <http://snt2cooc.pl>*
> *giza2bal.pl <http://giza2bal.pl>* *plain2snt*
> *snt2coocrmp*
> *hmmnorm* *plain2snt-hasvcb.py* *snt2plain*
>
> fangting@phon:~/working$ nohup nice
> $MOSES/scripts/training/train-model.perl -root-dir train -corpus ~/es-fi/
> OpenSubtitles2013.es-fi.clean -f es -e fi -alignment grow-diag-final-and
> -reordering msd-bidirectional-fe -lm 0:3:$HOME/lm/
> OpenSubtitles2013.es-fi.blm.fi:8 -mgiza -mgiza-cpus 8 -external-bin-dir
> $MOSES/tools >& training.out &
> [1] 22362
> fangting@phon:~/working$ ls
> training.out
> [1]+ Exit 255 nohup nice
> $MOSES/scripts/training/train-model.perl -root-dir train -corpus ~/es-fi/
> OpenSubtitles2013.es-fi.clean -f es -e fi -alignment grow-diag-final-and
> -reordering msd-bidirectional-fe -lm 0:3:$HOME/lm/
> OpenSubtitles2013.es-fi.blm.fi:8 -mgiza -mgiza-cpus 8 -external-bin-dir
> $MOSES/tools &> training.out
> fangting@phon:~/working$ cat training.out
> nohup: ignoring input
> Using SCRIPTS_ROOTDIR: /opt/mosesdecoder/scripts
> Using multi-thread GIZA
> ERROR: Cannot find /opt/mosesdecoder/tools/merge_alignment.py at
> /opt/mosesdecoder/scripts/training/train-model.perl line 384.
>
> Best,
> Fangting
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151009/f2864d1f/attachment-0001.html

------------------------------

Message: 3
Date: Fri, 9 Oct 2015 19:02:28 +0800 (CST)
From: "gang tang" <gangtang2014@126.com>
Subject: Re: [Moses-support] Issue with alignment
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <689d49ae.f062.1504c443129.Coremail.gangtang2014@126.com>
Content-Type: text/plain; charset="gbk"

Dear All,

Since there are no answers to my questions, I assume that there are no easy fixes to the alignment problem. However, just out of curiosity, shouldn't there be alignment tools that take lexical considerations into account while aligning parallel corpus? I mean, alignment tools that look up translations for specific words in a domain-specifc dictionary during alignment? Could there be any reason that it is not an interesting area to explore?


Best Regards,


Gang


? 2015-09-25 19:34:13?"gang tang" <gangtang2014@126.com> ???

Dear all,

I have a problem with alignment. I'd greatly appreciate if anyone can help solve my issue.

I have the following corpus:

?sandalo camufluge" -> "camufluge sandal"
"sandalo daino" -> "daino sandal"
"sandalo madras" -> "madras sandal"
"sandalo vernice" -> "vernice sandal"

The alignment software I used was GIZA++, and the alignment result was always 0-0 1-1, which meant that "sandalo" wasn't aligned with "sandal". And after training phrase.translation.table always had entries such as "sandalo" -> "camufluge", "sandalo" -> "daino", "sandalo"->"madras", and "sandalo"->"vernice", and no "sandalo"->"sandal". Is there any way this problem could be solved? Could I add more data to align "sandalo" with "sandal" and translate "sandalo" to "sandal"? How should I tune the system?

Thanks for your attention,

Gang







????iPhone6s???5288???????
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151009/dcf6c1cb/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 108, Issue 29
**********************************************

Tweet