Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: File not found in Step 5 (extract phrases) (Hieu Hoang)
2. Normalization of string-to-tree rules (Fabienne Braune)
3. Re: File not found in Step 5 (extract phrases) (WANG Kun)
4. Do I need to sort reordering model generated by EMS before
binarizing with processLexicalTableMin? (Jeremy Gwinnup)
5. Re: Do I need to sort reordering model generated by EMS
before binarizing with processLexicalTableMin?
(Marcin Junczys-Dowmunt)
6. Re: Normalization of string-to-tree rules (Rico Sennrich)
----------------------------------------------------------------------
Message: 1
Date: Thu, 13 Aug 2015 17:17:59 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] File not found in Step 5 (extract
phrases)
To: Kun Wang <kunwang@nlpr.ia.ac.cn>, moses-support
<moses-support@mit.edu>
Message-ID: <55CC9907.5010606@gmail.com>
Content-Type: text/plain; charset="windows-1252"
what OS are you using, and what do you get when you run
split --version
It looks like you may have your own split command. Moses requires the
GNU split or the BSD version
On 13/08/2015 16:55, Kun Wang wrote:
> erl 32 split "sort " /home/kwang/kw2T/decoder/mosesdecoder/scripts/../bin/extract-rules /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/corpus/fbis.eng /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/corpus/fbis.chn /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/model/aligned.grow-diag-final-and /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/model/extract --GlueGrammar /home/kwang/kw2T/s
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150813/6923a502/attachment-0001.htm
------------------------------
Message: 2
Date: Thu, 13 Aug 2015 16:22:14 +0200
From: Fabienne Braune <braune.fabienne@gmail.com>
Subject: [Moses-support] Normalization of string-to-tree rules
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAOLzUJa-sPbu1OgkD9Yi_v0se58DH67eq-8NQAfixo0mw4Y5Hw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hi everyone,
I have a doubt about how rule scoring for GHKM string-to-tree rules is
implemented in the current version of Moses. Are the scores normalized
over the source side of the rules only (so rules 1) and 2) would be
competing) or source side and aligned target non-terminals (so 1) and
2) are not competing)?
1) X/NP -> der X Hund, the ADJ dog
2) X/NP -> der X Hund, the NN dog
What is the reasoning behind the implemented normalization?
Thanks a lot for the help!
Cheers,
Fabienne
------------------------------
Message: 3
Date: Thu, 13 Aug 2015 23:29:17 +0800 (GMT+08:00)
From: "WANG Kun" <kunwang@nlpr.ia.ac.cn>
Subject: Re: [Moses-support] File not found in Step 5 (extract
phrases)
To: "Hieu Hoang" <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <1e0f0ba.20515.14f27adfb26.Coremail.kunwang@nlpr.ia.ac.cn>
Content-Type: text/plain; charset="gbk"
Hi Hieu,
Yes, the problem is that there is another split command in /usr/local/bin.
The error has been corrected by removing the split command in /usr/local/bin.
Thank you very much for your help!
Best regards,
Kun Wang
-----????????-----
??????: "Hieu Hoang" <hieuhoang@gmail.com>
????????: 2015??8??13?? ??????
??????: "Kun Wang" <kunwang@nlpr.ia.ac.cn>, moses-support <moses-support@mit.edu>
????:
????: Re: [Moses-support] File not found in Step 5 (extract phrases)
what OS are you using, and what do you get when you run
split --version
It looks like you may have your own split command. Moses requires the GNU split or the BSD version
On 13/08/2015 16:55, Kun Wang wrote:
erl 32 split "sort " /home/kwang/kw2T/decoder/mosesdecoder/scripts/../bin/extract-rules /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/corpus/fbis.eng /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/corpus/fbis.chn /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/model/aligned.grow-diag-final-and /home/kwang/kw2T/smt-work-dir/moses-hiero-fbis-dir/model/extract --GlueGrammar /home/kwang/kw2T/s
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
--
??????
????
========================================================
???? Wang Kun
National Laboratory of Pattern Recognition (NLPR)
Institute of Automation, Chinese Academy of Sciences
Beijing??China
Tel??8610 82614468
Email 1: kunwang@nlpr.ia.ac.cn
Email 2: jasonkw19@gmail.com
=========================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150813/ec1240e1/attachment-0001.htm
------------------------------
Message: 4
Date: Thu, 13 Aug 2015 11:31:15 -0400
From: Jeremy Gwinnup <jeremy@gwinnup.org>
Subject: [Moses-support] Do I need to sort reordering model generated
by EMS before binarizing with processLexicalTableMin?
To: moses-support@mit.edu
Message-ID: <179C5C8C-27FB-4869-93D0-D0116C39911B@gwinnup.org>
Content-Type: text/plain; charset=utf-8
Hi,
I?ve created a reordering model via an EMS script and wish to binarize outside of EMS with processLexicalTableMin - Do I still need to sort before binarizing?
Thanks!
-Jeremy
------------------------------
Message: 5
Date: Thu, 13 Aug 2015 17:45:37 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Do I need to sort reordering model
generated by EMS before binarizing with processLexicalTableMin?
To: Jeremy Gwinnup <jeremy@gwinnup.org>
Cc: moses-support@mit.edu
Message-ID: <1f6ed1597c0c8fd09e5d1cb82f004c48@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"
Hi Jeremy,
I believe reordering models come sorted out of the EMS process, so it
should just work if nothing else has been done to the model. Otherwise
the binarization tool will complain, so it will tell you to sort if it
is necessary.
Best,
Marcin
W dniu 2015-08-13 17:31, Jeremy Gwinnup napisa?(a):
> Hi,
>
> I've created a reordering model via an EMS script and wish to binarize outside of EMS with processLexicalTableMin - Do I still need to sort before binarizing?
>
> Thanks!
> -Jeremy
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]
Links:
------
[1] http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150813/0d0f5b46/attachment-0001.htm
------------------------------
Message: 6
Date: Thu, 13 Aug 2015 17:51:08 +0100
From: Rico Sennrich <rico.sennrich@gmx.ch>
Subject: Re: [Moses-support] Normalization of string-to-tree rules
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <55CCCAFC.8060805@gmx.ch>
Content-Type: text/plain; charset=windows-1252; format=flowed
Hi Fabienne,
there are three different implementations for GHKM extraction
p(LHS,RHS_t|RHS_s,target_nonterminals) (default)
p(RHS_t|RHS_s,LHS) (-alt-direct-rule-score-1)
p(LHS,RHS_t|RHS_s) (-alt-direct-rule-score-2)
by default, your rules 1 and 2 are not competing, because normalization
is over the source and aligned target non-terminals. I agree that this
is a bit weird. We generally use -alt-direct-rule-score-2. Here's some
example configs:
https://github.com/rsennrich/wmt2014-scripts/tree/master/example
best wishes,
Rico
On 13.08.2015 15:22, Fabienne Braune wrote:
> Hi everyone,
>
> I have a doubt about how rule scoring for GHKM string-to-tree rules is
> implemented in the current version of Moses. Are the scores normalized
> over the source side of the rules only (so rules 1) and 2) would be
> competing) or source side and aligned target non-terminals (so 1) and
> 2) are not competing)?
>
> 1) X/NP -> der X Hund, the ADJ dog
> 2) X/NP -> der X Hund, the NN dog
>
> What is the reasoning behind the implemented normalization?
>
> Thanks a lot for the help!
>
> Cheers,
> Fabienne
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 106, Issue 29
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 106, Issue 29"
Post a Comment