Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. hierarchical-to-string translation doesn't produce string
output (Oleg Parashchenko)
2. Re: hierarchical-to-string translation doesn't produce string
output (Hieu Hoang)
3. Re: hierarchical-to-string translation doesn't produce string
output (Oleg Parashchenko)
----------------------------------------------------------------------
Message: 1
Date: Fri, 18 Mar 2022 04:54:41 +0100
From: Oleg Parashchenko <olpa@uucode.com>
To: <moses-support@mit.edu>
Subject: [Moses-support] hierarchical-to-string translation doesn't
produce string output
Message-ID: <20220318045441.f4e22592ada543072c115da2@uucode.com>
Content-Type: text/plain; charset=US-ASCII
Hello,
I've created and tried a hierarchical model:
```
$ echo '<tree label="N">nhoj</tree>' | moses -f model/moses.ini
...
Line 0: Initialize search took 0.000 seconds total
Translating: <s> nhoj </s> ||| [0,0]=X (1) [0,1]=X (1) [0,2]=X (1)
[1,1]=X (1) [1,1]=N (1) [1,2]=X (1) [2,2]=X (1)
0 1 2
0 1 0
0 0
0
Line 0: Additional reporting took 0.000 seconds total
Line 0: Translation took 0.001 seconds total
Translation took 0.001 seconds
...
```
Something has happened, but I don't see the expected output "john".
Adding the parameter "--inputtype 3" to moses doesn't change the result.
What I'm doing wrong?
Here is how I've built the model:
```
$ cat corpus.en
john
mary
hello
bye
hello john
$ cat corpus.ne
<tree label="N">nhoj</tree>
<tree label="N">yram</tree>
<tree label="V">olleh</tree>
<tree label="V">eyb</tree>
<tree label="VP"><tree label="V">olleh</tree> <tree
label="N">hnoj</tree></tree>
$ lmplz -o 2 --discount_fallback < corpus.en > lm.en
...
$ /opt/moses/scripts/training/train-model.perl \
-corpus corpus \
-f ne -e en \
-lm 0:2:$(pwd)/lm.en \
-alignment grow-diag-final-and \
-external-bin-dir /opt/tools \
-mgiza \
-hierarchical \
2>&1 | tee train-log.txt
$ cat model/rule-table.gz | gunzip -c
<tree label="N">nhoj</tree> [X] ||| john [X] ||| 0.5 0.277778 1 1 |||
1-0 ||| 2 1 1 ||| |||
...
```
Thanks!
Regards,
Oleg
------------------------------
Message: 2
Date: Thu, 17 Mar 2022 21:09:10 -0700
From: Hieu Hoang <hieuhoang@gmail.com>
To: Oleg Parashchenko <olpa@uucode.com>
Cc: moses-support <moses-support@mit.edu>
Subject: Re: [Moses-support] hierarchical-to-string translation
doesn't produce string output
Message-ID:
<CAEKMkbgYfWFxAW16aNnyszSgLP-Pu13DXVUU+ypkGYXiEA8x6A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Can I have a look at your moses.ini file. You always need a glue rule
phrase take for scfg models to create the final output
On Thu, Mar 17, 2022, 8:58 PM Oleg Parashchenko <olpa@uucode.com> wrote:
> Hello,
>
> I've created and tried a hierarchical model:
>
> ```
> $ echo '<tree label="N">nhoj</tree>' | moses -f model/moses.ini
> ...
> Line 0: Initialize search took 0.000 seconds total
> Translating: <s> nhoj </s> ||| [0,0]=X (1) [0,1]=X (1) [0,2]=X (1)
> [1,1]=X (1) [1,1]=N (1) [1,2]=X (1) [2,2]=X (1)
>
> 0 1 2
> 0 1 0
> 0 0
> 0
> Line 0: Additional reporting took 0.000 seconds total
> Line 0: Translation took 0.001 seconds total
> Translation took 0.001 seconds
> ...
> ```
>
> Something has happened, but I don't see the expected output "john".
> Adding the parameter "--inputtype 3" to moses doesn't change the result.
>
> What I'm doing wrong?
>
>
> Here is how I've built the model:
>
> ```
> $ cat corpus.en
> john
> mary
> hello
> bye
> hello john
>
> $ cat corpus.ne
> <tree label="N">nhoj</tree>
> <tree label="N">yram</tree>
> <tree label="V">olleh</tree>
> <tree label="V">eyb</tree>
> <tree label="VP"><tree label="V">olleh</tree> <tree
> label="N">hnoj</tree></tree>
>
> $ lmplz -o 2 --discount_fallback < corpus.en > lm.en
> ...
>
> $ /opt/moses/scripts/training/train-model.perl \
> -corpus corpus \
> -f ne -e en \
> -lm 0:2:$(pwd)/lm.en \
> -alignment grow-diag-final-and \
> -external-bin-dir /opt/tools \
> -mgiza \
> -hierarchical \
> 2>&1 | tee train-log.txt
>
> $ cat model/rule-table.gz | gunzip -c
> <tree label="N">nhoj</tree> [X] ||| john [X] ||| 0.5 0.277778 1 1 |||
> 1-0 ||| 2 1 1 ||| |||
> ...
> ```
>
> Thanks!
>
> Regards,
> Oleg
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> https://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.mit.edu/mailman/private/moses-support/attachments/20220317/1d329f1d/attachment-0001.htm>
------------------------------
Message: 3
Date: Fri, 18 Mar 2022 06:16:26 +0100
From: Oleg Parashchenko <olpa@uucode.com>
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Subject: Re: [Moses-support] hierarchical-to-string translation
doesn't produce string output
Message-ID: <20220318061626.8476da4dc8d78fc93a24660a@uucode.com>
Content-Type: text/plain; charset=US-ASCII
Hello Hieu,
> ... You always need a glue rule
I assumed that training should create one. If I need to write one self,
where can I read the details? The page
https://www.statmt.org/moses/?n=Moses.SyntaxTutorial mentions only the
using of glue rules.
> Can I have a look at your moses.ini file
I've put the whole model at
http://uucode.com/download/tmp/mini-hie-to-string.tar.gz,
here is the ini file:
```
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4
path=/share/mini-hie-to-string/model/rule-table.gz input-factor=0
output-factor=0 KENLM name=LM0 factor=0
path=/share/mini-hie-to-string/lm.en order=2
# dense weights for feature functions
[weight]
# The default weights are NOT optimized for translation quality. You MUST tune the weights.
# Documentation for tuning is here: http://www.statmt.org/moses/?n=FactoredTraining.Tuning
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
LM0= 0.5
```
Regards,
Oleg
On Thu, 17 Mar 2022 21:09:10 -0700
Hieu Hoang <hieuhoang@gmail.com> wrote:
> Can I have a look at your moses.ini file. You always need a glue rule
> phrase take for scfg models to create the final output
...
------------------------------
Subject: Digest Footer
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
https://mailman.mit.edu/mailman/listinfo/moses-support
------------------------------
End of Moses-support Digest, Vol 184, Issue 2
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 184, Issue 2"
Post a Comment