Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. how to filter syntactic rule table when using mert.pl (hxshi)
2. I can't get any output from my syntactic baseline. (hxshi)
3. Re: I can't get any output from my syntactic baseline.
(Matthias Huck)
4. Re: I can't get any output from my syntactic baseline. (hxshi)
----------------------------------------------------------------------
Message: 1
Date: Sun, 25 Jan 2015 02:26:16 +0800
From: hxshi <hxshi@mtlab.hit.edu.cn>
Subject: [Moses-support] how to filter syntactic rule table when using
mert.pl
To: moses-support <moses-support@mit.edu>
Message-ID: <201501250226158651875@mtlab.hit.edu.cn>
Content-Type: text/plain; charset="gb2312"
Good evening! I found that when I tuning a syntactic baseline, baseline output an empty filtered rule table. And I think it maybe because of the mert.pl using the wrong filter script. Am I write about?
PS. I already set the --inputtype 3 when I run mert.pl.
Shi Huaxing
MI&T Lab
School of Computer Science and Technology
Harbin Institute of Technology
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150125/70b10854/attachment-0001.htm
------------------------------
Message: 2
Date: Sun, 25 Jan 2015 03:25:43 +0800
From: hxshi <hxshi@mtlab.hit.edu.cn>
Subject: [Moses-support] I can't get any output from my syntactic
baseline.
To: moses-support <moses-support@mit.edu>
Message-ID: <201501250325431803769@mtlab.hit.edu.cn>
Content-Type: text/plain; charset="gb2312"
I can't get any output with my syntactic baseline. Will anybody know what maybe wrong?
I trained a string2tree baseline. Got a rule-table such like this:
% [X][TH] ?????? [X][RA] [X] ||| [X][TH] is [X][RA] [PROP] ||| 1.67586e-05 4.51106e-08 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.735177 0.735177 ||| |||
% [X][TH] ?????? [X][RA] ?? [X] ||| [X][TH] is [X][RA] [PROP] ||| 2.06243e-06 6.65709e-09 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.0904762 0.0904762 ||| |||
% [X][TH] ?????? [X][RA] ?? [X] ||| [X][TH] is [X][RA] [PROP] ||| 1.30259e-06 4.61662e-11 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.0571429 0.0571429 ||| |||
And I always got no output when I using this baseline.
for example :
input 3 ??
output on screen:
3 ??
Translating line 2 in thread id 47362102691584
Line 2: Initialize search took 0.000 seconds total
Translating: <s> 3 ?? </s> ||| [0,0]=X (1) [0,1]=X (1) [0,2]=X (1) [0,3]=X (1) [1,1]=X (1) [1,2]=X (1) [1,3]=X (1) [2,2]=X (1) [2,3]=X (1) [3,3]=X (1)
0 1 2 3
0 8 8 0
0 29 0
0 0
0
Line 2: Additional reporting took 0.000 seconds total
Line 2: Translation took 0.003 seconds total
Translation took 0.000 seconds
Do you know what maybe wrong with my baseline?
I run decoder with
moses_chart -T -f moses.ini
trainning this baseline with:
train_model.pl --glue-grammar --target-syntax -max-phrase-length=999 --extract-options="--NonTermConsecSource --MinHoleSource 1 --MaxSpan 999 --MinWords 0 --MaxNonTerm 3" -lm 0:5:lmsri.en --corpustrain_case --f zh --e en -root-dir train_dir -external-bin-dir bin -mgiza -mgiza-cpus 6 -cores 10 --alignment grow-diag-final-and -score-options ' --GoodTuring'
the moses.ini as following:
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
# feature functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/workspace/moses-fbis-case-s2t-ch2en/training_dir/model/rule-table.gz input-factor=0 output-factor=0
KENLM name=LM0 factor=0 path=/home/workspace/data-lm/lmsri.en order=5
# dense weights for feature functions
[weight]
UnknownWordPenalty0= 1
WordPenalty0= -1
PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2
LM0= 0.5
Shi Huaxing
MI&T Lab
School of Computer Science and Technology
Harbin Institute of Technology
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150125/54b32ba2/attachment-0001.htm
------------------------------
Message: 3
Date: Sat, 24 Jan 2015 20:04:33 +0000
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] I can't get any output from my syntactic
baseline.
To: hxshi <hxshi@mtlab.hit.edu.cn>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <1422129873.1814.1.camel@inf.ed.ac.uk>
Content-Type: text/plain; charset="UTF-8"
Hi,
As Rico pointed out before: the glue rules are missing.
Cheers,
Matthias
On Sun, 2015-01-25 at 03:25 +0800, hxshi wrote:
> I can't get any output with my syntactic baseline. Will anybody know
> what maybe wrong?
>
> I trained a string2tree baseline. Got a rule-table such like this:
>
> % [X][TH] ??
> ? [X][RA] [X] ||| [X][TH] is [X][RA] [PROP] ||| 1.67586e-05 4.51106e-08 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.735177 0.735177 ||| |||
> % [X][TH] ??
> ? [X][RA] ? [X] ||| [X][TH] is [X][RA] [PROP] ||| 2.06243e-06 6.65709e-09 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.0904762 0.0904762 ||| |||
> % [X][TH] ??
> ? [X][RA] ? [X] ||| [X][TH] is [X][RA] [PROP] ||| 1.30259e-06 4.61662e-11 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.0571429 0.0571429 ||| |||
>
> And I always got no output when I using this baseline.
> for example :
>
> input 3 ?
>
> output on screen:
> 3 ?
> Translating line 2 in thread id 47362102691584
> Line 2: Initialize search took 0.000 seconds total
> Translating: <s> 3 ? </s> ||| [0,0]=X (1) [0,1]=X (1) [0,2]=X (1) [0,3]=X (1) [1,1]=X (1) [1,2]=X (1) [1,3]=X (1) [2,2]=X (1) [2,3]=X (1) [3,3]=X (1)
>
> 0 1 2 3
> 0 8 8 0
> 0 29 0
> 0 0
> 0
> Line 2: Additional reporting took 0.000 seconds total
> Line 2: Translation took 0.003 seconds total
> Translation took 0.000 seconds
>
> Do you know what maybe wrong with my baseline?
>
> I run decoder with
> moses_chart -T -f moses.ini
>
> trainning this baseline with:
> train_model.pl
> --glue-grammar --target-syntax -max-phrase-length=999 --extract-options="--NonTermConsecSource --MinHoleSource 1 --MaxSpan 999 --MinWords 0 --MaxNonTerm 3" -lm 0:5:lmsri.en --corpustrain_case --f zh --e en -root-dir train_dir -external-bin-dir bin -mgiza -mgiza-cpus 6 -cores 10 --alignment grow-diag-final-and -score-options ' --GoodTuring'
>
> the moses.ini as following:
> #########################
>
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
>
> [cube-pruning-pop-limit]
> 1000
>
> [non-terminals]
> X
>
> [search-algorithm]
> 3
>
> [inputtype]
> 3
>
> [max-chart-span]
> 20
> 1000
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/workspace/moses-fbis-case-s2t-ch2en/training_dir/model/rule-table.gz input-factor=0 output-factor=0
> KENLM name=LM0 factor=0 path=/home/workspace/data-lm/lmsri.en order=5
>
> # dense weights for feature functions
> [weight]
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2
> LM0= 0.5
>
> ______________________________________________________________________
> Shi Huaxing
>
>
> MI&T Lab
> School of Computer Science and Technology
> Harbin Institute of Technology
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 4
Date: Sun, 25 Jan 2015 10:07:40 +0800
From: hxshi <hxshi@mtlab.hit.edu.cn>
Subject: Re: [Moses-support] I can't get any output from my syntactic
baseline.
To: "Matthias Huck" <mhuck@inf.ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <201501251007403137661@mtlab.hit.edu.cn>
Content-Type: text/plain; charset="gb2312"
Thank you for your advices.
now it can translate something now.
but when I run it , the translations are as following:
Yili </tree> <tree <tree <tree label="COND"> <tree <tree propaganda activities
label="AG"> Urumqi <tree label="PROP"> electricity -LRB- <tree ?????? </tree> ???? </tree> ?????? -RRB- label="TH"> <tree label="DET"> </tree> </tree> <tree <tree <tree label="VACTN">
<tree Yili creatively </tree> <tree <tree label="COND"> <tree <tree propaganda activities
<tree label="PROP"> <tree <tree <tree </tree> label="PROP"> label="MOD">
voice accorded a warm welcome
Located Xibei border </tree> Yili <tree </tree> </tree> <tree label="TH"> label="REL">
<tree <tree <tree <tree label="TH"> label="HEAD"> <tree label="HEAD"> <tree Yili </tree>
<tree <tree </tree> <tree label="HEAD"> </tree> <tree
they are not what I expected. What is the problem? how can I get the output as string. by the way, the out put even not a tree.
Shi Huaxing
MI&T Lab
School of Computer Science and Technology
Harbin Institute of Technology
From: Matthias Huck
Date: 2015-01-25 04:04
To: hxshi
CC: moses-support
Subject: Re: [Moses-support] I can't get any output from my syntactic baseline.
Hi,
As Rico pointed out before: the glue rules are missing.
Cheers,
Matthias
On Sun, 2015-01-25 at 03:25 +0800, hxshi wrote:
> I can't get any output with my syntactic baseline. Will anybody know
> what maybe wrong?
>
> I trained a string2tree baseline. Got a rule-table such like this:
>
> % [X][TH] ????
> ?? [X][RA] [X] ||| [X][TH] is [X][RA] [PROP] ||| 1.67586e-05 4.51106e-08 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.735177 0.735177 ||| |||
> % [X][TH] ????
> ?? [X][RA] ?? [X] ||| [X][TH] is [X][RA] [PROP] ||| 2.06243e-06 6.65709e-09 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.0904762 0.0904762 ||| |||
> % [X][TH] ????
> ?? [X][RA] ?? [X] ||| [X][TH] is [X][RA] [PROP] ||| 1.30259e-06 4.61662e-11 0.0475 0.177966 ||| 1-0 2-1 3-2 ||| 2083.76 0.0571429 0.0571429 ||| |||
>
> And I always got no output when I using this baseline.
> for example :
>
> input 3 ??
>
> output on screen:
> 3 ??
> Translating line 2 in thread id 47362102691584
> Line 2: Initialize search took 0.000 seconds total
> Translating: <s> 3 ?? </s> ||| [0,0]=X (1) [0,1]=X (1) [0,2]=X (1) [0,3]=X (1) [1,1]=X (1) [1,2]=X (1) [1,3]=X (1) [2,2]=X (1) [2,3]=X (1) [3,3]=X (1)
>
> 0 1 2 3
> 0 8 8 0
> 0 29 0
> 0 0
> 0
> Line 2: Additional reporting took 0.000 seconds total
> Line 2: Translation took 0.003 seconds total
> Translation took 0.000 seconds
>
> Do you know what maybe wrong with my baseline?
>
> I run decoder with
> moses_chart -T -f moses.ini
>
> trainning this baseline with:
> train_model.pl
> --glue-grammar --target-syntax -max-phrase-length=999 --extract-options="--NonTermConsecSource --MinHoleSource 1 --MaxSpan 999 --MinWords 0 --MaxNonTerm 3" -lm 0:5:lmsri.en --corpustrain_case --f zh --e en -root-dir train_dir -external-bin-dir bin -mgiza -mgiza-cpus 6 -cores 10 --alignment grow-diag-final-and -score-options ' --GoodTuring'
>
> the moses.ini as following:
> #########################
>
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
>
> [cube-pruning-pop-limit]
> 1000
>
> [non-terminals]
> X
>
> [search-algorithm]
> 3
>
> [inputtype]
> 3
>
> [max-chart-span]
> 20
> 1000
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/workspace/moses-fbis-case-s2t-ch2en/training_dir/model/rule-table.gz input-factor=0 output-factor=0
> KENLM name=LM0 factor=0 path=/home/workspace/data-lm/lmsri.en order=5
>
> # dense weights for feature functions
> [weight]
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2
> LM0= 0.5
>
> ______________________________________________________________________
> Shi Huaxing
>
>
> MI&T Lab
> School of Computer Science and Technology
> Harbin Institute of Technology
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150125/571ee1fe/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 99, Issue 58
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 99, Issue 58"
Post a Comment