Home » Uncategories » Moses-support Digest, Vol 93, Issue 23

Moses-support Digest, Vol 93, Issue 23

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Problem with Hierarchical Model (Nisheeth Joshi)
2. Re: Phrase-table smoothing? (Hieu Hoang)
3. Re: Problem with Hierarchical Model (Hieu Hoang)
4. Re: Phrase-table smoothing? (Marcin Junczys-Dowmunt)
5. Re: Phrase-table smoothing? (Matthias Huck)


----------------------------------------------------------------------

Message: 1
Date: Sun, 20 Jul 2014 00:10:20 +0530
From: Nisheeth Joshi <njoshi.in@gmail.com>
Subject: [Moses-support] Problem with Hierarchical Model
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAPf3NF=DHNs_MCh4cumC_YJ4LAuMtA9XfBPbwDdAvkV_-3oybg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi

I am trying my hands on hierarchical model. I have trained the system using:

$MOSES/scripts/training/train-model.perl -root-dir moses-h/train
-corpus corpus/ilc.clean -f en -e hi -hierarchical -glue-grammar
-max-phrase-length 5 --score-options="--GoodTuring" -alignment
grow-diag-final-and -lm 0:3:$EnHi/lm/ilc.blm.hi:8 -external-bin-dir
$GIZA


But when I try to translate a sentence, I get nothing. I am enclosing
the file which has the output. The command that I have used is:

$MOSES/bin/moses -f moses-h/train/model/moses.ini

The above command works fine with phrase based model.


Kindly me know my mistake.

Thank you.

Sincerely,
Nisheeth
-------------- next part --------------
Defined parameters (per moses.ini or switch):
config: moses-h/train/model/moses.ini
cube-pruning-pop-limit: 1000
feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/home/nisheeth/MT/engines/en-hi/moses-h/train/model/rule-table.gz input-factor=0 output-factor=0 PhraseDictionaryMemory name=TranslationModel1 num-features=1 path=/home/nisheeth/MT/engines/en-hi/moses-h/train/model/glue-grammar input-factor=0 output-factor=0 IRSTLM name=LM0 factor=0 path=/home/nisheeth/MT/engines/en-hi/lm/ilc.blm.hi order=3
input-factors: 0
inputtype: 3
mapping: 0 T 0 1 T 1
max-chart-span: 20 1000
non-terminals: X
search-algorithm: 3
weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 TranslationModel1= 1.0 LM0= 0.5
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/home/nisheeth/MT/engines/en-hi/moses-h/train/model/rule-table.gz input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=PhraseDictionaryMemory name=TranslationModel1 num-features=1 path=/home/nisheeth/MT/engines/en-hi/moses-h/train/model/glue-grammar input-factor=0 output-factor=0
FeatureFunction: TranslationModel1 start: 7 end: 7
line=IRSTLM name=LM0 factor=0 path=/home/nisheeth/MT/engines/en-hi/lm/ilc.blm.hi order=3
FeatureFunction: LM0 start: 8 end: 8
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LM0
In LanguageModelIRST::Load: nGramOrder = 3
Language Model Type of /home/nisheeth/MT/engines/en-hi/lm/ilc.blm.hi is 1
Language Model Type is 1
mmap
loadtxt_ram()
done
starting to use OOV words [<unk>]
OOV code is 0
OOV code is 0
IRST: m_unknownId=0
Loading TranslationModel0
Start loading text phrase table. Moses format : [0.036] seconds
Reading /home/nisheeth/MT/engines/en-hi/moses-h/train/model/rule-table.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Loading TranslationModel1
Start loading text phrase table. Moses format : [19.850] seconds
Reading /home/nisheeth/MT/engines/en-hi/moses-h/train/model/glue-grammar
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
max-chart-span: 20
max-chart-span: 1000
IO from STDOUT/STDIN
Created input-output object : [19.855] seconds
Unknown input type: 3
reset mmap
Name:moses VmPeak:392608 kB VmRSS:350876 kB RSSMax:379512 kB user:20.607 sys:0.321 CPU:20.928 real:20.939

------------------------------

Message: 2
Date: Sat, 19 Jul 2014 20:16:41 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Phrase-table smoothing?
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjOe+g3hRy9NcsEV9yFUR3zV7orQa3PqWWjaL66RYanhg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

../train-model.perl -score-options '--GoodTuring' ....

I didn't know the reordering scores were smoothed


On 19 July 2014 10:00, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:

> Hi,
> did the moses training script at some point support phrase-table
> smoothing or did I hallucinate this? Currently, I am only seeing
> smoothing for reordering tables...
> Best,
> Marcin
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140719/c72d6701/attachment-0001.htm

------------------------------

Message: 3
Date: Sat, 19 Jul 2014 20:19:23 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Problem with Hierarchical Model
To: Nisheeth Joshi <njoshi.in@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbg8y+TZgXktio_kmFunBKc+3-tW352WLZCNLoeSscE7Rw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Did you provide an input sentence to the decoder? eg.
$MOSES/bin/moses_chart -i input.txt ...
or
$MOSES/bin/moses_chart ... < input.txt

Also, note that the decoder for the hierarchical model is
moses_chart
not
moses

Can you send me the moses.ini file you're using




On 19 July 2014 19:40, Nisheeth Joshi <njoshi.in@gmail.com> wrote:

> Hi
>
> I am trying my hands on hierarchical model. I have trained the system
> using:
>
> $MOSES/scripts/training/train-model.perl -root-dir moses-h/train
> -corpus corpus/ilc.clean -f en -e hi -hierarchical -glue-grammar
> -max-phrase-length 5 --score-options="--GoodTuring" -alignment
> grow-diag-final-and -lm 0:3:$EnHi/lm/ilc.blm.hi:8 -external-bin-dir
> $GIZA
>
>
> But when I try to translate a sentence, I get nothing. I am enclosing
> the file which has the output. The command that I have used is:
>
> $MOSES/bin/moses -f moses-h/train/model/moses.ini
>
> The above command works fine with phrase based model.
>
>
> Kindly me know my mistake.
>
> Thank you.
>
> Sincerely,
> Nisheeth
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140719/0e02423a/attachment-0001.htm

------------------------------

Message: 4
Date: Sat, 19 Jul 2014 21:21:53 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Phrase-table smoothing?
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <53CAC551.8080709@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed

Thanks,
there is an option "'reordering-smooth" in the script, might be
something else.

W dniu 19.07.2014 21:16, Hieu Hoang pisze:
> ../train-model.perl -score-options '--GoodTuring' ....
>
> I didn't know the reordering scores were smoothed
>
>
> On 19 July 2014 10:00, Marcin Junczys-Dowmunt <junczys@amu.edu.pl
> <mailto:junczys@amu.edu.pl>> wrote:
>
> Hi,
> did the moses training script at some point support phrase-table
> smoothing or did I hallucinate this? Currently, I am only seeing
> smoothing for reordering tables...
> Best,
> Marcin
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>



------------------------------

Message: 5
Date: Sat, 19 Jul 2014 23:45:53 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] Phrase-table smoothing?
To: moses-support <moses-support@mit.edu>
Message-ID: <1405809953.1393.1.camel@hucklap.site>
Content-Type: text/plain; charset="UTF-8"

The reordering model is typically smoothed with priors of the
orientation classes.


On Sat, 2014-07-19 at 21:21 +0200, Marcin Junczys-Dowmunt wrote:
> Thanks,
> there is an option "'reordering-smooth" in the script, might be
> something else.
>
> W dniu 19.07.2014 21:16, Hieu Hoang pisze:
> > ../train-model.perl -score-options '--GoodTuring' ....
> >
> > I didn't know the reordering scores were smoothed
> >
> >
> > On 19 July 2014 10:00, Marcin Junczys-Dowmunt <junczys@amu.edu.pl
> > <mailto:junczys@amu.edu.pl>> wrote:
> >
> > Hi,
> > did the moses training script at some point support phrase-table
> > smoothing or did I hallucinate this? Currently, I am only seeing
> > smoothing for reordering tables...
> > Best,
> > Marcin
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > --
> > Hieu Hoang
> > Research Associate
> > University of Edinburgh
> > http://www.hoang.co.uk/hieu
> >
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 93, Issue 23
*********************************************

0 Response to "Moses-support Digest, Vol 93, Issue 23"

Post a Comment

Subscribe to: Post Comments (Atom)