Moses-support Digest, Vol 124, Issue 16

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Output files of mgiza (Matthias Huck)
2. Re: Output files of mgiza (Tom McCoy)
3. Re: Problem in Tuning (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Mon, 13 Feb 2017 18:22:46 +0100
From: Matthias Huck <mhuck@cis.lmu.de>
Subject: Re: [Moses-support] Output files of mgiza
To: Hieu Hoang <hieuhoang@gmail.com>, Tom McCoy
<richard.mccoy@yale.edu>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <1487006566.25930.75.camel@cis.lmu.de>
Content-Type: text/plain; charset="UTF-8"

Hi,

mgiza can be configured to write a Model 1 file to disk.
Use the configuration option "model1dumpfrequency".

https://web.archive.org/web/20150919195919/http://www.kyloo.net/software/doku.php/mgiza:configure

Cheers,
Matthias


On Mon, 2017-02-13 at 16:50 +0000, Hieu Hoang wrote:
> the slide refers to giza++, not mgiza. I didn't know there was substantial
> difference between them, but then I could be wrong.
>
> code for giza++ is here if you want to try it
> https://github.com/moses-smt/giza-pp
>
> Hieu Hoang
> http://moses-smt.org/
>
> On 13 February 2017 at 16:46, Tom McCoy <richard.mccoy@yale.edu> wrote:
>
> > Thanks for the quick reply! What I mostly want is the translation
> > probabilities that are learned for IBM Model 1--e.g., the probability that *hello
> > *will be aligned with *bonjour, *i.e. t(*hello*|*bonjour*). In Giza++, at
> > least, these probabilities are given in output files called *ti.final or
> > *ti.actual.final. These files are discussed on slide 30 of this
> > presentation: https://www.cse.iitb.ac.in/~anoopk/
> > publications/presentations/moses_giza_intro.pdf
> > Is there any way to access these files in mgiza?
> >
> > On Mon, Feb 13, 2017 at 6:41 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
> >
> > > I'm not sure if there's any such thing as 'final' or 'actual.final'. Your
> > > output seems to match the output from a typical run
> > >
> > > http://www.statmt.org/moses/RELEASE-3.0/models/fr-en/training/giza.1/
> > >
> > > The file 'A3.final' is used by the subsequent symmetrization step
> > >
> > > On 11/02/2017 19:42, Tom McCoy wrote:
> > >
> > > Hi,
> > >
> > > I am using mgiza on Mac OSX. It runs without errors and gives the
> > > following output files:
> > >
> > > - 117-02-11.134300.tommccoy.A3.final.part000
> > > - 117-02-11.134300.tommccoy.A3.final.part001
> > > - 117-02-11.134300.tommccoy.A3.final.part002
> > > - 117-02-11.134300.tommccoy.A3.final.part003
> > > - 117-02-11.134300.tommccoy.Decoder.config
> > > - 117-02-11.134300.tommccoy.a3.final
> > > - 117-02-11.134300.tommccoy.d3.final
> > > - 117-02-11.134300.tommccoy.d4.final
> > > - 117-02-11.134300.tommccoy.gizacfg
> > > - 117-02-11.134300.tommccoy.n3.final
> > > - 117-02-11.134300.tommccoy.p0_3.final
> > > - 117-02-11.134300.tommccoy.perpt3.final
> > >
> > > However, this is missing the *.ti.final and *.actual.ti.final output
> > > files, both of which I need. Any thoughts on how to get these files to be
> > > outputted?
> > >
> > > Thanks!
> > > Tom McCoy
> > >
> > >
> > > _______________________________________________
> > > Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > >
> > > --
> > > Hieu Hoanghttp://moses-smt.org/
> > >
> > >
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 2
Date: Mon, 13 Feb 2017 13:03:42 -0500
From: Tom McCoy <richard.mccoy@yale.edu>
Subject: Re: [Moses-support] Output files of mgiza
To: Matthias Huck <mhuck@cis.lmu.de>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAF=ri2++wf0sNTVFyAz-0tgiETsmeR-LRu2cV9tFOa7HDtYVjQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

>
> mgiza can be configured to write a Model 1 file to disk.
> Use the configuration option "model1dumpfrequency".


That did the trick--thanks so much!
Tom

On Mon, Feb 13, 2017 at 12:22 PM, Matthias Huck <mhuck@cis.lmu.de> wrote:

> Hi,
>
> mgiza can be configured to write a Model 1 file to disk.
> Use the configuration option "model1dumpfrequency".
>
> https://web.archive.org/web/20150919195919/http://www.
> kyloo.net/software/doku.php/mgiza:configure
>
> Cheers,
> Matthias
>
>
> On Mon, 2017-02-13 at 16:50 +0000, Hieu Hoang wrote:
> > the slide refers to giza++, not mgiza. I didn't know there was
> substantial
> > difference between them, but then I could be wrong.
> >
> > code for giza++ is here if you want to try it
> > https://github.com/moses-smt/giza-pp
> >
> > Hieu Hoang
> > http://moses-smt.org/
> >
> > On 13 February 2017 at 16:46, Tom McCoy <richard.mccoy@yale.edu> wrote:
> >
> > > Thanks for the quick reply! What I mostly want is the translation
> > > probabilities that are learned for IBM Model 1--e.g., the probability
> that *hello
> > > *will be aligned with *bonjour, *i.e. t(*hello*|*bonjour*). In Giza++,
> at
> > > least, these probabilities are given in output files called *ti.final
> or
> > > *ti.actual.final. These files are discussed on slide 30 of this
> > > presentation: https://www.cse.iitb.ac.in/~anoopk/
> > > publications/presentations/moses_giza_intro.pdf
> > > Is there any way to access these files in mgiza?
> > >
> > > On Mon, Feb 13, 2017 at 6:41 AM, Hieu Hoang <hieuhoang@gmail.com>
> wrote:
> > >
> > > > I'm not sure if there's any such thing as 'final' or 'actual.final'.
> Your
> > > > output seems to match the output from a typical run
> > > >
> > > > http://www.statmt.org/moses/RELEASE-3.0/models/fr-en/
> training/giza.1/
> > > >
> > > > The file 'A3.final' is used by the subsequent symmetrization step
> > > >
> > > > On 11/02/2017 19:42, Tom McCoy wrote:
> > > >
> > > > Hi,
> > > >
> > > > I am using mgiza on Mac OSX. It runs without errors and gives the
> > > > following output files:
> > > >
> > > > - 117-02-11.134300.tommccoy.A3.final.part000
> > > > - 117-02-11.134300.tommccoy.A3.final.part001
> > > > - 117-02-11.134300.tommccoy.A3.final.part002
> > > > - 117-02-11.134300.tommccoy.A3.final.part003
> > > > - 117-02-11.134300.tommccoy.Decoder.config
> > > > - 117-02-11.134300.tommccoy.a3.final
> > > > - 117-02-11.134300.tommccoy.d3.final
> > > > - 117-02-11.134300.tommccoy.d4.final
> > > > - 117-02-11.134300.tommccoy.gizacfg
> > > > - 117-02-11.134300.tommccoy.n3.final
> > > > - 117-02-11.134300.tommccoy.p0_3.final
> > > > - 117-02-11.134300.tommccoy.perpt3.final
> > > >
> > > > However, this is missing the *.ti.final and *.actual.ti.final output
> > > > files, both of which I need. Any thoughts on how to get these files
> to be
> > > > outputted?
> > > >
> > > > Thanks!
> > > > Tom McCoy
> > > >
> > > >
> > > > _______________________________________________
> > > > Moses-support mailing listMoses-support@mit.eduhttp://
> mailman.mit.edu/mailman/listinfo/moses-support
> > > >
> > > >
> > > > --
> > > > Hieu Hoanghttp://moses-smt.org/
> > > >
> > > >
> > >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170213/41cd808a/attachment-0001.html

------------------------------

Message: 3
Date: Tue, 14 Feb 2017 11:01:37 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Problem in Tuning
To: Arpita Dutta <duttaarpi@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDAGyCddjef+86buORSAB1hrgfAoNnGTmLVe12fjBESLiA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

you are specifying the use of the IRST language modeling toolkit, but
have not compiled Moses with ut.

-phi

On Mon, Feb 13, 2017 at 1:10 AM, Arpita Dutta <duttaarpi@gmail.com> wrote:
> Hi
>
> I installed Moses on Ubuntu 14.10 and run the training corpus part
> successfully with
> the English-bengali corpus but now when I am running tuning command
> I am getting the following error,
>
>
> nits@cse:~/Desktop/ad/demo$
> /home/nits/Desktop/ad/mosesdecoder/scripts/training/mert-moses.pl
> /home/nits/Desktop/ad/demo/data/tuning.en
> /home/nits/Desktop/ad/demo/data/tuning.ben
> /home/nits/Desktop/ad/mosesdecoder/bin/moses
> /home/nits/Desktop/ad/demo/model/moses.ini --working-dir
> /home/nits/Desktop/ad/demo/tuning --rootdir
> /home/nits/Desktop/ad/mosesdecoder/scripts
> Using SCRIPTS_ROOTDIR: /home/nits/Desktop/ad/mosesdecoder/scripts
> Assuming --mertdir=/home/nits/Desktop/ad/mosesdecoder/bin
> Assuming the tables are already filtered, reusing filtered/moses.ini
> Asking moses for feature names and values from filtered/moses.ini
> Executing: /home/nits/Desktop/ad/mosesdecoder/bin/moses -config
> filtered/moses.ini -inputtype 0 -show-weights
> exec: /home/nits/Desktop/ad/mosesdecoder/bin/moses -config
> filtered/moses.ini -inputtype 0 -show-weights
> Executing: /home/nits/Desktop/ad/mosesdecoder/bin/moses -config
> filtered/moses.ini -inputtype 0 -show-weights > ./features.list 2> /dev/null
> Exit code: 1
> ERROR: Failed to run '/home/nits/Desktop/ad/mosesdecoder/bin/moses -config
> filtered/moses.ini -inputtype 0 -show-weights'. at
> /home/nits/Desktop/ad/mosesdecoder/scripts/training/mert-moses.pl line 1748.
>
> nits@cse:~/Desktop/ad/demo$ /home/nits/Desktop/ad/mosesdecoder/bin/moses -f
> /home/nits/Desktop/ad/demo/tuning/filtered/moses.ini -i
> /home/nits/Desktop/ad/demo/tuning/filtered/input.14464
> Defined parameters (per moses.ini or switch):
> config: /home/nits/Desktop/ad/demo/tuning/filtered/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=/home/nits/Desktop/ad/demo/tuning/filtered/phrase-table.0-0.1.1.gz
> input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0
> num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0
> output-factor=0
> path=/home/nits/Desktop/ad/demo/tuning/filtered/reordering-table.wbe-msd-bidirectional-fe.0-0.1
> Distortion IRSTLM name=LM0 factor=0
> path=/home/nits/Desktop/ad/demo/lm/train.ilm.gz order=3
> input-factors: 0
> input-file: /home/nits/Desktop/ad/demo/tuning/filtered/input.14464
> mapping: 0 T 0
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3
> 0.3 Distortion0= 0.3 LM0= 0.5
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=/home/nits/Desktop/ad/demo/tuning/filtered/phrase-table.0-0.1.1.gz
> input-factor=0 output-factor=0
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/nits/Desktop/ad/demo/tuning/filtered/reordering-table.wbe-msd-bidirectional-fe.0-0.1
> Initializing Lexical Reordering Feature..
> FeatureFunction: LexicalReordering0 start: 7 end: 12
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 13
> line=IRSTLM name=LM0 factor=0
> path=/home/nits/Desktop/ad/demo/lm/train.ilm.gz order=3
> Exception: moses/FF/Factory.cpp:388 in void
> Moses::FeatureRegistry::Construct(const string&, const string&) threw
> UnknownFeatureException because `i == registry_.end()'.
> Feature name IRSTLM is not registered.
>
>
> Please help me.
>
> Arpita Dutta
>
> PhD Scholar
>
> Nit Silchar
>
> Assam-788010
>
> India
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 124, Issue 16
**********************************************

Tweet