Moses-support Digest, Vol 96, Issue 9

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Factored Model / moses.exe.srackdump (Philipp Koehn)
2. Re: Factored Model / moses.exe.srackdump (Marwa Refaie)


----------------------------------------------------------------------

Message: 1
Date: Tue, 7 Oct 2014 16:54:06 -0400
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Factored Model / moses.exe.srackdump
To: Marwa Refaie <basmallah@hotmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDcYLc8kOM6sRduYsTKMak5VT-D2T7aUGmQEqzK3aWdGg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

the following paths do not seem right:

SRILM name=LM0 factor=0 path=/cygdrive/c order=5
SRILM name=LM1 factor=1 path=/cygdrive/c order=5

Also, your translation step maps the two input factors the the two
output factors. But you also have a generation step - that is
redundant, you can leave the generation step out.

-phi

On Tue, Oct 7, 2014 at 4:41 PM, Marwa Refaie <basmallah@hotmail.com> wrote:
>
>
>
>
>
>
>
> Marwa N. Refaie
>
>
>
> ________________________________
> From: basmallah@hotmail.com
> To: basmallah@hotmail.com
> Subject: Factored Model / moses.exe.srackdump
> Date: Tue, 7 Oct 2014 20:40:07 +0000
>
> I build the two language model by SRILM, one for surface and one for pos
> tags, I used pos in both language sides.. train ::
>
> c:/mosesdecoder-master/scripts/training/train-model.perl -external-bin-dir
> c:/mosesdecoder-master/tools/bin -root-dir work -corpus
> c:/mosesdecoder-master/try/ai/sep/fsmt/data/UNpos.lo -f en -e ar
> -Translation-factors 0,1-0,1 -lm
> 0:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/surface.lm:0 -lm
> 1:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/pos.lm:0
> -generation-factors 0-1 -decoding-steps t0,g0 >& training.out&
>
>
> it's done & moses.ini found, but I got this error when try to translate
> (test),,,,
> Any help please !!!!
>
> #########################
> ### MOSES CONFIG FILE ###
> #########################
>
> # input factors
> [input-factors]
> 0
> 1
>
> # mapping steps
> [mapping]
> 0 T 0
> 0 G 0
>
> [distortion-limit]
> 6
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4
> path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.0,1-0,1.gz
> input-factor=0,1 output-factor=0,1
> Generation name=GenerationModel0 num-features=2
> path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/generation.0-1.gz
> input-factor=0 output-factor=1
> Distortion
> SRILM name=LM0 factor=0 path=/cygdrive/c order=5
> SRILM name=LM1 factor=1 path=/cygdrive/c order=5
>
> # dense weights for feature functions
> [weight]
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2
> GenerationModel0= 0.3 0
> Distortion0= 0.3
> LM0= 0.5
> LM1= 0.5
> .....................................................................................
>
> OUTPUT
>
>
> config: C:/mosesdecoder-master/try/ai/sep/fsmt/work/model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4
> path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.0,1-0,1.gz
> input-factor=0,1 output-factor=0,1 Generation name=GenerationModel0
> num-features=2
> path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/generation.0-1.gz
> input-factor=0 output-factor=1 Distortion SRILM name=LM0 factor=0
> path=/cygdrive/c order=5 SRILM name=LM1 factor=1 path=/cygdrive/c order=5
> input-factors: 0 1
> mapping: 0 T 0 0 G 0
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2 GenerationModel0= 0.3 0 Distortion0= 0.3
> LM0= 0.5 LM1= 0.5
> c:/mosesdecoder-master/bin
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> num-features=4
> path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.0,1-0,1.gz
> input-factor=0,1 output-factor=0,1
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=Generation name=GenerationModel0 num-features=2
> path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/generation.0-1.gz
> input-factor=0 output-factor=1
> FeatureFunction: GenerationModel0 start: 7 end: 8
> line=Distortion
> FeatureFunction: Distortion0 start: 9 end: 9
> line=SRILM name=LM0 factor=0 path=/cygdrive/c order=5
> FeatureFunction: LM0 start: 10 end: 10
> line=SRILM name=LM1 factor=1 path=/cygdrive/c order=5
> FeatureFunction: LM1 start: 11 end: 11
> terminate called after throwing an instance of 'std::string'
> Aborted (core dumped)
>
>
>
>
>
> Marwa N. Refaie
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 2
Date: Tue, 7 Oct 2014 21:28:04 +0000
From: Marwa Refaie <basmallah@hotmail.com>
Subject: Re: [Moses-support] Factored Model / moses.exe.srackdump
To: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <DUB118-W2F1A75232FC99896FFD0FBAA20@phx.gbl>
Content-Type: text/plain; charset="windows-1256"

Thank's for help ,, I fix all what you mentioned now I'm stucked in this error ::
Start loading text SCFG phrase table. Moses format : [1.000] secondsReading /cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table. 0,1-0,1.gz----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80 ---85---90---95--100Either your data contains <s> in a position other than the first word or your la nguage model is missing <s>. Did you build your ARPA using IRSTLM and forget to run add-start-end.sh?[1]+ Done c:/mosesdecoder-master/scripts/training/train-mode !
l.perl -external-bin-dir c:/mosesdecoder-master/tools/bin -root-dir work -corpus c:/mosesdecoder-master/try/ai/sep/fsmt/data/UNpos.lo -f en -e ar -Translation-f actors 0,1-0,1 -lm 0:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/su rface.lm -lm 1:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/pos.lm & >training.outAborted (core dumped)

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>As mentioned I build non factored model , it worked well, but when I start use the pos & surface language model for my training data I got these errors ??I used the SRILM.


Marwa N. Refaie



> Date: Tue, 7 Oct 2014 16:54:06 -0400
> Subject: Re: [Moses-support] Factored Model / moses.exe.srackdump
> From: pkoehn@inf.ed.ac.uk
> To: basmallah@hotmail.com
> CC: moses-support@mit.edu
>
> Hi,
>
> the following paths do not seem right:
>
> SRILM name=LM0 factor=0 path=/cygdrive/c order=5
> SRILM name=LM1 factor=1 path=/cygdrive/c order=5
>
> Also, your translation step maps the two input factors the the two
> output factors. But you also have a generation step - that is
> redundant, you can leave the generation step out.
>
> -phi
>
> On Tue, Oct 7, 2014 at 4:41 PM, Marwa Refaie <basmallah@hotmail.com> wrote:
> >
> >
> >
> >
> >
> >
> >
> > Marwa N. Refaie
> >
> >
> >
> > ________________________________
> > From: basmallah@hotmail.com
> > To: basmallah@hotmail.com
> > Subject: Factored Model / moses.exe.srackdump
> > Date: Tue, 7 Oct 2014 20:40:07 +0000
> >
> > I build the two language model by SRILM, one for surface and one for pos
> > tags, I used pos in both language sides.. train ::
> >
> > c:/mosesdecoder-master/scripts/training/train-model.perl -external-bin-dir
> > c:/mosesdecoder-master/tools/bin -root-dir work -corpus
> > c:/mosesdecoder-master/try/ai/sep/fsmt/data/UNpos.lo -f en -e ar
> > -Translation-factors 0,1-0,1 -lm
> > 0:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/surface.lm:0 -lm
> > 1:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/pos.lm:0
> > -generation-factors 0-1 -decoding-steps t0,g0 >& training.out&
> >
> >
> > it's done & moses.ini found, but I got this error when try to translate
> > (test),,,,
> > Any help please !!!!
> >
> > #########################
> > ### MOSES CONFIG FILE ###
> > #########################
> >
> > # input factors
> > [input-factors]
> > 0
> > 1
> >
> > # mapping steps
> > [mapping]
> > 0 T 0
> > 0 G 0
> >
> > [distortion-limit]
> > 6
> >
> > # feature functions
> > [feature]
> > UnknownWordPenalty
> > WordPenalty
> > PhrasePenalty
> > PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4
> > path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.0,1-0,1.gz
> > input-factor=0,1 output-factor=0,1
> > Generation name=GenerationModel0 num-features=2
> > path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/generation.0-1.gz
> > input-factor=0 output-factor=1
> > Distortion
> > SRILM name=LM0 factor=0 path=/cygdrive/c order=5
> > SRILM name=LM1 factor=1 path=/cygdrive/c order=5
> >
> > # dense weights for feature functions
> > [weight]
> > UnknownWordPenalty0= 1
> > WordPenalty0= -1
> > PhrasePenalty0= 0.2
> > TranslationModel0= 0.2 0.2 0.2 0.2
> > GenerationModel0= 0.3 0
> > Distortion0= 0.3
> > LM0= 0.5
> > LM1= 0.5
> > .....................................................................................
> >
> > OUTPUT
> >
> >
> > config: C:/mosesdecoder-master/try/ai/sep/fsmt/work/model/moses.ini
> > distortion-limit: 6
> > feature: UnknownWordPenalty WordPenalty PhrasePenalty
> > PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4
> > path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.0,1-0,1.gz
> > input-factor=0,1 output-factor=0,1 Generation name=GenerationModel0
> > num-features=2
> > path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/generation.0-1.gz
> > input-factor=0 output-factor=1 Distortion SRILM name=LM0 factor=0
> > path=/cygdrive/c order=5 SRILM name=LM1 factor=1 path=/cygdrive/c order=5
> > input-factors: 0 1
> > mapping: 0 T 0 0 G 0
> > weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
> > TranslationModel0= 0.2 0.2 0.2 0.2 GenerationModel0= 0.3 0 Distortion0= 0.3
> > LM0= 0.5 LM1= 0.5
> > c:/mosesdecoder-master/bin
> > line=UnknownWordPenalty
> > FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> > line=WordPenalty
> > FeatureFunction: WordPenalty0 start: 1 end: 1
> > line=PhrasePenalty
> > FeatureFunction: PhrasePenalty0 start: 2 end: 2
> > line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> > num-features=4
> > path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.0,1-0,1.gz
> > input-factor=0,1 output-factor=0,1
> > FeatureFunction: TranslationModel0 start: 3 end: 6
> > line=Generation name=GenerationModel0 num-features=2
> > path=/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/generation.0-1.gz
> > input-factor=0 output-factor=1
> > FeatureFunction: GenerationModel0 start: 7 end: 8
> > line=Distortion
> > FeatureFunction: Distortion0 start: 9 end: 9
> > line=SRILM name=LM0 factor=0 path=/cygdrive/c order=5
> > FeatureFunction: LM0 start: 10 end: 10
> > line=SRILM name=LM1 factor=1 path=/cygdrive/c order=5
> > FeatureFunction: LM1 start: 11 end: 11
> > terminate called after throwing an instance of 'std::string'
> > Aborted (core dumped)
> >
> >
> >
> >
> >
> > Marwa N. Refaie
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141007/135ac809/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 96, Issue 9
********************************************

Tweet