Home » Uncategories » Moses-support Digest, Vol 114, Issue 62

Moses-support Digest, Vol 114, Issue 62

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: tuning not working properly in factored model
(Carlos Escolano)
2. Re: tuning not working properly in factored model (Matthias Huck)
3. Re: tuning not working properly in factored model
(Carlos Escolano)


----------------------------------------------------------------------

Message: 1
Date: Thu, 28 Apr 2016 18:16:43 +0200
From: Carlos Escolano <carlos.e.p93@gmail.com>
Subject: Re: [Moses-support] tuning not working properly in factored
model
To: Ondrej Bojar <bojar@ufal.mff.cuni.cz>
Cc: moses-support@mit.edu, Marta Ruiz <martaruizcostajussa@gmail.com>
Message-ID:
<CACvrzNRPszWxrBWpc5=pJTzqxm-PuV5OuGY+gXzzOxf50cY3nw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

Thank you for your answer.

You are right. While the phrase table has all three factors in the
run.X.best.out only the form appears.

I'll check why this is happening.

Best Regards,

Carlos



2016-04-28 8:46 GMT+02:00 Ondrej Bojar <bojar@ufal.mff.cuni.cz>:

> Dear Carlos,
>
> My frequent mistake in this respect is the match of factor representation
> in run.X.best.out and the reference sentences.
>
> Technically, both is possible: evaluating only the first factor (form) or
> all factors of each token. BLEU does not care. Mismatch will cause terribly
> low scores.
>
> O.
>
>
> On April 27, 2016 9:48:50 PM CEST, Carlos Escolano <carlos.e.p93@gmail.com>
> wrote:
> >Hi,
> >
> >I trained a chinese to spanish unfacored model and all worked
> >perfectly.
> >But when I try to train a factored model for the same task I have some
> >trouble while tuning. The factors I'm using are only words for chinese
> >and
> >words, lemmas and POS tags for spanish.
> >
> >Training seems to finish correctly and the phrase tables shows all the
> >factors but when tuning t it only does 2 runs and prints a message
> >saying
> >that weights have not change in the last run. Leaving the original
> >weights.
> >Also when translating, the BLEU obtained is worse than the obtained
> >with
> >the not factored model.
> >
> >
> >These are my calls for training and tuning the model:
> >
> >$SCRIPTS_ROOTDIR/training/train-model.perl \
> > -external-bin-dir $GIZA_DIR/mgiza-bin -mgiza \
> > --corpus $WORKING_DIR/train/train \
> > --alignment grow-diag-final-and \
> > --score-options '--GoodTuring' \
> > --root-dir $WORKING_DIR/baseline/ \
> > --f zh --e es \
> > --lm 0:5:$WORKING_DIR/baseline/lm/words.lm.es:0 \
> > --translation-factors 0-0,1,2 \
> > --reordering msd-bidirectional-fe \
> > --reordering-factors 0-0 \
> >
> >$MOSES_SCRIPTS/training/mert-moses.pl \
> > $WORKING_DIR/dev/dev.zh \
> > $WORKING_DIR/dev/dev.es \
>
> >$MOSES_DIR/moses-cmd/bin/gcc-4.8.5/release/link-static/threading-multi/moses
> >\
> > $WORKING_DIR/baseline/model/moses.ini \
> > --nbest 100 \
> >--working-dir $WORKING_DIR/baseline/tuning/ \
> >--decoder-flags "-drop-unknown -mbr -threads 24 -mp -v 0" \
> > --rootdir $MOSES_SCRIPTS \
> >--mertdir $MOSES_DIR/bin/ \
> >-threads 24 \
> > --filtercmd '/veu4/usuaris24/xtrans/mosesdecoder/scripts/training/
> >filter-model-given-input.pl'
> >
>
> >/veu4/usuaris24/smt/softlic/mosesdecoder/scripts//ems/support/reuse-weights.perl
> >\
> > $WORKING_DIR/baseline/tuning/moses.ini <
> >$WORKING_DIR/baseline/model/moses.ini >
> >$WORKING_DIR/baseline/tuning/moses.weight-reused.ini
> >
> >
> >Best regards,
> >
> >Carlos
> >
> >
> >------------------------------------------------------------------------
> >
> >_______________________________________________
> >Moses-support mailing list
> >Moses-support@mit.edu
> >http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> Ondrej Bojar (mailto:obo@cuni.cz / bojar@ufal.mff.cuni.cz)
> http://www.cuni.cz/~obo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160428/ca3b29a0/attachment-0001.html

------------------------------

Message: 2
Date: Thu, 28 Apr 2016 20:14:40 +0200
From: Matthias Huck <mhuck@cis.lmu.de>
Subject: Re: [Moses-support] tuning not working properly in factored
model
To: Carlos Escolano <carlos.e.p93@gmail.com>, Ondrej Bojar
<bojar@ufal.mff.cuni.cz>
Cc: moses-support@mit.edu, Marta Ruiz <martaruizcostajussa@gmail.com>
Message-ID: <1461867280.18419.69.camel@cis.lmu.de>
Content-Type: text/plain; charset="UTF-8"

Hi,

Moses can be configured to output the target-side factors of your choice.
Add something like this to your moses.ini:

[output-factors]
0
1
2

Cheers,
Matthias


On Thu, 2016-04-28 at 18:16 +0200, Carlos Escolano wrote:
> Hi,
>
> Thank you for your answer.
>
> You are right. While the phrase table has all three factors in the
> run.X.best.out only the form appears.
>
> I'll check why this is happening.
>
> Best Regards,
>
> Carlos
>
>
>
> 2016-04-28 8:46 GMT+02:00 Ondrej Bojar <bojar@ufal.mff.cuni.cz>:
>
> > Dear Carlos,
> >
> > My frequent mistake in this respect is the match of factor representation
> > in run.X.best.out and the reference sentences.
> >
> > Technically, both is possible: evaluating only the first factor (form) or
> > all factors of each token. BLEU does not care. Mismatch will cause terribly
> > low scores.
> >
> > O.
> >
> >
> > On April 27, 2016 9:48:50 PM CEST, Carlos Escolano <carlos.e.p93@gmail.com>
> > wrote:
> > > Hi,
> > >
> > > I trained a chinese to spanish unfacored model and all worked
> > > perfectly.
> > > But when I try to train a factored model for the same task I have some
> > > trouble while tuning. The factors I'm using are only words for chinese
> > > and
> > > words, lemmas and POS tags for spanish.
> > >
> > > Training seems to finish correctly and the phrase tables shows all the
> > > factors but when tuning t it only does 2 runs and prints a message
> > > saying
> > > that weights have not change in the last run. Leaving the original
> > > weights.
> > > Also when translating, the BLEU obtained is worse than the obtained
> > > with
> > > the not factored model.
> > >
> > >
> > > These are my calls for training and tuning the model:
> > >
> > > $SCRIPTS_ROOTDIR/training/train-model.perl \
> > > -external-bin-dir $GIZA_DIR/mgiza-bin -mgiza \
> > > --corpus $WORKING_DIR/train/train \
> > > --alignment grow-diag-final-and \
> > > --score-options '--GoodTuring' \
> > > --root-dir $WORKING_DIR/baseline/ \
> > > --f zh --e es \
> > > --lm 0:5:$WORKING_DIR/baseline/lm/words.lm.es:0 \
> > > --translation-factors 0-0,1,2 \
> > > --reordering msd-bidirectional-fe \
> > > --reordering-factors 0-0 \
> > >
> > > $MOSES_SCRIPTS/training/mert-moses.pl \
> > > $WORKING_DIR/dev/dev.zh \
> > > $WORKING_DIR/dev/dev.es \
> >
> > > $MOSES_DIR/moses-cmd/bin/gcc-4.8.5/release/link-static/threading-multi/moses
> > > \
> > > $WORKING_DIR/baseline/model/moses.ini \
> > > --nbest 100 \
> > > --working-dir $WORKING_DIR/baseline/tuning/ \
> > > --decoder-flags "-drop-unknown -mbr -threads 24 -mp -v 0" \
> > > --rootdir $MOSES_SCRIPTS \
> > > --mertdir $MOSES_DIR/bin/ \
> > > -threads 24 \
> > > --filtercmd '/veu4/usuaris24/xtrans/mosesdecoder/scripts/training/
> > > filter-model-given-input.pl'
> > >
> >
> > > /veu4/usuaris24/smt/softlic/mosesdecoder/scripts//ems/support/reuse-weights.perl
> > > \
> > > $WORKING_DIR/baseline/tuning/moses.ini <
> > > $WORKING_DIR/baseline/model/moses.ini >
> > > $WORKING_DIR/baseline/tuning/moses.weight-reused.ini
> > >
> > >
> > > Best regards,
> > >
> > > Carlos
> > >
> > >
> > > ------------------------------------------------------------------------
> > >
> > > _______________________________________________
> > > Moses-support mailing list
> > > Moses-support@mit.edu
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> > --
> > Ondrej Bojar (mailto:obo@cuni.cz / bojar@ufal.mff.cuni.cz)
> > http://www.cuni.cz/~obo
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 3
Date: Thu, 28 Apr 2016 21:09:08 +0200
From: Carlos Escolano <carlos.e.p93@gmail.com>
Subject: Re: [Moses-support] tuning not working properly in factored
model
To: Matthias Huck <mhuck@cis.lmu.de>
Cc: moses-support@mit.edu, Ondrej Bojar <bojar@ufal.mff.cuni.cz>,
Marta Ruiz <martaruizcostajussa@gmail.com>
Message-ID:
<CACvrzNQorgs8LTyYZMa=4+8VSvOMqBvTJcy9Pc1stMewySfDQQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

Thank you for your answer

I've tried setting the output factors in the moses.ini before and
mert-moses.pl throws the following error:

Loading table into memory...done.
terminate called after throwing an instance of 'util::Exception'
what(): moses/mbr.cpp:112 in const Moses::TrellisPath doMBR(const
Moses::TrellisPathList&, const Moses::AllOptions&) threw util::Exception
because `oFactors.size() != 1'.
Need exactly one output factor!

Using the moses.ini without tuning all factors are generated, It's during
the tuning process that only the forms appear.

Best regards,

Carlos

2016-04-28 20:14 GMT+02:00 Matthias Huck <mhuck@cis.lmu.de>:

> Hi,
>
> Moses can be configured to output the target-side factors of your choice.
> Add something like this to your moses.ini:
>
> [output-factors]
> 0
> 1
> 2
>
> Cheers,
> Matthias
>
>
> On Thu, 2016-04-28 at 18:16 +0200, Carlos Escolano wrote:
> > Hi,
> >
> > Thank you for your answer.
> >
> > You are right. While the phrase table has all three factors in the
> > run.X.best.out only the form appears.
> >
> > I'll check why this is happening.
> >
> > Best Regards,
> >
> > Carlos
> >
> >
> >
> > 2016-04-28 8:46 GMT+02:00 Ondrej Bojar <bojar@ufal.mff.cuni.cz>:
> >
> > > Dear Carlos,
> > >
> > > My frequent mistake in this respect is the match of factor
> representation
> > > in run.X.best.out and the reference sentences.
> > >
> > > Technically, both is possible: evaluating only the first factor (form)
> or
> > > all factors of each token. BLEU does not care. Mismatch will cause
> terribly
> > > low scores.
> > >
> > > O.
> > >
> > >
> > > On April 27, 2016 9:48:50 PM CEST, Carlos Escolano <
> carlos.e.p93@gmail.com>
> > > wrote:
> > > > Hi,
> > > >
> > > > I trained a chinese to spanish unfacored model and all worked
> > > > perfectly.
> > > > But when I try to train a factored model for the same task I have
> some
> > > > trouble while tuning. The factors I'm using are only words for
> chinese
> > > > and
> > > > words, lemmas and POS tags for spanish.
> > > >
> > > > Training seems to finish correctly and the phrase tables shows all
> the
> > > > factors but when tuning t it only does 2 runs and prints a message
> > > > saying
> > > > that weights have not change in the last run. Leaving the original
> > > > weights.
> > > > Also when translating, the BLEU obtained is worse than the obtained
> > > > with
> > > > the not factored model.
> > > >
> > > >
> > > > These are my calls for training and tuning the model:
> > > >
> > > > $SCRIPTS_ROOTDIR/training/train-model.perl \
> > > > -external-bin-dir $GIZA_DIR/mgiza-bin -mgiza \
> > > > --corpus $WORKING_DIR/train/train \
> > > > --alignment grow-diag-final-and \
> > > > --score-options '--GoodTuring' \
> > > > --root-dir $WORKING_DIR/baseline/ \
> > > > --f zh --e es \
> > > > --lm 0:5:$WORKING_DIR/baseline/lm/words.lm.es:0 \
> > > > --translation-factors 0-0,1,2 \
> > > > --reordering msd-bidirectional-fe \
> > > > --reordering-factors 0-0 \
> > > >
> > > > $MOSES_SCRIPTS/training/mert-moses.pl \
> > > > $WORKING_DIR/dev/dev.zh \
> > > > $WORKING_DIR/dev/dev.es \
> > >
> > > >
> $MOSES_DIR/moses-cmd/bin/gcc-4.8.5/release/link-static/threading-multi/moses
> > > > \
> > > > $WORKING_DIR/baseline/model/moses.ini \
> > > > --nbest 100 \
> > > > --working-dir $WORKING_DIR/baseline/tuning/ \
> > > > --decoder-flags "-drop-unknown -mbr -threads 24 -mp -v 0" \
> > > > --rootdir $MOSES_SCRIPTS \
> > > > --mertdir $MOSES_DIR/bin/ \
> > > > -threads 24 \
> > > > --filtercmd '/veu4/usuaris24/xtrans/mosesdecoder/scripts/training/
> > > > filter-model-given-input.pl'
> > > >
> > >
> > > >
> /veu4/usuaris24/smt/softlic/mosesdecoder/scripts//ems/support/reuse-weights.perl
> > > > \
> > > > $WORKING_DIR/baseline/tuning/moses.ini <
> > > > $WORKING_DIR/baseline/model/moses.ini >
> > > > $WORKING_DIR/baseline/tuning/moses.weight-reused.ini
> > > >
> > > >
> > > > Best regards,
> > > >
> > > > Carlos
> > > >
> > > >
> > > >
> ------------------------------------------------------------------------
> > > >
> > > > _______________________________________________
> > > > Moses-support mailing list
> > > > Moses-support@mit.edu
> > > > http://mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > > --
> > > Ondrej Bojar (mailto:obo@cuni.cz / bojar@ufal.mff.cuni.cz)
> > > http://www.cuni.cz/~obo
> > >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160428/864b91b2/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 114, Issue 62
**********************************************

0 Response to "Moses-support Digest, Vol 114, Issue 62"

Post a Comment

Subscribe to: Post Comments (Atom)