Moses-support Digest, Vol 110, Issue 49

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Segmentation Fault (Core Dumped) (Rajen Chatterjee)
2. Re: How much tuning data? (Philipp Koehn)
3. Re: EMS system error (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Mon, 28 Dec 2015 18:29:07 +0100
From: Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
Subject: Re: [Moses-support] Segmentation Fault (Core Dumped)
To: gozde gul <goezde.guel@googlemail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAC4-+Nyh4t9VMh=7YLsUdsA_PRx3abVpi2hs0iJ1wsyQ+=kHNw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Gozde,

Apart from the possible problem of memory, another problem can be that your
mapping do not generate the LM factors.
I think that for "t2" instead of 0-0 should have 0-0,1,2 (since you are
using 3 LM, one for each factor), and similar changes will be required for
another decoding path.

On Mon, Dec 28, 2015 at 1:30 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:

> you should start with simple factored models 1st, perhaps using only 1
> translation model with 2 factors. Then move onto 1 translation model and 1
> generation model.
>
> The factored you are difficult to control, they use a lot of memory and
> takes a lot of time. You may be getting errors because it runs out of
> memory.
>
>
> On 28/12/15 10:01, gozde gul wrote:
>
> Hi,
>
> I am trying to perform a 3 factored translation from English to Turkish.
> My example input is as follows:
>
> En: Life+NNP|Life|NNP end+VBZ_never+RB|end|VBZ_never+RB but+CC|but|CC
> earthly+JJ|earthly|JJ life+NN|life|NN do+VBZ|do|VBZ .+.|.|.
> Tr: Hayat|hayat|+Noun+A3sg+Pnon+Nom hi?|hi?|+Adverb
> bitmez|bit|+Verb+Neg+Aor+A3sg fakat|fakat|+Conj
> d?nyadaki|d?nya|+Noun+A3sg+Pnon+Loc^DB+Adj+Rel
> hayat|hayat|+Noun+A3sg+Pnon+Nom biter|bit|+Verb+Pos+Aor+A3sg .|.|+Punc
>
> My translation and generation factors and decoding steps are as follows. I
> am pretty sure they are correct:
> --translation-factors 1-1+2-2+0-0 \
> --generation-factors 1,2-0 \
> --decoding-steps t2:t0,t1,g0
>
> I have created language models for all 3 factors with irstlm with the
> steps explained in moses website.
>
> If I train with the following model, it creates a moses.ini. When I
> manually check the phrase tables and generation table they look meaningful.
> ~/mosesdecoder/scripts/training/train-model.perl \
> --parallel --mgiza \
> --external-bin-dir ~/workspace/bin/training-tools/mgizapp \
> --root-dir ~/FactoredModel/SmallModel/ \
> --corpus
> ~/FactoredModel/SmallModel/factored-corpus/training/korpus_1000K.en-tr.KO.recleaned_new
> \
> --f en --e tr --alignment grow-diag-final-and \
> --reordering msd-bidirectional-fe \
> --lm 0:3:$HOME/corpus/FilteredCorpus/training/lm/surLM/sur.lm.blm.tr:8 \
> --lm 1:3:$HOME/corpus/FilteredCorpus/training/lm/lemmaLM/lemma.lm.blm.tr:8
> \
> --lm 2:3:$HOME/corpus/FilteredCorpus/training/lm/postagLM/
> postags.lm.blm.tr:8 \
> --alignment-factors 1-1 \
> --translation-factors 1-1+2-2+0-0 \
> --generation-factors 1,2-0 \
> --decoding-steps t2:t0,t1,g0 >& ~/FactoredModel/trainingSmall3lm.out
>
> But when I try to decode a very simple one-line sentence, I get a
> "Segmentation fault (Core Dumped)" message. You can see the detailed
> decoding log here
> <https://www.dropbox.com/s/2xgi681k2ssus5z/error.txt?dl=0>. I tried many
> things and I'm in a dead end, so I would really appreciate your help.
>
> Thanks,
>
> Gozde
>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoanghttp://www.hoang.co.uk/hieu
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
-Regards,
Rajen Chatterjee.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151228/fc31f821/attachment-0001.html

------------------------------

Message: 2
Date: Mon, 28 Dec 2015 16:00:14 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] How much tuning data?
To: Vincent Nguyen <vnguyen@neuf.fr>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDVodKJMH3MeLmFCe7YfKLhR8BGpKixREdxrmA7Xnb3FQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

the size of the tuning set is one factor in the variance between different
tuning runs.

Run MERT (or kbmira) multiple times, to see what test BLEU scores you will
get. If they vary too much between tuning runs, then a larger tuning set
would be better.

The motivation to keep variance low is that you can actually observe the
impact of your improved system over the baseline.

-phi

On Mon, Dec 28, 2015 at 11:16 AM, Vincent Nguyen <vnguyen@neuf.fr> wrote:

>
> this is fine for tuning. if you want to make it quicker, drag it down to
> 1000 sentences.
>
>
> Le 28/12/2015 16:37, Read, James C a ?crit :
>
> Hi,
>
>
> I'm setting up some Moses baseline systems for various language pairs to
> compare the systems against my own work. I've largely been following the
> baseline tutorial here http://www.statmt.org/moses/?n=moses.baseline
>
>
> What would a fair amount of tuning data be to do the Moses system justice?
> I've currently managed to isolate a set of about 1,900 sentences which are
> roughly common to all language pairs. Would that be enough? To much? Not
> enough?
>
>
> James
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151228/82798c3d/attachment-0001.html

------------------------------

Message: 3
Date: Mon, 28 Dec 2015 16:06:40 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] EMS system error
To: Sunayana Gawde <sunayanagawde17@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDD18---WFCDHwbMWpOO2Wyt0BjKAH1204aTKSAN9iBLrQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

as long as you get error messages from experiment.perl, nothing will be
executed.

The error message indicates that IRSTLM is not installed. You do not need
it (you also do not need SRILM), so you should just comment out references
to them and use KENLM.

If everything runs through, the BLEU score will be in the file
evaluation/report.1

-phi

On Mon, Dec 28, 2015 at 7:04 AM, Sunayana Gawde <sunayanagawde17@gmail.com>
wrote:

> Hi all,
>
> I tried giving my training tuning and test data in respective files and
> then i did the change in config file as: home-dir = /home/sunayana
>
> then i used the command :
>
> nohup nice ~/mosesdecoder/scripts/ems/experiment.perl -config config -exec &> log &
>
> it gives me this :
> sunayana@sunayana:~/working/experiments$ nohup nice ~/mosesdecoder/scripts/ems/experiment.perl -config config -exec &> log &
> [1] 5823
>
>
> where do i check for BLEU score?
>
> I checked the log file here :/home/sunayana/working/experiments/steps
>
> there i get:
>
> nohup: ignoring input
> STARTING UP AS PROCESS 5823 ON sunayana AT Mon Dec 28 17:27:04 IST 2015
> LOAD CONFIG...
> find: `/home/sunayana/irstlm/bin*': No such file or directory
> GENERAL:irstlm-dir: file /home/sunayana/irstlm/bin does not exist!
> Died at /home/sunayana/mosesdecoder/scripts/ems/experiment.perl line 437.
>
> I also installed image magick, ghost view tool and Graphviz
>
> Please help me in this.
>
>
>
> --
> *Thanks & Regards*
>
> Ms. Sunayana R. Gawde.
>
> DCST, Goa University.
> * P**leas**e don't print t**his e-mail unles**s you really need to.*
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151228/2891f173/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 110, Issue 49
**********************************************

Tweet