Moses-support Digest, Vol 111, Issue 7

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Segmentation Fault (Core Dumped) (gozde gul)


----------------------------------------------------------------------

Message: 1
Date: Thu, 7 Jan 2016 11:01:59 +0200
From: gozde gul <goezde.guel@googlemail.com>
Subject: Re: [Moses-support] Segmentation Fault (Core Dumped)
To: Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAFCwZ8dRv=ZXQDF8zzniZgDZO50aSwqbVC+5JZ-f9DJxgUQL+w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Rajeen,

Thanks a lot for your help. When I change t2 from 0-0 to 0-0,1,2 as you
suggested I don't get a segmentation fault anymore. I tried reading the
manual but I don't really understand why. How are the language models used
during decoding can you briefly explain ?

I thought when I define my translation factors as such:
1-1+2-2+0-0
Language model 1 (lemmaLM) will be used for 1-1, Language model 2 (postagLM)
will be used for 2-2 and Language model 0 (surLM) will be sufficient for
0-0 ? Why do I need to generate other factors for translation surface
forms, I don't really get it.

Thanks a lot for your help,

G?zde


On Mon, Dec 28, 2015 at 7:29 PM, Rajen Chatterjee <
rajen.k.chatterjee@gmail.com> wrote:

> Hi Gozde,
>
> Apart from the possible problem of memory, another problem can be that
> your mapping do not generate the LM factors.
> I think that for "t2" instead of 0-0 should have 0-0,1,2 (since you are
> using 3 LM, one for each factor), and similar changes will be required for
> another decoding path.
>
> On Mon, Dec 28, 2015 at 1:30 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> you should start with simple factored models 1st, perhaps using only 1
>> translation model with 2 factors. Then move onto 1 translation model and 1
>> generation model.
>>
>> The factored you are difficult to control, they use a lot of memory and
>> takes a lot of time. You may be getting errors because it runs out of
>> memory.
>>
>>
>> On 28/12/15 10:01, gozde gul wrote:
>>
>> Hi,
>>
>> I am trying to perform a 3 factored translation from English to Turkish.
>> My example input is as follows:
>>
>> En: Life+NNP|Life|NNP end+VBZ_never+RB|end|VBZ_never+RB but+CC|but|CC
>> earthly+JJ|earthly|JJ life+NN|life|NN do+VBZ|do|VBZ .+.|.|.
>> Tr: Hayat|hayat|+Noun+A3sg+Pnon+Nom hi?|hi?|+Adverb
>> bitmez|bit|+Verb+Neg+Aor+A3sg fakat|fakat|+Conj
>> d?nyadaki|d?nya|+Noun+A3sg+Pnon+Loc^DB+Adj+Rel
>> hayat|hayat|+Noun+A3sg+Pnon+Nom biter|bit|+Verb+Pos+Aor+A3sg .|.|+Punc
>>
>> My translation and generation factors and decoding steps are as follows.
>> I am pretty sure they are correct:
>> --translation-factors 1-1+2-2+0-0 \
>> --generation-factors 1,2-0 \
>> --decoding-steps t2:t0,t1,g0
>>
>> I have created language models for all 3 factors with irstlm with the
>> steps explained in moses website.
>>
>> If I train with the following model, it creates a moses.ini. When I
>> manually check the phrase tables and generation table they look meaningful.
>> ~/mosesdecoder/scripts/training/train-model.perl \
>> --parallel --mgiza \
>> --external-bin-dir ~/workspace/bin/training-tools/mgizapp \
>> --root-dir ~/FactoredModel/SmallModel/ \
>> --corpus
>> ~/FactoredModel/SmallModel/factored-corpus/training/korpus_1000K.en-tr.KO.recleaned_new
>> \
>> --f en --e tr --alignment grow-diag-final-and \
>> --reordering msd-bidirectional-fe \
>> --lm 0:3:$HOME/corpus/FilteredCorpus/training/lm/surLM/sur.lm.blm.tr:8 \
>> --lm 1:3:$HOME/corpus/FilteredCorpus/training/lm/lemmaLM/
>> lemma.lm.blm.tr:8 \
>> --lm 2:3:$HOME/corpus/FilteredCorpus/training/lm/postagLM/
>> postags.lm.blm.tr:8 \
>> --alignment-factors 1-1 \
>> --translation-factors 1-1+2-2+0-0 \
>> --generation-factors 1,2-0 \
>> --decoding-steps t2:t0,t1,g0 >& ~/FactoredModel/trainingSmall3lm.out
>>
>> But when I try to decode a very simple one-line sentence, I get a
>> "Segmentation fault (Core Dumped)" message. You can see the detailed
>> decoding log here
>> <https://www.dropbox.com/s/2xgi681k2ssus5z/error.txt?dl=0>. I tried many
>> things and I'm in a dead end, so I would really appreciate your help.
>>
>> Thanks,
>>
>> Gozde
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> --
>> Hieu Hoanghttp://www.hoang.co.uk/hieu
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> -Regards,
> Rajen Chatterjee.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160107/f0835ec9/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 111, Issue 7
*********************************************

Tweet