Home » Uncategories » Moses-support Digest, Vol 111, Issue 8

Moses-support Digest, Vol 111, Issue 8

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Segmentation Fault (Core Dumped) (Rajen Chatterjee)
2. tuning gives zero weights (haouarid@iro.umontreal.ca)
3. Incremental training - mmlex out of bounds error
(Laurent Barontini)


----------------------------------------------------------------------

Message: 1
Date: Thu, 7 Jan 2016 18:08:40 +0100
From: Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
Subject: Re: [Moses-support] Segmentation Fault (Core Dumped)
To: gozde gul <goezde.guel@googlemail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAC4-+NzDLM_nsAS7uS7O=ZQMBVmmtOYXknEU+3yPcJ1_hfz0hQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Gozde,

You have the following setting:

--translation-factors 1-1+2-2+0-0 \
--generation-factors 1,2-0 \
--decoding-steps t2:t0,t1,g0

In the 1st decoding path (i.e t2), the decoder can only see 0-0, it can
not see "1-1+2-2" mapping because they belong to another decoding path (
t0,t1,g0 ).
If I understand correctly, each decoding path should have its own mapping
steps to produce all the required (i.e corresponding to your LM) output
factors. So changing 0-0 to 0-0,1,2 works.


On Thu, Jan 7, 2016 at 10:01 AM, gozde gul <goezde.guel@googlemail.com>
wrote:

> Dear Rajeen,
>
> Thanks a lot for your help. When I change t2 from 0-0 to 0-0,1,2 as you
> suggested I don't get a segmentation fault anymore. I tried reading the
> manual but I don't really understand why. How are the language models used
> during decoding can you briefly explain ?
>
> I thought when I define my translation factors as such:
> 1-1+2-2+0-0
> Language model 1 (lemmaLM) will be used for 1-1, Language model 2 (
> postagLM) will be used for 2-2 and Language model 0 (surLM) will be
> sufficient for 0-0 ? Why do I need to generate other factors for
> translation surface forms, I don't really get it.
>
> Thanks a lot for your help,
>
> G?zde
>
>
> On Mon, Dec 28, 2015 at 7:29 PM, Rajen Chatterjee <
> rajen.k.chatterjee@gmail.com> wrote:
>
>> Hi Gozde,
>>
>> Apart from the possible problem of memory, another problem can be that
>> your mapping do not generate the LM factors.
>> I think that for "t2" instead of 0-0 should have 0-0,1,2 (since you are
>> using 3 LM, one for each factor), and similar changes will be required for
>> another decoding path.
>>
>> On Mon, Dec 28, 2015 at 1:30 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> you should start with simple factored models 1st, perhaps using only 1
>>> translation model with 2 factors. Then move onto 1 translation model and 1
>>> generation model.
>>>
>>> The factored you are difficult to control, they use a lot of memory and
>>> takes a lot of time. You may be getting errors because it runs out of
>>> memory.
>>>
>>>
>>> On 28/12/15 10:01, gozde gul wrote:
>>>
>>> Hi,
>>>
>>> I am trying to perform a 3 factored translation from English to Turkish.
>>> My example input is as follows:
>>>
>>> En: Life+NNP|Life|NNP end+VBZ_never+RB|end|VBZ_never+RB but+CC|but|CC
>>> earthly+JJ|earthly|JJ life+NN|life|NN do+VBZ|do|VBZ .+.|.|.
>>> Tr: Hayat|hayat|+Noun+A3sg+Pnon+Nom hi?|hi?|+Adverb
>>> bitmez|bit|+Verb+Neg+Aor+A3sg fakat|fakat|+Conj
>>> d?nyadaki|d?nya|+Noun+A3sg+Pnon+Loc^DB+Adj+Rel
>>> hayat|hayat|+Noun+A3sg+Pnon+Nom biter|bit|+Verb+Pos+Aor+A3sg .|.|+Punc
>>>
>>> My translation and generation factors and decoding steps are as follows.
>>> I am pretty sure they are correct:
>>> --translation-factors 1-1+2-2+0-0 \
>>> --generation-factors 1,2-0 \
>>> --decoding-steps t2:t0,t1,g0
>>>
>>> I have created language models for all 3 factors with irstlm with the
>>> steps explained in moses website.
>>>
>>> If I train with the following model, it creates a moses.ini. When I
>>> manually check the phrase tables and generation table they look meaningful.
>>> ~/mosesdecoder/scripts/training/train-model.perl \
>>> --parallel --mgiza \
>>> --external-bin-dir ~/workspace/bin/training-tools/mgizapp \
>>> --root-dir ~/FactoredModel/SmallModel/ \
>>> --corpus
>>> ~/FactoredModel/SmallModel/factored-corpus/training/korpus_1000K.en-tr.KO.recleaned_new
>>> \
>>> --f en --e tr --alignment grow-diag-final-and \
>>> --reordering msd-bidirectional-fe \
>>> --lm 0:3:$HOME/corpus/FilteredCorpus/training/lm/surLM/sur.lm.blm.tr:8 \
>>> --lm 1:3:$HOME/corpus/FilteredCorpus/training/lm/lemmaLM/
>>> lemma.lm.blm.tr:8 \
>>> --lm 2:3:$HOME/corpus/FilteredCorpus/training/lm/postagLM/
>>> postags.lm.blm.tr:8 \
>>> --alignment-factors 1-1 \
>>> --translation-factors 1-1+2-2+0-0 \
>>> --generation-factors 1,2-0 \
>>> --decoding-steps t2:t0,t1,g0 >& ~/FactoredModel/trainingSmall3lm.out
>>>
>>> But when I try to decode a very simple one-line sentence, I get a
>>> "Segmentation fault (Core Dumped)" message. You can see the detailed
>>> decoding log here
>>> <https://www.dropbox.com/s/2xgi681k2ssus5z/error.txt?dl=0>. I tried
>>> many things and I'm in a dead end, so I would really appreciate your help.
>>>
>>> Thanks,
>>>
>>> Gozde
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>> --
>>> Hieu Hoanghttp://www.hoang.co.uk/hieu
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> -Regards,
>> Rajen Chatterjee.
>>
>
>


--
-Regards,
Rajen Chatterjee.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160107/2ba7075a/attachment-0001.html

------------------------------

Message: 2
Date: Thu, 7 Jan 2016 22:17:43 -0500
From: haouarid@iro.umontreal.ca
Subject: [Moses-support] tuning gives zero weights
To: Moses-support@mit.edu
Message-ID:
<6a890a1992c40b4211dddbdb0e4864d8.squirrel@webmail.iro.umontreal.ca>
Content-Type: text/plain;charset=iso-8859-1

Hi,

I'm tuning a translation table with the mert-moses.pl script and it
returns weights with values equal to zero.

Is there an explanation for this behavior?

Thank you,
Dorra


------------------------------

Message: 3
Date: Fri, 8 Jan 2016 09:09:57 +0000
From: Laurent Barontini <barontil@hotmail.com>
Subject: [Moses-support] Incremental training - mmlex out of bounds
error
To: "moses-support@mit.edu" <moses-support@mit.edu>, Laurent Barontini
<barontil@hotmail.com>
Message-ID: <DUB112-W96866BB4211E9F1FA9F786CEF60@phx.gbl>
Content-Type: text/plain; charset="windows-1252"

Hi All,



To try incremental mode, I?ve trained the europarl (fr en)
corpus with the option ?-final-alignment-model hmm?

I obtain following files:

In
/train/giza.fr-en/:

En-fr.Ahmm.5.gz

En-fr.aahmm.5

En-fr.cooc

En-fr.gizacfg

En-fr.hhmm.5

En-fr.thmm.5



And in /train/giza.en-fr/ :

Fr-en.Ahmm.5.gz

Fr-en.aahmm.5

Fr-en.cooc

Fr-en.gizacfg

Fr-en.hhmm.5

Fr-en.thmm.5



In /train/model/:

Aligned.grow-diag-final-and

Extract.inv.sorted.gz

Extract.o.sorted.gz

Extract.sorted.gz

Moses.ini

Phrase-table.gz

Reordering-table.wbe-msd-bidirectional-fe.gz



Moses is built with option ?with-mm.



What I?ve done to build binary file:

% zcat fr-en.Ahmm.5.gz | mtt-build ?i ?o /mypath/corp.fr

% zcat en-fr.Ahmm.5.gz | mtt-build ?i ?o /mypath/corp.en

% cat aligned.grow-diag-final-and | symal2mam
/mypath/corp.fr-en.mam

No problem til there...

% mmlex-build /mypath/corp fr en ? o /mypath/corp.fr-en.lex

Error:

Terminate called after throwing an instance of
?util::Exception?

what(): out of bounds at line 1

Terminate called recursively

Abandon



So I wanted to know if the steps before mmlex-build are
good?

Did someone manage to use incremental mode?



Thanks in advance,

Laurent














-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160108/2ad3dd79/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 111, Issue 8
*********************************************

0 Response to "Moses-support Digest, Vol 111, Issue 8"

Post a Comment

Subscribe to: Post Comments (Atom)