Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Compact lex reordering table on OSX/clang (Hieu Hoang)
2. Re: Compact lex reordering table on OSX/clang (Jeroen Vermeulen)
3. Re: Segmentation Fault during Tuning (Alex Martinez)
----------------------------------------------------------------------
Message: 1
Date: Tue, 13 Oct 2015 14:00:27 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Compact lex reordering table on OSX/clang
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjFWMJdaStdF0MQYeZEUV1QejoTczDz9a4sweSGKHTxvw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
i'll take a closer look when I have time. I think it's been happening for a
while but I've ignored it.
btw, i've pulled unblockpt into master
Hieu Hoang
http://www.hoang.co.uk/hieu
On 13 October 2015 at 12:05, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
wrote:
> Hi,
>
> yes, definitely wrong turn, all code should be in CompactPT.
> I am not sure this is actually a code bug, is it working with g++ on macOS?
>
> W dniu 2015-10-13 12:50, Jeroen Vermeulen napisa?(a):
>
> On 10/13/2015 04:59 PM, Hieu Hoang wrote:
>
> you're quite right, i've added a check
> https://github.com/moses-smt/mosesdecoder/commit/982d52e5b657f4c1fa7369e577cfd75a8af16543
> However, that the the problem I'm having on OSX. It opens but it crashes on
> loading. I suspect one of the datatypes has slightly different size on
> clang/OSX compared to gcc/Linux
>
> Before the loading gets to this point, CanonicalHuffman.Load() does
> something that intrigues me, as a reader who doesn't really grok the
> code: it fread()s an array of Data.
>
> If Data is the class I find in mert/Data.h, then AFAICT the compiler
> would be well within its rights to break this. Not only is it not a
> POD, it contains pointers, including in strings and vectors. You
> wouldn't expect that to work. Did I take a wrong turn somewhere?
>
>
> Jeroen
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151013/52d558c3/attachment-0001.html
------------------------------
Message: 2
Date: Tue, 13 Oct 2015 20:10:43 +0700
From: Jeroen Vermeulen <jtv@precisiontranslationtools.com>
Subject: Re: [Moses-support] Compact lex reordering table on OSX/clang
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <561D02D3.9060208@precisiontranslationtools.com>
Content-Type: text/plain; charset=utf-8
On 10/13/2015 06:05 PM, Marcin Junczys-Dowmunt wrote:
> yes, definitely wrong turn, all code should be in CompactPT.
Ah, I'd missed that this is a template. So what's being loaded there is
an array of float. Not that much that can go wrong there, within one
CPU architecture...
My next stab in the dark would be the MmapAllocator... which seems to
be where the error happens.
But now we're into higher magic, as far as I'm concerned. Could there
be a fatal difference in how std::vector interacts with the allocator?
Jeroen
------------------------------
Message: 3
Date: Tue, 13 Oct 2015 14:12:31 +0000 (GMT)
From: Alex Martinez <cmxela@me.com>
Subject: Re: [Moses-support] Segmentation Fault during Tuning
To: Philipp Koehn <phi@jhu.edu>
Cc: moses-support@mit.edu
Message-ID: <49b66e4f-37de-412d-ba88-a45d99fedc34@me.com>
Content-Type: text/plain; charset="utf-8"
Hi,
with this modification it works
Thanks a lot
Alex
El 12 oct 2015 a las 09:09, Philipp Koehn <phi@jhu.edu> escribi?:
Hi,
in t2, you do generate an output lemma factor - which may be the cause of this problem (even though you do not seem to use the output lemma anywhere else).
Does it still core dump, if you change translation factors to:
translation-factors = "lemma -> lemma, pos -> pos, word -> word + lemma + pos"
-phi
On Sat, Oct 10, 2015 at 9:52 AM, Alex Martinez <cmxela@me.com> wrote:
Hello,
I'm trying to build a factored system using EMS based on this example from the tutorial:
---------------------------------------------------------------------
% train-model.perl \
? ? --corpus factored-corpus/proj-syndicate.1000 \
? ? --root-dir morphgen-backoff \
? ? --f de --e en \
? ? --lm 0:3:factored-corpus/surface.lm:0 \
? ? --lm 2:3:factored-corpus/pos.lm:0 \
? ? --translation-factors 1-1+3-2+0-0,2 \
? ? --generation-factors 1-2+1,2-0 \
? ? --decoding-steps t0,g0,t1,g1:t2 \
? ? --external-bin-dir .../tools
----------------------------------------------------------------------
I'm getting a segmentation fault during tuning and I have the feeling that the problem is related to the line defining the decoding-steps.
What I have on my EMS config file to get a similar model is:
--------------------------------------------------------------------
### factored training: specify here which factors used
# if none specified, single factor training is assumed
# (one translation step, surface to surface)
#
input-factors = word lemma pos
output-factors = word lemma pos
alignment-factors = "word+lemma -> word+lemma"
translation-factors = "lemma -> lemma, pos -> pos, word -> word + pos"
reordering-factors = "word -> word"
generation-factors = "lemma -> pos, lemma+pos -> word"
decoding-steps = "t0,g0,t1,g1:t2"
generation-type = single
prune-generation = "$moses-bin-dir/pruneGeneration 100"
-------------------------------------------------------------------------
The training fails in the tuning step and I'm getting this in the TUNING_tune.1.STDERR:
Executing: /opt/moses/bin/moses -threads all -v 0 ? -config /mnt/a62/devel/en_es/processfin/model/moses.bin.ini.1 -weight-overwrite 'WordPenalty0= -0.128205 TranslationModel0= 0.025641 0.025641 0.025641 0.025641 LM2= 0.064103 LM0= 0.064103 GenerationModel1= 0.038462 0.000000 TranslationModel2= 0.025641 0.025641 0.025641 0.025641 GenerationModel0= 0.038462 PhrasePenalty0= 0.025641 Distortion0= 0.038462 TranslationModel1= 0.025641 0.025641 0.025641 0.025641 LexicalReordering0= 0.038462 0.038462 0.038462 0.038462 0.038462 0.038462 LM1= 0.064103' ?-n-best-list run1.best100.out 100 distinct ?-input-file /mnt/a62/devel/en_es/data/corpora.tuning.en > run1.out
Segmentation fault (core dumped)
Exit code: 139
The decoder died. CONFIG WAS -weight-overwrite 'WordPenalty0= -0.128205 TranslationModel0= 0.025641 0.025641 0.025641 0.025641 LM2= 0.064103 LM0= 0.064103 GenerationModel1= 0.038462 0.000000 TranslationModel2= 0.025641 0.025641 0.025641 0.025641 GenerationModel0= 0.038462 PhrasePenalty0= 0.025641 Distortion0= 0.038462 TranslationModel1= 0.025641 0.025641 0.025641 0.025641 LexicalReordering0= 0.038462 0.038462 0.038462 0.038462 0.038462 0.038462 LM1= 0.064103'?
cp: cannot stat ?/mnt/a62/devel/en_es/processfin/tuning/tmp.1/moses.ini?: No such file or directory
-------------------------------------------
If I change this line in the config file from
decoding-steps = "t0,g0,t1,g1:t2"
?to
decoding-steps = "t0,g0,t1,g1"
then the training ends without errors.?
I'll appreciate suggestions on how to solve that.
Alex
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151013/8d556233/attachment-0001.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 108, Issue 48
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 108, Issue 48"
Post a Comment