Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Hierarchical Reordering model segmentation fault
(Barry Haddow)
2. Re: Error during Tuning (Barry Haddow)
3. Re: Error during Tuning (Barry Haddow)
4. Re: Error during Tuning (Barry Haddow)
----------------------------------------------------------------------
Message: 1
Date: Sun, 18 Aug 2013 17:27:50 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Hierarchical Reordering model
segmentation fault
To: xiaofeng wu <xiaofeng.wu11@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <20130818172750.18757a7pq2dgpzwg@www.staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
format="flowed"
Hi Xiaofeng
The hierarchical reordering has not been heavily used, as far as I
know, so there is a good chance that bugs remain in corner cases. The
code where the crash occurs has not been altered significantly in 3.5
years. Please let us know if you find out what the problem is,
cheers - Barry
Quoting xiaofeng wu <xiaofeng.wu11@gmail.com> on Fri, 16 Aug 2013
18:21:15 +0100:
> Hi mr
>
> When I run hierarchical reordering with phrase model, I got the following
> error:
>
> PhraseExtract v1.4, written by Philipp Koehn
> phrase extraction from an aligned parallel corpus
> *** Segmentation fault
> Register dump:
>
> RAX: 000000000000005c RBX: 0000000000000000 RCX: 00000000027335d0
> RDX: 000000000040d150 RSI: 00007fffbd6ee828 RDI: 00007fffbd6ee7f0
> RBP: 00007fffbd6ee708 R8 : 0000000000000002 R9 : 0000000000000002
> R10: 0000000002720f80 R11: 00007fffbd6ee730 R12: 00007fffbd6ee738
> R13: 00007fffbd6ee7c8 R14: 00007fffbd6ee7f8 R15: 0000000000000007
> RSP: 00007fffbd6ee390
>
> RIP: 000000000040de04 EFLAGS: 00010206
>
> CS: 0033 FS: 0000 GS: 0000
>
> Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 0000007c
>
> FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
> RIP: 00000000 RDP: 00000000
>
> ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
> ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
> ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
> ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
> mxcsr: 1f80
> XMM0: 0000000000000000000000007f757773 XMM1:
> 0000000000000000000000007f757773
> XMM2: 0000000000000000000000007f757773 XMM3:
> 0000000000000000000000007f757773
> XMM4: 0000000000000000000000007f757773 XMM5:
> 0000000000000000000000007f757773
> XMM6: 0000000000000000000000007f757773 XMM7:
> 0000000000000000000000007f757773
> XMM8: 0000000000000000000000007f757773 XMM9:
> 0000000000000000000000007f757773
> XMM10: 0000000000000000000000007f757773 XMM11:
> 0000000000000000000000007f757773
> XMM12: 0000000000000000000000007f757773 XMM13:
> 0000000000000000000000007f757773
> XMM14: 0000000000000000000000007f757773 XMM15:
> 0000000000000000000000007f757773
>
> Backtrace:
> /home/lly/plateform/moses/scripts/../bin/extract[0x40de04]
> /home/lly/plateform/moses/scripts/../bin/extract[0x413b94]
> /home/lly/plateform/moses/scripts/../bin/extract[0x40be2d]
> /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fbac28dd455]
> /home/lly/plateform/moses/scripts/../bin/extract[0x40d059]
>
> Memory map:
>
> 00400000-0045e000 r-xp 00000000 00:21 29066575
> /home/lly/plateform/moses/bin/extract
> 0065d000-0065e000 r--p 0005d000 00:21 29066575
> /home/lly/plateform/moses/bin/extract
> 0065e000-0065f000 rw-p 0005e000 00:21 29066575
> /home/lly/plateform/moses/bin/extract
> 0065f000-00660000 rw-p 00000000 00:00 0
> 02642000-0273a000 rw-p 00000000 00:00 0
> [heap]
> 7fbac26ad000-7fbac26bb000 r-xp 00000000 08:11 5036
> /usr/lib64/libbz2.so.1.0.6
> 7fbac26bb000-7fbac28ba000 ---p 0000e000 08:11 5036
> /usr/lib64/libbz2.so.1.0.6
> 7fbac28ba000-7fbac28bb000 r--p 0000d000 08:11 5036
> /usr/lib64/libbz2.so.1.0.6
> 7fbac28bb000-7fbac28bc000 rw-p 0000e000 08:11 5036
> /usr/lib64/libbz2.so.1.0.6
> 7fbac28bc000-7fbac2a57000 r-xp 00000000 08:11 524302
> /lib64/libc-2.15.so
> 7fbac2a57000-7fbac2c57000 ---p 0019b000 08:11 524302
> /lib64/libc-2.15.so
> 7fbac2c57000-7fbac2c5b000 r--p 0019b000 08:11 524302
> /lib64/libc-2.15.so
> 7fbac2c5b000-7fbac2c5d000 rw-p 0019f000 08:11 524302
> /lib64/libc-2.15.so
> 7fbac2c5d000-7fbac2c61000 rw-p 00000000 00:00 0
> 7fbac2c61000-7fbac2c78000 r-xp 00000000 08:11 524328
> /lib64/libpthread-2.15.so
> 7fbac2c78000-7fbac2e77000 ---p 00017000 08:11 524328
> /lib64/libpthread-2.15.so
> 7fbac2e77000-7fbac2e78000 r--p 00016000 08:11 524328
> /lib64/libpthread-2.15.so
> 7fbac2e78000-7fbac2e79000 rw-p 00017000 08:11 524328
> /lib64/libpthread-2.15.so
> 7fbac2e79000-7fbac2e7e000 rw-p 00000000 00:00 0
> 7fbac2e7e000-7fbac2e93000 r-xp 00000000 08:11 524446
> /lib64/libgcc_s.so.1
> 7fbac2e93000-7fbac3092000 ---p 00015000 08:11 524446
> /lib64/libgcc_s.so.1
> 7fbac3092000-7fbac3093000 r--p 00014000 08:11 524446
> /lib64/libgcc_s.so.1
> 7fbac3093000-7fbac3094000 rw-p 00015000 08:11 524446
> /lib64/libgcc_s.so.1
> 7fbac3094000-7fbac3189000 r-xp 00000000 08:11 524310
> /lib64/libm-2.15.so
> 7fbac3189000-7fbac3389000 ---p 000f5000 08:11 524310
> /lib64/libm-2.15.so
> 7fbac3389000-7fbac338a000 r--p 000f5000 08:11 524310
> /lib64/libm-2.15.so
> 7fbac338a000-7fbac338b000 rw-p 000f6000 08:11 524310
> /lib64/libm-2.15.so
> 7fbac338b000-7fbac3473000 r-xp 00000000 08:11 6602
> /usr/lib64/libstdc++.so.6.0.17
> 7fbac3473000-7fbac3673000 ---p 000e8000 08:11 6602
> /usr/lib64/libstdc++.so.6.0.17
> 7fbac3673000-7fbac367b000 r--p 000e8000 08:11 6602
> /usr/lib64/libstdc++.so.6.0.17
> 7fbac367b000-7fbac367d000 rw-p 000f0000 08:11 6602
> /usr/lib64/libstdc++.so.6.0.17
> 7fbac367d000-7fbac3692000 rw-p 00000000 00:00 0
> 7fbac3692000-7fbac3699000 r-xp 00000000 08:11 524332
> /lib64/librt-2.15.so
> 7fbac3699000-7fbac3898000 ---p 00007000 08:11 524332
> /lib64/librt-2.15.so
> 7fbac3898000-7fbac3899000 r--p 00006000 08:11 524332
> /lib64/librt-2.15.so
> 7fbac3899000-7fbac389a000 rw-p 00007000 08:11 524332
> /lib64/librt-2.15.so
> 7fbac389a000-7fbac38b2000 r-xp 00000000 08:11 7833
> /usr/lib64/libboost_iostreams.so.1.49.0
> 7fbac38b2000-7fbac3ab2000 ---p 00018000 08:11 7833
> /usr/lib64/libboost_iostreams.so.1.49.0
> 7fbac3ab2000-7fbac3ab3000 r--p 00018000 08:11 7833
> /usr/lib64/libboost_iostreams.so.1.49.0
> 7fbac3ab3000-7fbac3ab4000 rw-p 00019000 08:11 7833
> /usr/lib64/libboost_iostreams.so.1.49.0
> 7fbac3ab4000-7fbac3ac9000 r-xp 00000000 08:11 524340
> /lib64/libz.so.1.2.7
> 7fbac3ac9000-7fbac3cc8000 ---p 00015000 08:11 524340
> /lib64/libz.so.1.2.7
> 7fbac3cc8000-7fbac3cc9000 r--p 00014000 08:11 524340
> /lib64/libz.so.1.2.7
> 7fbac3cc9000-7fbac3cca000 rw-p 00015000 08:11 524340
> /lib64/libz.so.1.2.7
> 7fbac3cca000-7fbac3cce000 r-xp 00000000 08:11 524299
> /lib64/libSegFault.so
> 7fbac3cce000-7fbac3ecd000 ---p 00004000 08:11 524299
> /lib64/libSegFault.so
> 7fbac3ecd000-7fbac3ece000 r--p 00003000 08:11 524299
> /lib64/libSegFault.so
> 7fbac3ece000-7fbac3ecf000 rw-p 00004000 08:11 524299
> /lib64/libSegFault.so
> 7fbac3ecf000-7fbac3ee9000 r-xp 00000000 08:11 7817
> /usr/lib64/libboost_thread.so.1.49.0
> 7fbac3ee9000-7fbac40e8000 ---p 0001a000 08:11 7817
> /usr/lib64/libboost_thread.so.1.49.0
> 7fbac40e8000-7fbac40ea000 r--p 00019000 08:11 7817
> /usr/lib64/libboost_thread.so.1.49.0
> 7fbac40ea000-7fbac40eb000 rw-p 0001b000 08:11 7817
> /usr/lib64/libboost_thread.so.1.49.0
> 7fbac40eb000-7fbac40ee000 r-xp 00000000 08:11 7820
> /usr/lib64/libboost_system.so.1.49.0
> 7fbac40ee000-7fbac42ed000 ---p 00003000 08:11 7820
> /usr/lib64/libboost_system.so.1.49.0
> 7fbac42ed000-7fbac42ee000 r--p 00002000 08:11 7820
> /usr/lib64/libboost_system.so.1.49.0
> 7fbac42ee000-7fbac42ef000 rw-p 00003000 08:11 7820
> /usr/lib64/libboost_system.so.1.49.0
> 7fbac42ef000-7fbac4314000 r-xp 00000000 08:11 4928
> /usr/lib64/liblzma.so.5.0.3
> 7fbac4314000-7fbac4513000 ---p 00025000 08:11 4928
> /usr/lib64/liblzma.so.5.0.3
> 7fbac4513000-7fbac4514000 r--p 00024000 08:11 4928
> /usr/lib64/liblzma.so.5.0.3
> 7fbac4514000-7fbac4515000 rw-p 00025000 08:11 4928
> /usr/lib64/liblzma.so.5.0.3
> 7fbac4515000-7fbac4536000 r-xp 00000000 08:11 534085
> /lib64/ld-2.15.so
> 7fbac4710000-7fbac4718000 rw-p 00000000 00:00 0
> 7fbac4735000-7fbac4736000 rw-p 00000000 00:00 0
> 7fbac4736000-7fbac4737000 r--p 00021000 08:11 534085
> /lib64/ld-2.15.so
> 7fbac4737000-7fbac4738000 rw-p 00022000 08:11 534085
> /lib64/ld-2.15.so
> 7fbac4738000-7fbac4739000 rw-p 00000000 00:00 0
> 7fffbd6eb000-7fffbd8d9000 rw-p 00000000 00:00 0
> [stack]
> 7fffbd9ff000-7fffbda00000 r-xp 00000000 00:00 0
> [vdso]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
> [vsyscall]
> Segmentation fault
> ..................................................................................................................................
> when I go to gdb.
>
> (gdb) bt
> #0 _M_lower_bound (__y=<optimized out>, __k=<synthetic pointer>, __x=0x5c,
> this=<optimized out>)
> at /usr/include/c++/4.7/bits/stl_tree.h:1114
> #1 find (__k=<synthetic pointer>, this=0x7fffffe15170) at
> /usr/include/c++/4.7/bits/stl_tree.h:1557
> #2 find (__x=<synthetic pointer>, this=0x7fffffe15170) at
> /usr/include/c++/4.7/bits/stl_set.h:617
> #3 MosesTraining::getOrientHierModel (sentence=..., modelType=<optimized
> out>,
> connectedLeftTop=connectedLeftTop@entry=false,
> connectedRightTop=connectedRightTop@entry=false,
> startF=startF@entry=2, endF=endF@entry=2, startE=startE@entry=6,
> endE=endE@entry=6, countF=countF@entry=0,
> zero=zero@entry=49, unit=unit@entry=-1, ge=ge@entry=0x40d150
> <MosesTraining::lt(int, int)>,
> Python Exception <type 'exceptions.IndexError'> list index out of range:
> Python Exception <type 'exceptions.IndexError'> list index out of range:
> lt=lt@entry=0x40d140 <MosesTraining::ge(int, int)>,
> inBottomRight=std::map with 47 elements, inBottomLeft=
> Python Exception <type 'exceptions.IndexError'> list index out of range:
> Python Exception <type 'exceptions.IndexError'> list index out of range:
> std::map with 47 elements, outBottomRight=std::map with 29 elements,
> outBottomLeft=std::map with 29 elements,
> phraseOrient=phraseOrient@entry=MosesTraining::UNKNOWN) at
> phrase-extract/extract-main.cpp:574
> #4 0x0000000000413b94 in MosesTraining::ExtractTask::extract
> (this=this@entry=0x73b110, sentence=...)
> at phrase-extract/extract-main.cpp:454
> #5 0x000000000040be2d in Run (this=0x73b110) at
> phrase-extract/extract-main.cpp:303
> #6 main (argc=<optimized out>, argv=<optimized out>) at
> phrase-extract/extract-main.cpp:275
>
>
> Best
> --
> Xiaofeng Wu
> CNGL, School of Computing,
> Dublin City University,
> Glasnevin, Dublin 9.
> Email: xiaofengwu@computing.dcu.ie
> Tel: +353 (0)1 700 6715
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 2
Date: Sun, 18 Aug 2013 17:39:29 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Error during Tuning
To: Heidi Heweidy <heidi.heweidy@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <20130818173929.17351dl3ppuxju9w@www.staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
format="flowed"
Hi Heidi
Inside the mert working directory, there should be a file called
extract.err. Look at the error message in this file.
It could be that the input and reference you are using for tuning are
mismatched,
cheers - Barry
Quoting Heidi Heweidy <heidi.heweidy@gmail.com> on Sun, 18 Aug 2013
16:12:33 +0200:
> I have an arabic to english system that works fine after training but when
> I start tuning i end up with this in the mert.out file:
> ERROR: Failed to run '/home/tjr/working/mert-work/extractor.sh'. at
> /home/tjr/mmosesdecoder/scripts/training/mert-moses.pl line 1554.
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 3
Date: Sun, 18 Aug 2013 17:53:13 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Error during Tuning
To: Heidi Heweidy <heidi.heweidy@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <20130818175313.209015ps6kqs7rpc@www.staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
format="flowed"
Hi Heidi
You have to supply an input set and a reference set to mert-moses.pl
for tuning. This error suggests that they have different numbers of
lines in them - so they are not parallel,
cheers - Barry
Quoting Heidi Heweidy <heidi.heweidy@gmail.com> on Sun, 18 Aug 2013
18:45:31 +0200:
> Inside of it, i get:
>
> Binary write mode is NOT selected
> Scorer type: BLEU
> name: case value: true
> Loading reference from /home/tjr/corpus/ar-en.tune.true.en
> ............................Data::m_score_type BLEU
> Data::Scorer type from Scorer: BLEU
> loading nbest from run1.best100.out.gz
> Exception: Sentence id (2844) not found in reference set
>
> I do not get the exception, which reference set is this referring to and
> what does it mean that it is not found?
>
>
> On Sun, Aug 18, 2013 at 6:39 PM, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk>wrote:
>
>> Hi Heidi
>>
>> Inside the mert working directory, there should be a file called
>> extract.err. Look at the error message in this file.
>>
>> It could be that the input and reference you are using for tuning are
>> mismatched,
>>
>> cheers - Barry
>>
>>
>> Quoting Heidi Heweidy <heidi.heweidy@gmail.com> on Sun, 18 Aug 2013
>> 16:12:33 +0200:
>>
>> I have an arabic to english system that works fine after training but when
>>> I start tuning i end up with this in the mert.out file:
>>> ERROR: Failed to run '/home/tjr/working/mert-work/**extractor.sh'. at
>>> /home/tjr/mmosesdecoder/**scripts/training/mert-moses.pl line 1554.
>>>
>>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 4
Date: Sun, 18 Aug 2013 18:05:34 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Error during Tuning
To: Heidi Heweidy <heidi.heweidy@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <20130818180534.888317qtafi6y8qo@www.staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
format="flowed"
Hi Heidi
Can you give the exact argument that you use to run tuning?
cheers - Barry
Quoting Heidi Heweidy <heidi.heweidy@gmail.com> on Sun, 18 Aug 2013
18:55:59 +0200:
> my training set have the same number of lines, same goes for my tuning set,
> but each set is not the same number of lines as the other. i dont see the
> problem because in the moses baseline tutorial, this is how it works too,
> am i wrong?
>
>
> On Sun, Aug 18, 2013 at 6:53 PM, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk>wrote:
>
>> Hi Heidi
>>
>> You have to supply an input set and a reference set to mert-moses.pl for
>> tuning. This error suggests that they have different numbers of lines in
>> them - so they are not parallel,
>>
>> cheers - Barry
>>
>>
>> Quoting Heidi Heweidy <heidi.heweidy@gmail.com> on Sun, 18 Aug 2013
>> 18:45:31 +0200:
>>
>> Inside of it, i get:
>>>
>>> Binary write mode is NOT selected
>>> Scorer type: BLEU
>>> name: case value: true
>>> Loading reference from /home/tjr/corpus/ar-en.tune.**true.en
>>> ............................**Data::m_score_type BLEU
>>> Data::Scorer type from Scorer: BLEU
>>> loading nbest from run1.best100.out.gz
>>> Exception: Sentence id (2844) not found in reference set
>>>
>>> I do not get the exception, which reference set is this referring to and
>>> what does it mean that it is not found?
>>>
>>>
>>> On Sun, Aug 18, 2013 at 6:39 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk
>>> >**wrote:
>>>
>>> Hi Heidi
>>>>
>>>> Inside the mert working directory, there should be a file called
>>>> extract.err. Look at the error message in this file.
>>>>
>>>> It could be that the input and reference you are using for tuning are
>>>> mismatched,
>>>>
>>>> cheers - Barry
>>>>
>>>>
>>>> Quoting Heidi Heweidy <heidi.heweidy@gmail.com> on Sun, 18 Aug 2013
>>>> 16:12:33 +0200:
>>>>
>>>> I have an arabic to english system that works fine after training but
>>>> when
>>>>
>>>>> I start tuning i end up with this in the mert.out file:
>>>>> ERROR: Failed to run '/home/tjr/working/mert-work/****extractor.sh'. at
>>>>> /home/tjr/mmosesdecoder/****scripts/training/mert-moses.pl line 1554.
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> The University of Edinburgh is a charitable body, registered in
>>>> Scotland, with registration number SC005336.
>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 82, Issue 23
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 82, Issue 23"
Post a Comment