Home » Uncategories » Moses-support Digest, Vol 98, Issue 16

Moses-support Digest, Vol 98, Issue 16

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Segmentation fault when loading generation model (Steven Huang)
2. Re: Segmentation fault when loading generation model (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 2 Dec 2014 10:39:08 +0800
From: Steven Huang <d98922047@ntu.edu.tw>
Subject: [Moses-support] Segmentation fault when loading generation
model
To: Moses-support@mit.edu, ??? <farmer.tw@gmail.com>
Message-ID:
<CAG-iPUqau_0vspXpr7RTKROmZ+9sh=+ewhKO1QUsZ28C-DVYuQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

When I am trying to start moses, segmentation fault is reported when it
comes to load generation model (see the attached file "err_log.txt" and
"moses.ini").

However, if I use smaller corpus to do training with the same
configuration, moses can start normally.

What is the probable cause?
Thanks a lot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141202/4911aa8f/attachment-0001.htm
-------------- next part --------------
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses -f train/model/moses.ini
Defined parameters (per moses.ini or switch):
config: train/model/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/tmp2/p00922006/1128a/train/model/phrase-table.1,2-1,2.gz input-factor=1,2 output-factor=1,2 Generation name=GenerationModel0 num-features=2 path=/tmp2/p00922006/1128a/train/model/generation.1,2-0.gz input-factor=1,2 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0,2 output-factor=0,2 path=/tmp2/p00922006/1128a/train/model/reordering-table.0,2-0,2.wbe-msd-bidirectional-fe.gz Distortion KENLM lazyken=0 name=LM0 factor=0 path=/tmp2/p00922006/1128Mod/utf8/en-ch-surface.blm.ch order=3 KENLM lazyken=0 name=LM1 factor=2 path=/tmp2/p00922006/1128Mod/utf8.pos/en-ch-pos.blm.ch order=3
input-factors: 0 1 2
mapping: 0 T 0 0 G 0
weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 GenerationModel0= 0.3 0 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3 Distortion0= 0.3 LM0= 0.5 LM1= 0.5
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/tmp2/p00922006/1128a/train/model/phrase-table.1,2-1,2.gz input-factor=1,2 output-factor=1,2
FeatureFunction: TranslationModel0 start: 3 end: 6
line=Generation name=GenerationModel0 num-features=2 path=/tmp2/p00922006/1128a/train/model/generation.1,2-0.gz input-factor=1,2 output-factor=0
FeatureFunction: GenerationModel0 start: 7 end: 8
line=LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0,2 output-factor=0,2 path=/tmp2/p00922006/1128a/train/model/reordering-table.0,2-0,2.wbe-msd-bidirectional-fe.gz
FeatureFunction: LexicalReordering0 start: 9 end: 14
Initializing LexicalReordering..
line=Distortion
FeatureFunction: Distortion0 start: 15 end: 15
line=KENLM lazyken=0 name=LM0 factor=0 path=/tmp2/p00922006/1128Mod/utf8/en-ch-surface.blm.ch order=3
FeatureFunction: LM0 start: 16 end: 16
line=KENLM lazyken=0 name=LM1 factor=2 path=/tmp2/p00922006/1128Mod/utf8.pos/en-ch-pos.blm.ch order=3
FeatureFunction: LM1 start: 17 end: 17
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading GenerationModel0
*** Segmentation fault
Register dump:

RAX: 00000000009c6610 RBX: 00007fff75b3f0d0 RCX: 0000000012c836c0
RDX: 0000000000000000 RSI: 00007fff75b3f550 RDI: 00000000008fb880
RBP: 0000000000000000 R8 : 0000000000000000 R9 : 0000000012c84600
R10: 0000000000000000 R11: 00007fda40419ab0 R12: 0000000000000001
R13: 00007fff75b3f550 R14: 00007fff75b3f5a5 R15: 0000000000000002
RSP: 00007fff75b3eec0

RIP: 000000000043d1b9 EFLAGS: 00010246

CS: 0033 FS: 0000 GS: 0000

Trap: 0000000e Error: 00000004 OldMask: 00000000 CR2: 00000000

FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
RIP: 00000000 RDP: 00000000

ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
mxcsr: 1fa4
XMM0: 000000000000000000000000000000ff XMM1: 000000000000000000000000000000ff
XMM2: 000000000000000000000000000000ff XMM3: 000000000000000000000000000000ff
XMM4: 000000000000000000000000000000ff XMM5: 000000000000000000000000000000ff
XMM6: 000000000000000000000000000000ff XMM7: 000000000000000000000000000000ff
XMM8: 000000000000000000000000000000ff XMM9: 000000000000000000000000000000ff
XMM10: 000000000000000000000000000000ff XMM11: 000000000000000000000000000000ff
XMM12: 000000000000000000000000000000ff XMM13: 000000000000000000000000000000ff
XMM14: 000000000000000000000000000000ff XMM15: 000000000000000000000000000000ff

Backtrace:
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses[0x43d1b9]
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses[0x4a8bf6]
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses[0x4b16b8]
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses[0x4b3184]
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses[0x40ce62]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fda402cfb45]
/home/master/00/p00922006/langTools/mosesdecoder/bin/moses[0x40be39]

Memory map:

00400000-006fa000 r-xp 00000000 00:29 68821615 /nfs/master/00/p00922006/langTools/mosesdecoder/bin/moses
008f9000-008fb000 rw-p 002f9000 00:29 68821615 /nfs/master/00/p00922006/langTools/mosesdecoder/bin/moses
008fb000-008fd000 rw-p 00000000 00:00 0
009b1000-12c8c000 rw-p 00000000 00:00 0 [heap]
7fd9eff2e000-7fd9f668d000 rw-p 00000000 00:00 0
7fd9f69e0000-7fd9fa6e5000 rw-p 00000000 00:00 0
7fd9fa7c3000-7fd9fa9c4000 rw-p 00000000 00:00 0
7fd9fa9c4000-7fda402ae000 r--s 00000000 08:03 131228 /tmp2/p00922006/1128Mod/utf8/en-ch-surface.blm.ch
7fda402ae000-7fda4044d000 r-xp 00000000 08:01 1445358 /lib/x86_64-linux-gnu/libc-2.19.so
7fda4044d000-7fda4064d000 ---p 0019f000 08:01 1445358 /lib/x86_64-linux-gnu/libc-2.19.so
7fda4064d000-7fda40651000 r--p 0019f000 08:01 1445358 /lib/x86_64-linux-gnu/libc-2.19.so
7fda40651000-7fda40653000 rw-p 001a3000 08:01 1445358 /lib/x86_64-linux-gnu/libc-2.19.so
7fda40653000-7fda40657000 rw-p 00000000 00:00 0
7fda40657000-7fda4066f000 r-xp 00000000 08:01 1445032 /lib/x86_64-linux-gnu/libpthread-2.19.so
7fda4066f000-7fda4086e000 ---p 00018000 08:01 1445032 /lib/x86_64-linux-gnu/libpthread-2.19.so
7fda4086e000-7fda4086f000 r--p 00017000 08:01 1445032 /lib/x86_64-linux-gnu/libpthread-2.19.so
7fda4086f000-7fda40870000 rw-p 00018000 08:01 1445032 /lib/x86_64-linux-gnu/libpthread-2.19.so
7fda40870000-7fda40874000 rw-p 00000000 00:00 0
7fda40874000-7fda4088a000 r-xp 00000000 08:01 1445182 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fda4088a000-7fda40a89000 ---p 00016000 08:01 1445182 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fda40a89000-7fda40a8a000 rw-p 00015000 08:01 1445182 /lib/x86_64-linux-gnu/libgcc_s.so.1
7fda40a8a000-7fda40b8a000 r-xp 00000000 08:01 1445384 /lib/x86_64-linux-gnu/libm-2.19.so
7fda40b8a000-7fda40d89000 ---p 00100000 08:01 1445384 /lib/x86_64-linux-gnu/libm-2.19.so
7fda40d89000-7fda40d8a000 r--p 000ff000 08:01 1445384 /lib/x86_64-linux-gnu/libm-2.19.so
7fda40d8a000-7fda40d8b000 rw-p 00100000 08:01 1445384 /lib/x86_64-linux-gnu/libm-2.19.so
7fda40d8b000-7fda40e77000 r-xp 00000000 08:01 394660 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7fda40e77000-7fda41077000 ---p 000ec000 08:01 394660 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7fda41077000-7fda4107f000 r--p 000ec000 08:01 394660 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7fda4107f000-7fda41081000 rw-p 000f4000 08:01 394660 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20
7fda41081000-7fda41096000 rw-p 00000000 00:00 0
7fda41096000-7fda4109d000 r-xp 00000000 08:01 1445918 /lib/x86_64-linux-gnu/librt-2.19.so
7fda4109d000-7fda4129c000 ---p 00007000 08:01 1445918 /lib/x86_64-linux-gnu/librt-2.19.so
7fda4129c000-7fda4129d000 r--p 00006000 08:01 1445918 /lib/x86_64-linux-gnu/librt-2.19.so
7fda4129d000-7fda4129e000 rw-p 00007000 08:01 1445918 /lib/x86_64-linux-gnu/librt-2.19.so
7fda4129e000-7fda412b1000 r-xp 00000000 00:29 68823015 /nfs/master/00/p00922006/langTools/boost/libboost_thread-mt.so.5
7fda412b1000-7fda414b1000 ---p 00013000 00:29 68823015 /nfs/master/00/p00922006/langTools/boost/libboost_thread-mt.so.5
7fda414b1000-7fda414b3000 rw-p 00013000 00:29 68823015 /nfs/master/00/p00922006/langTools/boost/libboost_thread-mt.so.5
7fda414b3000-7fda414b7000 r-xp 00000000 08:01 1445964 /lib/x86_64-linux-gnu/libSegFault.so
7fda414b7000-7fda416b6000 ---p 00004000 08:01 1445964 /lib/x86_64-linux-gnu/libSegFault.so
7fda416b6000-7fda416b7000 r--p 00003000 08:01 1445964 /lib/x86_64-linux-gnu/libSegFault.so
7fda416b7000-7fda416b8000 rw-p 00004000 08:01 1445964 /lib/x86_64-linux-gnu/libSegFault.so
7fda416b8000-7fda416bb000 r-xp 00000000 00:29 68823007 /nfs/master/00/p00922006/langTools/boost/libboost_system-mt.so.5
7fda416bb000-7fda418ba000 ---p 00003000 00:29 68823007 /nfs/master/00/p00922006/langTools/boost/libboost_system-mt.so.5
7fda418ba000-7fda418bb000 rw-p 00002000 00:29 68823007 /nfs/master/00/p00922006/langTools/boost/libboost_system-mt.so.5
7fda418bb000-7fda418be000 r-xp 00000000 08:01 1445383 /lib/x86_64-linux-gnu/libdl-2.19.so
7fda418be000-7fda41abd000 ---p 00003000 08:01 1445383 /lib/x86_64-linux-gnu/libdl-2.19.so
7fda41abd000-7fda41abe000 r--p 00002000 08:01 1445383 /lib/x86_64-linux-gnu/libdl-2.19.so
7fda41abe000-7fda41abf000 rw-p 00003000 08:01 1445383 /lib/x86_64-linux-gnu/libdl-2.19.so
7fda41abf000-7fda41ace000 r-xp 00000000 08:01 1446504 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fda41ace000-7fda41ccd000 ---p 0000f000 08:01 1446504 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fda41ccd000-7fda41cce000 r--p 0000e000 08:01 1446504 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fda41cce000-7fda41ccf000 rw-p 0000f000 08:01 1446504 /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7fda41ccf000-7fda41ce3000 r-xp 00000000 00:29 68822887 /nfs/master/00/p00922006/langTools/boost/libboost_filesystem-mt.so.5
7fda41ce3000-7fda41ee3000 ---p 00014000 00:29 68822887 /nfs/master/00/p00922006/langTools/boost/libboost_filesystem-mt.so.5
7fda41ee3000-7fda41ee4000 rw-p 00014000 00:29 68822887 /nfs/master/00/p00922006/langTools/boost/libboost_filesystem-mt.so.5
7fda41ee4000-7fda41ef1000 r-xp 00000000 00:29 68822903 /nfs/master/00/p00922006/langTools/boost/libboost_iostreams-mt.so.5
7fda41ef1000-7fda420f1000 ---p 0000d000 00:29 68822903 /nfs/master/00/p00922006/langTools/boost/libboost_iostreams-mt.so.5
7fda420f1000-7fda420f2000 rw-p 0000d000 00:29 68822903 /nfs/master/00/p00922006/langTools/boost/libboost_iostreams-mt.so.5
7fda420f2000-7fda4210c000 r-xp 00000000 08:01 1444986 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fda4210c000-7fda4230b000 ---p 0001a000 08:01 1444986 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fda4230b000-7fda4230c000 r--p 00019000 08:01 1444986 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fda4230c000-7fda4230d000 rw-p 0001a000 08:01 1444986 /lib/x86_64-linux-gnu/libz.so.1.2.8
7fda4230d000-7fda4232d000 r-xp 00000000 08:01 1445072 /lib/x86_64-linux-gnu/ld-2.19.so
7fda4234b000-7fda423ac000 r--s 00000000 08:03 131132 /tmp2/p00922006/1128Mod/utf8.pos/en-ch-pos.blm.ch
7fda423ac000-7fda423ed000 rw-p 00000000 00:00 0
7fda4241c000-7fda424be000 rw-p 00000000 00:00 0
7fda424ef000-7fda424f8000 rw-p 00000000 00:00 0
7fda4252b000-7fda4252d000 rw-p 00000000 00:00 0
7fda4252d000-7fda4252e000 r--p 00020000 08:01 1445072 /lib/x86_64-linux-gnu/ld-2.19.so
7fda4252e000-7fda4252f000 rw-p 00021000 08:01 1445072 /lib/x86_64-linux-gnu/ld-2.19.so
7fda4252f000-7fda42530000 rw-p 00000000 00:00 0
7fff75b21000-7fff75b42000 rw-p 00000000 00:00 0 [stack]
7fff75bf2000-7fff75bf4000 r-xp 00000000 00:00 0 [vdso]
7fff75bf4000-7fff75bf6000 r--p 00000000 00:00 0 [vvar]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini
Type: application/octet-stream
Size: 1238 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141202/4911aa8f/attachment-0001.obj

------------------------------

Message: 2
Date: Tue, 02 Dec 2014 09:38:59 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Segmentation fault when loading
generation model
To: Steven Huang <d98922047@ntu.edu.tw>, Moses-support@mit.edu, ???
<farmer.tw@gmail.com>
Message-ID: <547D88B3.40704@gmail.com>
Content-Type: text/plain; charset="utf-8"

the data used to train the generation table - has it been tokenized and
reserved characters escaped?

you can try to track down exactly which line in the generation table is
causing the problem by adding debug information to
GenerationDictionary::Load()

On 02/12/14 02:39, Steven Huang wrote:
> Hi,
>
> When I am trying to start moses, segmentation fault is reported when
> it comes to load generation model (see the attached file "err_log.txt"
> and "moses.ini").
>
> However, if I use smaller corpus to do training with the same
> configuration, moses can start normally.
>
> What is the probable cause?
> Thanks a lot.
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141202/4b323093/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 98, Issue 16
*********************************************

0 Response to "Moses-support Digest, Vol 98, Issue 16"

Post a Comment

Subscribe to: Post Comments (Atom)