Home » Uncategories » Moses-support Digest, Vol 101, Issue 42

Moses-support Digest, Vol 101, Issue 42

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Unique k-best lists in moses? (Chris Dyer)
2. lattice decoding (Matt Post)
3. Re: Moses segmentation fault under multi-thread (Liangyou Li)


----------------------------------------------------------------------

Message: 1
Date: Thu, 12 Mar 2015 16:05:39 -0400
From: Chris Dyer <cdyer@cs.cmu.edu>
Subject: Re: [Moses-support] Unique k-best lists in moses?
To: Philip Williams <philip.williams@mac.com>, Guillaume Lample
<guillaume.lample@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEHEvxME5_3oiGmN-JsyfYMipC7cgd+WwG3xBGOH6LS3b88f-g@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Thanks, Phil. I had forgotten about the "distinct" option, so this was
very helpful, even if it's a bit slow. It's alright for us.

On Wed, Mar 11, 2015 at 6:56 PM, Philip Williams
<philip.williams@mac.com> wrote:
> Hi Chris,
>
> if you add the argument "distinct" to the -n-best-list option then that
> should have the same effect, albeit far less efficiently (internally,
> Moses simply generates a longer list and then filters it).
>
> I'm guessing you already knew about that option though. If that's too
> slow for what you're doing then give me a day or two and I'll find some
> time to implement the Huang et al. trick. Clearly, that's what we
> should be doing anyway.
>
> Oh wait, are you talking about phrase-based or syntax-based? For
> phrase-based we don't -- but obviously could -- use Huang and Chiang's
> (2005) algorithm for k-best extraction. As things stand, doing the
> Huang et al. unique trick in Moses would involve more effort to
> implement for phrase-based than for syntax-based.
>
> Phil
>
> On 11 Mar 2015, at 20:03, Chris Dyer <cdyer@cs.cmu.edu> wrote:
>
>> Hi all, can anyone tell me if moses can be configured to generate
>> "unique" k-best lists using the Huang et al. (2006) dynamic
>> programming algorithm (Section 5.2 of
>> http://www.cis.upenn.edu/~lhuang3/amta06-sdtedl.pdf)?
>> Thanks--
>> Chris
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 2
Date: Thu, 12 Mar 2015 17:35:46 -0400
From: Matt Post <post@cs.jhu.edu>
Subject: [Moses-support] lattice decoding
To: moses-support@mit.edu
Message-ID: <46A4A2A8-4D05-4BB5-9212-9D34E8BA6CE7@cs.jhu.edu>
Content-Type: text/plain; charset=us-ascii

Hi,

Is there a technical writeup of Moses' phrase-based lattice decoding? The only real description I could find is Chris Dyer's "Generalizing Word Lattice Translation" paper, and that is quite high level.

Matt


------------------------------

Message: 3
Date: Thu, 12 Mar 2015 23:36:36 +0000
From: Liangyou Li <llysuda@gmail.com>
Subject: Re: [Moses-support] Moses segmentation fault under
multi-thread
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CAHeunNU_cEFVgReujsrNCUPeW1HVf5fF3hiO4q_p9DfH4wOHsg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Barry

I tried to use valgrind on a small sample (100 sent.), but there's no any
error.
Since it happens kind of randomly, I would re-test it.
I also run the decoder on another dataset, the error appear again.

The command to compile Boost_1_55_0 is the same as the one on Moses
webpage. After setting the BOOST_BUILD_PATH and BOOST_ROOT to this local
boost dir, I compile moses with command:

/home/lly/plateform/boost_1_55_0/bjam -j8
--with-srilm=/home/lly/plateform/srilm
--with-boost=/home/lly/plateform/boost_1_55_0 -q variant=debug

The error msg is as follows:

*** Segmentation fault
Register dump:

RAX: 00007f0dd4812210 RBX: 0000000000000000 RCX: 0000000000000006
RDX: 00007f0d9fceeab0 RSI: 0000000000000000 RDI: 00007f0dd4812210
RBP: 00007f0dd48121c0 R8 : 00007f0de01bd030 R9 : 00007f0de01a2c90
R10: 00007f0de01bd030 R11: 0000000000000000 R12: 0000000000000000
R13: 00007fff25dc8e90 R14: 0000000000000000 R15: 00007f0dd4813700
RSP: 00007f0dd48121a8

RIP: 00007f0ddff505e3 EFLAGS: 00010202

CS: 0033 FS: 0000 GS: 0000

Trap: 0000000e Error: 00000005 OldMask: 00000000 CR2: ffffffe8

FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
RIP: 00000000 RDP: 00000000

ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
mxcsr: 1fa4
XMM0: 00000000000000000000000000000000 XMM1:
00000000000000000000000000000000
XMM2: 00000000000000000000000000000000 XMM3:
00000000000000000000000000000000
XMM4: 00000000000000000000000000000000 XMM5:
00000000000000000000000000000000
XMM6: 00000000000000000000000000000000 XMM7:
00000000000000000000000000000000
XMM8: 00000000000000000000000000000000 XMM9:
00000000000000000000000000000000
XMM10: 00000000000000000000000000000000 XMM11:
00000000000000000000000000000000
XMM12: 00000000000000000000000000000000 XMM13:
00000000000000000000000000000000
XMM14: 00000000000000000000000000000000 XMM15:
00000000000000000000000000000000

Backtrace:
/usr/lib64/libstdc++.so.6(_ZStlsIcSt11char_traitsIcESaIcEERSt13basic_ostreamIT_T0_ES7_RKSbIS4_S5_T1_E+0x3)[0x7f0ddff505e3]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x58400d]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x43ffd6]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x44057e]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x4402fe]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x561d16]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x561a08]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x47baa2]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x530bdd]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534f41]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534ea4]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534e49]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534e0e]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x8b3892]
/lib64/libpthread.so.0(+0x7e0e)[0x7f0ddf796e0e]
/lib64/libc.so.6(clone+0x6d)[0x7f0ddf4ce2cd]

Memory map:

00400000-00bac000 r-xp 00000000 00:21 36148887
/home/lly/plateform/moses-3.0/bin/moses
00dab000-00dac000 r--p 007ab000 00:21 36148887
/home/lly/plateform/moses-3.0/bin/moses
00dac000-00dae000 rw-p 007ac000 00:21 36148887
/home/lly/plateform/moses-3.0/bin/moses
00dae000-00e14000 rw-p 00000000 00:00 0
01d11000-7725c000 rw-p 00000000 00:00 0
[heap]
7f0ca8000000-7f0ca96ef000 rw-p 00000000 00:00 0
7f0ca96ef000-7f0cac000000 ---p 00000000 00:00 0
7f0cb0000000-7f0cb208c000 rw-p 00000000 00:00 0
7f0cb208c000-7f0cb4000000 ---p 00000000 00:00 0
7f0cb8000000-7f0cbc000000 rw-p 00000000 00:00 0
7f0cbc000000-7f0cbdb78000 rw-p 00000000 00:00 0
7f0cbdb78000-7f0cc0000000 ---p 00000000 00:00 0
7f0cc0000000-7f0cc310b000 rw-p 00000000 00:00 0
7f0cc310b000-7f0cc4000000 ---p 00000000 00:00 0
7f0cc4000000-7f0cc7c46000 rw-p 00000000 00:00 0
7f0cc7c46000-7f0cc8000000 ---p 00000000 00:00 0
7f0cc8000000-7f0ccc000000 rw-p 00000000 00:00 0
7f0ccc000000-7f0cd0000000 rw-p 00000000 00:00 0
7f0cd0000000-7f0cd264f000 rw-p 00000000 00:00 0
7f0cd264f000-7f0cd4000000 ---p 00000000 00:00 0
7f0cd8000000-7f0cda190000 rw-p 00000000 00:00 0
7f0cda190000-7f0cdc000000 ---p 00000000 00:00 0
7f0cdc000000-7f0cddec3000 rw-p 00000000 00:00 0
7f0cddec3000-7f0ce0000000 ---p 00000000 00:00 0
7f0ce0000000-7f0ce4000000 rw-p 00000000 00:00 0
7f0ce4000000-7f0ce8000000 rw-p 00000000 00:00 0
7f0ce8000000-7f0cec000000 rw-p 00000000 00:00 0
7f0cec000000-7f0cf0000000 rw-p 00000000 00:00 0
7f0cf0000000-7f0cf23d9000 rw-p 00000000 00:00 0
7f0cf23d9000-7f0cf4000000 ---p 00000000 00:00 0
7f0cf4000000-7f0cf8000000 rw-p 00000000 00:00 0
7f0cfc000000-7f0d00000000 rw-p 00000000 00:00 0
7f0d00000000-7f0d04000000 rw-p 00000000 00:00 0
7f0d04000000-7f0d08000000 rw-p 00000000 00:00 0
7f0d0c000000-7f0d0f4f0000 rw-p 00000000 00:00 0
7f0d0f4f0000-7f0d10000000 ---p 00000000 00:00 0
7f0d14000000-7f0d17d65000 rw-p 00000000 00:00 0
7f0d17d65000-7f0d18000000 ---p 00000000 00:00 0
7f0d1c000000-7f0d20000000 rw-p 00000000 00:00 0
7f0d20000000-7f0d24000000 rw-p 00000000 00:00 0
7f0d24000000-7f0d27f50000 rw-p 00000000 00:00 0
7f0d27f50000-7f0d28000000 ---p 00000000 00:00 0
7f0d28000000-7f0d2c000000 rw-p 00000000 00:00 0
7f0d2c000000-7f0d2dda8000 rw-p 00000000 00:00 0
7f0d2dda8000-7f0d30000000 ---p 00000000 00:00 0
7f0d30000000-7f0d34000000 rw-p 00000000 00:00 0
7f0d34000000-7f0d38000000 rw-p 00000000 00:00 0
7f0d38000000-7f0d3a518000 rw-p 00000000 00:00 0
7f0d3a518000-7f0d3c000000 ---p 00000000 00:00 0
7f0d3c000000-7f0d40000000 rw-p 00000000 00:00 0
7f0d40000000-7f0d44000000 rw-p 00000000 00:00 0
7f0d44000000-7f0d48000000 rw-p 00000000 00:00 0
7f0d4c000000-7f0d50000000 rw-p 00000000 00:00 0
7f0d50000000-7f0d54000000 rw-p 00000000 00:00 0
7f0d54000000-7f0d58000000 rw-p 00000000 00:00 0
7f0d58000000-7f0d5b80d000 rw-p 00000000 00:00 0
7f0d5b80d000-7f0d5c000000 ---p 00000000 00:00 0
7f0d5c000000-7f0d60000000 rw-p 00000000 00:00 0
7f0d60000000-7f0d64000000 rw-p 00000000 00:00 0
7f0d64000000-7f0d6714a000 rw-p 00000000 00:00 0
7f0d6714a000-7f0d68000000 ---p 00000000 00:00 0
7f0d68000000-7f0d6c000000 rw-p 00000000 00:00 0
7f0d6c000000-7f0d70000000 rw-p 00000000 00:00 0
7f0d70000000-7f0d74000000 rw-p 00000000 00:00 0
7f0d74000000-7f0d78000000 rw-p 00000000 00:00 0
7f0d78000000-7f0d7c000000 rw-p 00000000 00:00 0
7f0d7c000000-7f0d80000000 rw-p 00000000 00:00 0
7f0d80000000-7f0d84000000 rw-p 00000000 00:00 0
7f0d84000000-7f0d88000000 rw-p 00000000 00:00 0
7f0d88000000-7f0d8c000000 rw-p 00000000 00:00 0
7f0d8c000000-7f0d90000000 rw-p 00000000 00:00 0
7f0d90000000-7f0d94000000 rw-p 00000000 00:00 0
7f0d94000000-7f0d98000000 rw-p 00000000 00:00 0
7f0d98000000-7f0d9c000000 rw-p 00000000 00:00 0
7f0d9c000000-7f0da0000000 rw-p 00000000 00:00 0
7f0da0000000-7f0da4000000 rw-p 00000000 00:00 0
7f0da4000000-7f0da8000000 rw-p 00000000 00:00 0
7f0da8000000-7f0dac000000 rw-p 00000000 00:00 0
7f0dac000000-7f0db0000000 rw-p 00000000 00:00 0
7f0db0000000-7f0db4000000 rw-p 00000000 00:00 0
7f0db4000000-7f0db8000000 rw-p 00000000 00:00 0
7f0db8000000-7f0dbc000000 rw-p 00000000 00:00 0
7f0dbc000000-7f0dc0000000 rw-p 00000000 00:00 0
7f0dc0000000-7f0dc4000000 rw-p 00000000 00:00 0
7f0dc4000000-7f0dc8000000 rw-p 00000000 00:00 0
7f0dc8000000-7f0dcc000000 rw-p 00000000 00:00 0
7f0dcc000000-7f0dd0000000 rw-p 00000000 00:00 0
7f0dd3011000-7f0dd3012000 ---p 00000000 00:00 0
7f0dd3012000-7f0dd3812000 rw-p 00000000 00:00 0
[stack:2289]
7f0dd3812000-7f0dd3813000 ---p 00000000 00:00 0
7f0dd3813000-7f0dd4013000 rw-p 00000000 00:00 0
[stack:2288]
7f0dd4013000-7f0dd4014000 ---p 00000000 00:00 0
7f0dd4014000-7f0dd4814000 rw-p 00000000 00:00 0
[stack:2287]
7f0dd4814000-7f0dd4815000 ---p 00000000 00:00 0
7f0dd4815000-7f0dd5015000 rw-p 00000000 00:00 0
[stack:2286]
7f0dd5015000-7f0dd5016000 ---p 00000000 00:00 0
7f0dd5016000-7f0dd5816000 rw-p 00000000 00:00 0
[stack:2285]
7f0dd5816000-7f0dd5817000 ---p 00000000 00:00 0
7f0dd5817000-7f0dd6017000 rw-p 00000000 00:00 0
[stack:2284]
7f0dd6017000-7f0dd6018000 ---p 00000000 00:00 0
7f0dd6018000-7f0dd6818000 rw-p 00000000 00:00 0
[stack:2283]
7f0dd6818000-7f0dd6819000 ---p 00000000 00:00 0
7f0dd6819000-7f0dd7019000 rw-p 00000000 00:00 0
[stack:2282]
7f0dd7019000-7f0dd701a000 ---p 00000000 00:00 0
7f0dd701a000-7f0dd781a000 rw-p 00000000 00:00 0
[stack:2281]
7f0dd781a000-7f0dd781b000 ---p 00000000 00:00 0
7f0dd781b000-7f0dd801b000 rw-p 00000000 00:00 0
[stack:2280]
7f0dd801b000-7f0dd801c000 ---p 00000000 00:00 0
7f0dd801c000-7f0dd881c000 rw-p 00000000 00:00 0
[stack:2279]
7f0dd881c000-7f0dd881d000 ---p 00000000 00:00 0
7f0dd881d000-7f0dd901d000 rw-p 00000000 00:00 0
[stack:2278]
7f0dd901d000-7f0dd901e000 ---p 00000000 00:00 0
7f0dd901e000-7f0dd981e000 rw-p 00000000 00:00 0
[stack:2277]
7f0dd981e000-7f0dd981f000 ---p 00000000 00:00 0
7f0dd981f000-7f0dda01f000 rw-p 00000000 00:00 0
[stack:2276]
7f0dda01f000-7f0dda020000 ---p 00000000 00:00 0
7f0dda020000-7f0dda820000 rw-p 00000000 00:00 0
[stack:2275]
7f0dda820000-7f0dda821000 ---p 00000000 00:00 0
7f0dda821000-7f0ddb021000 rw-p 00000000 00:00 0
[stack:2274]
7f0ddb021000-7f0ddb022000 ---p 00000000 00:00 0
7f0ddb022000-7f0ddb822000 rw-p 00000000 00:00 0
[stack:2273]
7f0ddb822000-7f0ddb823000 ---p 00000000 00:00 0
7f0ddb823000-7f0ddc023000 rw-p 00000000 00:00 0
[stack:2272]
7f0ddc023000-7f0ddc024000 ---p 00000000 00:00 0
7f0ddc024000-7f0ddc824000 rw-p 00000000 00:00 0
[stack:2270]
7f0ddc824000-7f0ddc825000 ---p 00000000 00:00 0
7f0ddc825000-7f0ddd025000 rw-p 00000000 00:00 0
[stack:2269]
7f0ddd025000-7f0ddd026000 ---p 00000000 00:00 0
7f0ddd026000-7f0ddd826000 rw-p 00000000 00:00 0
[stack:2268]
7f0ddd826000-7f0ddd827000 ---p 00000000 00:00 0
7f0ddd827000-7f0dde027000 rw-p 00000000 00:00 0
[stack:2267]
7f0dde027000-7f0dde028000 ---p 00000000 00:00 0
7f0dde028000-7f0dde828000 rw-p 00000000 00:00 0
[stack:2266]
7f0dde828000-7f0dde829000 ---p 00000000 00:00 0
7f0dde829000-7f0ddf08b000 rw-p 00000000 00:00 0
[stack:2265]
7f0ddf1ae000-7f0ddf3ea000 r--s 00000000 00:21 35413235
/home/lly/work/perceptron/lm/en.binlm3
7f0ddf3ea000-7f0ddf585000 r-xp 00000000 08:11 262155
/lib64/libc-2.15.so
7f0ddf585000-7f0ddf785000 ---p 0019b000 08:11 262155
/lib64/libc-2.15.so
7f0ddf785000-7f0ddf789000 r--p 0019b000 08:11 262155
/lib64/libc-2.15.so
7f0ddf789000-7f0ddf78b000 rw-p 0019f000 08:11 262155
/lib64/libc-2.15.so
7f0ddf78b000-7f0ddf78f000 rw-p 00000000 00:00 0
7f0ddf78f000-7f0ddf7a6000 r-xp 00000000 08:11 262181
/lib64/libpthread-2.15.so
7f0ddf7a6000-7f0ddf9a5000 ---p 00017000 08:11 262181
/lib64/libpthread-2.15.so
7f0ddf9a5000-7f0ddf9a6000 r--p 00016000 08:11 262181
/lib64/libpthread-2.15.so
7f0ddf9a6000-7f0ddf9a7000 rw-p 00017000 08:11 262181
/lib64/libpthread-2.15.so
7f0ddf9a7000-7f0ddf9ab000 rw-p 00000000 00:00 0
7f0ddf9ab000-7f0ddf9c0000 r-xp 00000000 08:11 262300
/lib64/libgcc_s.so.1
7f0ddf9c0000-7f0ddfbbf000 ---p 00015000 08:11 262300
/lib64/libgcc_s.so.1
7f0ddfbbf000-7f0ddfbc0000 r--p 00014000 08:11 262300
/lib64/libgcc_s.so.1
7f0ddfbc0000-7f0ddfbc1000 rw-p 00015000 08:11 262300
/lib64/libgcc_s.so.1
7f0ddfbc1000-7f0ddfcb6000 r-xp 00000000 08:11 262163
/lib64/libm-2.15.so
7f0ddfcb6000-7f0ddfeb6000 ---p 000f5000 08:11 262163
/lib64/libm-2.15.so
7f0ddfeb6000-7f0ddfeb7000 r--p 000f5000 08:11 262163
/lib64/libm-2.15.so
7f0ddfeb7000-7f0ddfeb8000 rw-p 000f6000 08:11 262163
/lib64/libm-2.15.so
7f0ddfeb8000-7f0ddffa0000 r-xp 00000000 08:11 1710646
/usr/lib64/libstdc++.so.6.0.17
7f0ddffa0000-7f0de01a0000 ---p 000e8000 08:11 1710646
/usr/lib64/libstdc++.so.6.0.17
7f0de01a0000-7f0de01a8000 r--p 000e8000 08:11 1710646
/usSegmentation fault
r/lib64/libstdc++.so.6.0.17
7f0de01a8000-7f0de01aa000 rw-p 000f0000 08:11 1710646
/usr/lib64/libstdc++.so.6.0.17
7f0de01aa000-7f0de01bf000 rw-p 00000000 00:00 0
7f0de01bf000-7f0de01c6000 r-xp 00000000 08:11 262185
/lib64/librt-2.15.so
7f0de01c6000-7f0de03c5000 ---p 00007000 08:11 262185
/lib64/librt-2.15.so
7f0de03c5000-7f0de03c6000 r--p 00006000 08:11 262185
/lib64/librt-2.15.so
7f0de03c6000-7f0de03c7000 rw-p 00007000 08:11 2621: 0000000000000000 R12:
0000000000000000
R13: 00007fff25dc8e90 R14: 0000000000000000 R15: 00007f0dd6817700
RSP: 00007f0dd68161a8

RIP: 00007f0ddff505e3 EFLAGS: 00010206

CS: 0033 FS: 0000 GS: 0000

Trap: 0000000e Error: 00000005 OldMask: 00000000 CR2: ffffffe8

FPUCW: 0000037f FPUSW: 00000000 TAG: 00000000
RIP: 00000000 RDP: 00000000

ST(0) 0000 0000000000000000 ST(1) 0000 0000000000000000
ST(2) 0000 0000000000000000 ST(3) 0000 0000000000000000
ST(4) 0000 0000000000000000 ST(5) 0000 0000000000000000
ST(6) 0000 0000000000000000 ST(7) 0000 0000000000000000
mxcsr: 1fa4
XMM0: 00000000000000000000000000000000 XMM1:
00000000000000000000000000000000
XMM2: 00000000000000000000000000000000 XMM3:
00000000000000000000000000000000
XMM4: 00000000000000000000000000000000 XMM5:
00000000000000000000000000000000
XMM6: 00000000000000000000000000000000 XMM7:
00000000000000000000000000000000
XMM8: 00000000000000000000000000000000 XMM9:
00000000000000000000000000000000
XMM10: 00000000000000000000000000000000 XMM11:
00000000000000000000000000000000
XMM12: 00000000000000000000000000000000 XMM13:
00000000000000000000000000000000
XMM14: 00000000000000000000000000000000 XMM15:
00000000000000000000000000000000

Backtrace:
185 /lib64/librt-2.15.so
7f0de03c7000-7f0de03dc000 r-xp 00000000 08:11 262194
/lib64/libz.so.1.2.7
7f0de03dc000-7f0de05db000 ---p 00015000 08:11 262194
/lib64/libz.so.1.2.7
7f0de05db000-7f0de05dc000 r--p 00014000 08:11 262194
/lib64/libz.so.1.2.7
7f0de05dc000-7f0de05dd000 rw-p 00015000 08:11 262194
/lib64/libz.so.1.2.7
7f0de05dd000-7f0de05e1000 r-xp 00000000 08:11 262152
/lib64/libSegFault.so
7f0de05e1000-7f0de07e0000 ---p 00004000 08:11 262152
/lib64/libSegFault.so
7f0de07e0000-7f0de07e1000 r--p 00003000 08:11 262152
/lib64/libSegFault.so
7f0de07e1000-7f0de07e2000 rw-p 00004000 08:11 262152
/lib64/libSegFault.so
7f0de07e2000-7f0de07e5000 r-xp 00000000 08:11 262161
/lib64/libdl-2.15.so
7f0de07e5000-7f0de09e4000 ---p 00003000 08:11 262161
/lib64/libdl-2.15.so
7f0de09e4000-7f0de09e5000 r--p 00002000 08:11 262161
/lib64/libdl-2.15.so
7f0de09e5000-7f0de09e6000 rw-p 00003000 08:11 262161
/lib64/libdl-2.15.so
7f0de09e6000-7f0de0a0b000 r-xp 00000000 08:11 1708628
/usr/lib64/liblzma.so.5.0.3
7f0de0a0b000-7f0de0c0a000 ---p 00025000 08:11 1708628
/usr/lib64/liblzma.so.5.0.3
7f0de0c0a000-7f0de0c0b000 r--p 00024000 08:11 1708628
/usr/lib64/liblzma.so.5.0.3
7f0de0c0b000-7f0de0c0c000 rw-p 00025000 08:11 1708628
/usr/lib64/liblzma.so.5.0.3
7f0de0c0c000-7f0de0c2d000 r-xp 00000000 08:11 280462
/lib64/ld-2.15.so
7f0de0d06000-7f0de0e0e000 rw-p 00000000 00:00 0
7f0de0e2c000-7f0de0e2d000 rw-p 00000000 00:00 0
7f0de0e2d000-7f0de0e2e000 r--p 00021000 08:11 280462
/lib64/ld-2.15.so
7f0de0e2e000-7f0de0e2f000 rw-p 00022000 08:11 280462
/lib64/ld-2.15.so
7f0de0e2f000-7f0de0e30000 rw-p 00000000 00:00 0
7fff25daa000-7fff25dcc000 rw-p 00000000 00:00 0
[stack]
7fff25dff000-7fff25e00000 r-xp 00000000 00:00 0
[vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
[vsyscall]
/usr/lib64/libstdc++.so.6(_ZStlsIcSt11char_traitsIcESaIcEERSt13basic_ostreamIT_T0_ES7_RKSbIS4_S5_T1_E+0x3)[0x7f0ddff505e3]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x58400d]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x43ffd6]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x44057e]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x4402fe]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x561d16]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x561a08]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x47baa2]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x530bdd]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534f41]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534ea4]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534e49]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x534e0e]
/home/lly/plateform/moses-3.0/bin/moses_chart[0x8b3892]
/lib64/libpthread.so.0(+0x7e0e)[0x7f0ddf796e0e]
sh: line 1: 2260 Segmentation fault
/home/lly/plateform/moses-3.0/bin/moses_chart -threads all -config
/home/lly/work/perceptron/tune/moses-sparse.ini -weight-overwrite
'PhrasePenalty0= -0.086708 WordPenalty0= -0.133063 TranslationModel0=
0.129426 0.015298 0.147046 -0.033093 TranslationModel1= 0.340195 LM0=
0.115169' -weight-file run2.sparse-weights -n-best-list run2.best100.out
100 distinct -input-file /home/lly/work/perceptron/tune/devset.train.zh >
run2.out


Regards
Liangyou


On Wed, Mar 11, 2015 at 8:58 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk>
wrote:

> Hi Liangyou
>
> This is a bit strange. The stacktrace suggests corruption in the feature
> vector, but it would be surprising that we haven't seen it before. Can you
> reproduce the problem when running under valgrind? The memory could be
> getting corrupted by a completely different source, which we'll really only
> find with a memory checker.
>
> cheers - Barry
>
>
> On 11/03/15 17:50, Liangyou Li wrote:
>
> Thanks Matthias.
>
> This problem doesn't always happen.
> Even on the same data, sometimes it can run without crash, sometimes it
> doesn't.
> But, by now, the error only happens when the decoder outputs features.
> (-n-best-list or -output-search-graph-hypergraph)
>
> Cheers
> Liangyou
>
>
> On Wed, Mar 11, 2015 at 5:38 PM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:
>
>> Hi,
>>
>> I've recently been using these sparse feature functions without any
>> issues in multi-threaded chart-based decoding. There might be a problem
>> with thread safety, but I currently can't tell why you got the
>> segmentation fault. You should investigate this in more detail.
>>
>> Cheers,
>> Matthias
>>
>>
>>
>> On Wed, 2015-03-11 at 12:12 +0000, Liangyou Li wrote:
>> > When I run moses with sparse features in multi-threads ( parameter
>> -threads all), I got segmentation fault.
>> >
>> >
>> > The three sparse features are:
>> > SourceWordDeletionFeature factor=0
>> > TargetWordInsertionFeature factor=0
>> > WordTranslationFeature input-factor=0 output-factor=0
>> >
>> >
>> > The exact command I used is:
>> > moses_chart -threads all -f moses.ini -i input -n-best-list nbest
>> 100 distinct
>> >
>> >
>> > This error happens kind of randomly?. But it only happens when sparse
>> features and multi-threads are used.
>> >
>> >
>> > I've tried several times to use gdb to trace the error. Fortunately, I
>> just get the back-trace info, as listed:
>> >
>> >
>> > Program received signal SIGSEGV, Segmentation fault.
>> > [Switching to Thread 0x7fffdae77700 (LWP 32390)]
>> > 0x00007ffff711f5e3 in std::basic_ostream<char, std::char_traits<char>
>> >& std::operator<< <char, std::char_traits<char>, std::allocator<char>
>> >(std::basic_ostream<char, std::char_traits<char> >&,
>> std::basic_string<char, std::char_traits<char>, std::allocator<char> >
>> const&) () from /usr/lib64/libstdc++.so.6
>> > Missing separate debuginfos, use: zypper install
>> glibc-debuginfo-2.15-22.17.1.x86_64
>> libgcc47-debuginfo-4.7.1_20120723-1.1.1.x86_64
>> liblzma5-debuginfo-5.0.3-12.2.2.x86_64
>> libstdc++47-debuginfo-4.7.1_20120723-1.1.1.x86_64
>> zlib-debuginfo-1.2.7-2.1.2.x86_64
>> > (gdb) backtrace
>> > #0 0x00007ffff711f5e3 in std::basic_ostream<char,
>> std::char_traits<char> >& std::operator<< <char, std::char_traits<char>,
>> std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&,
>> std::basic_string<char, std::char_traits<char>, std::allocator<char> >
>> const&) () from /usr/lib64/libstdc++.so.6
>> > #1 0x0000000000584711 in Moses::operator<< (out=..., name=...) at
>> moses/FeatureVector.cpp:141
>> > #2 0x0000000000440232 in
>> Moses::ScoreComponentCollection::GetVectorForProducer (this=0x7fffad0e9380,
>> sp=0xe2aab0) at moses/ScoreComponentCollection.cpp:293
>> > #3 0x00000000004407da in
>> Moses::ScoreComponentCollection::OutputFeatureScores (this=0x7fffad0e9380,
>> out=..., ff=0xe2aab0, lastName="LM0") at
>> moses/ScoreComponentCollection.cpp:351
>> > #4 0x000000000044055a in
>> Moses::ScoreComponentCollection::OutputAllFeatureScores
>> (this=0x7fffad0e9380, out=...) at moses/ScoreComponentCollection.cpp:319
>> > #5 0x0000000000561ce4 in Moses::ChartManager::OutputNBestList
>> (this=0x7fffacbe0010, collector=0x9320d210, nBestList=std::vector of length
>> 100, capacity 128 = {...}, translationId=569)
>> > at moses/ChartManager.cpp:381
>> > #6 0x00000000005619d6 in Moses::ChartManager::OutputNBest
>> (this=0x7fffacbe0010, collector=0x9320d210) at moses/ChartManager.cpp:335
>> > #7 0x0000000000467cd6 in Moses::TranslationTask::Run (this=0x214d90b0)
>> at moses/TranslationTask.cpp:111
>> > #8 0x0000000000530b69 in Moses::ThreadPool::Execute
>> (this=0x7fffffffd710) at moses/ThreadPool.cpp:61
>> > #9 0x0000000000534ecd in boost::_mfi::mf0<void,
>> Moses::ThreadPool>::operator() (this=0xf263088, p=0x7fffffffd710) at
>> /home/lly/plateform/boost_1_55_0/include/boost/bind/mem_fn_template.hpp:49
>> > #10 0x0000000000534e30 in
>> boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*>
>> >::operator()<boost::_mfi::mf0<void, Moses::ThreadPool>, boost::_bi::list0>
>> (this=0xf263098, f=..., a=...)
>> > at /home/lly/plateform/boost_1_55_0/include/boost/bind/bind.hpp:253
>> > #11 0x0000000000534dd5 in boost::_bi::bind_t<void,
>> boost::_mfi::mf0<void, Moses::ThreadPool>,
>> boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > >::operator()
>> (this=0xf263088)
>> > at
>> /home/lly/plateform/boost_1_55_0/include/boost/bind/bind_template.hpp:20
>> > #12 0x0000000000534d9a in
>> boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void,
>> Moses::ThreadPool>, boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*>
>> > > >::run (this=0xf262ed0)
>> > at
>> /home/lly/plateform/boost_1_55_0/include/boost/thread/detail/thread.hpp:117
>> > #13 0x00000000008b4752 in thread_proxy ()
>> > #14 0x00007ffff6965e0e in start_thread () from /lib64/libpthread.so.0
>> > #15 0x00007ffff669d2cd in clone () from /lib64/libc.so.6
>> >
>> >
>> > ?Has anyone ?had the problem before ? Any ideas on solving this ?
>> > Many Thanks ?
>> >
>> >
>> >
>> >
>> > PS: In my experiment, I found the function " void
>> CompletedRuleCollection::Add(const TargetPhraseCollection &tpc, const
>> StackVec &stackVec, const std::vector<float> &stackScores, const
>> ChartParserCallback &outColl) " does not consider " m_ruleLimit " . So
>> after adding parameter " -rule-limit 0 ", the decoder can only collect one
>> translation option.
>> >
>> >
>> >
>> >
>> > Cheers
>> > Liangyou
>> >
>> >
>> >
>> >
>> >
>> >
>> > --
>> >
>> >
>> > Liangyou Li
>> > CNGL
>> > School of Computing
>> > Dublin City University
>> >
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>
>
> --
>
> Liangyou Li
> CNGL
> School of Computing
> Dublin City University
>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>


--

Liangyou Li
CNGL
School of Computing
Dublin City University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150312/e663e728/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 101, Issue 42
**********************************************

0 Response to "Moses-support Digest, Vol 101, Issue 42"

Post a Comment

Subscribe to: Post Comments (Atom)