Home » Uncategories » Moses-support Digest, Vol 101, Issue 36

Moses-support Digest, Vol 101, Issue 36

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses segmentation fault under multi-thread (Matthias Huck)
2. Re: Moses segmentation fault under multi-thread (Liangyou Li)
3. Re: Recasing on EMS (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Wed, 11 Mar 2015 17:38:56 +0000
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] Moses segmentation fault under
multi-thread
To: Liangyou Li <llysuda@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <1426095536.2192.666.camel@portedgar>
Content-Type: text/plain; charset="UTF-8"

Hi,

I've recently been using these sparse feature functions without any
issues in multi-threaded chart-based decoding. There might be a problem
with thread safety, but I currently can't tell why you got the
segmentation fault. You should investigate this in more detail.

Cheers,
Matthias



On Wed, 2015-03-11 at 12:12 +0000, Liangyou Li wrote:
> When I run moses with sparse features in multi-threads ( parameter -threads all), I got segmentation fault.
>
>
> The three sparse features are:
> SourceWordDeletionFeature factor=0
> TargetWordInsertionFeature factor=0
> WordTranslationFeature input-factor=0 output-factor=0
>
>
> The exact command I used is:
> moses_chart -threads all -f moses.ini -i input -n-best-list nbest 100 distinct
>
>
> This error happens kind of randomly?. But it only happens when sparse features and multi-threads are used.
>
>
> I've tried several times to use gdb to trace the error. Fortunately, I just get the back-trace info, as listed:
>
>
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7fffdae77700 (LWP 32390)]
> 0x00007ffff711f5e3 in std::basic_ostream<char, std::char_traits<char> >& std::operator<< <char, std::char_traits<char>, std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /usr/lib64/libstdc++.so.6
> Missing separate debuginfos, use: zypper install glibc-debuginfo-2.15-22.17.1.x86_64 libgcc47-debuginfo-4.7.1_20120723-1.1.1.x86_64 liblzma5-debuginfo-5.0.3-12.2.2.x86_64 libstdc++47-debuginfo-4.7.1_20120723-1.1.1.x86_64 zlib-debuginfo-1.2.7-2.1.2.x86_64
> (gdb) backtrace
> #0 0x00007ffff711f5e3 in std::basic_ostream<char, std::char_traits<char> >& std::operator<< <char, std::char_traits<char>, std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /usr/lib64/libstdc++.so.6
> #1 0x0000000000584711 in Moses::operator<< (out=..., name=...) at moses/FeatureVector.cpp:141
> #2 0x0000000000440232 in Moses::ScoreComponentCollection::GetVectorForProducer (this=0x7fffad0e9380, sp=0xe2aab0) at moses/ScoreComponentCollection.cpp:293
> #3 0x00000000004407da in Moses::ScoreComponentCollection::OutputFeatureScores (this=0x7fffad0e9380, out=..., ff=0xe2aab0, lastName="LM0") at moses/ScoreComponentCollection.cpp:351
> #4 0x000000000044055a in Moses::ScoreComponentCollection::OutputAllFeatureScores (this=0x7fffad0e9380, out=...) at moses/ScoreComponentCollection.cpp:319
> #5 0x0000000000561ce4 in Moses::ChartManager::OutputNBestList (this=0x7fffacbe0010, collector=0x9320d210, nBestList=std::vector of length 100, capacity 128 = {...}, translationId=569)
> at moses/ChartManager.cpp:381
> #6 0x00000000005619d6 in Moses::ChartManager::OutputNBest (this=0x7fffacbe0010, collector=0x9320d210) at moses/ChartManager.cpp:335
> #7 0x0000000000467cd6 in Moses::TranslationTask::Run (this=0x214d90b0) at moses/TranslationTask.cpp:111
> #8 0x0000000000530b69 in Moses::ThreadPool::Execute (this=0x7fffffffd710) at moses/ThreadPool.cpp:61
> #9 0x0000000000534ecd in boost::_mfi::mf0<void, Moses::ThreadPool>::operator() (this=0xf263088, p=0x7fffffffd710) at /home/lly/plateform/boost_1_55_0/include/boost/bind/mem_fn_template.hpp:49
> #10 0x0000000000534e30 in boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> >::operator()<boost::_mfi::mf0<void, Moses::ThreadPool>, boost::_bi::list0> (this=0xf263098, f=..., a=...)
> at /home/lly/plateform/boost_1_55_0/include/boost/bind/bind.hpp:253
> #11 0x0000000000534dd5 in boost::_bi::bind_t<void, boost::_mfi::mf0<void, Moses::ThreadPool>, boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > >::operator() (this=0xf263088)
> at /home/lly/plateform/boost_1_55_0/include/boost/bind/bind_template.hpp:20
> #12 0x0000000000534d9a in boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Moses::ThreadPool>, boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > > >::run (this=0xf262ed0)
> at /home/lly/plateform/boost_1_55_0/include/boost/thread/detail/thread.hpp:117
> #13 0x00000000008b4752 in thread_proxy ()
> #14 0x00007ffff6965e0e in start_thread () from /lib64/libpthread.so.0
> #15 0x00007ffff669d2cd in clone () from /lib64/libc.so.6
>
>
> ?Has anyone ?had the problem before ? Any ideas on solving this ?
> Many Thanks ?
>
>
>
>
> PS: In my experiment, I found the function " void CompletedRuleCollection::Add(const TargetPhraseCollection &tpc, const StackVec &stackVec, const std::vector<float> &stackScores, const ChartParserCallback &outColl) " does not consider " m_ruleLimit " . So after adding parameter " -rule-limit 0 ", the decoder can only collect one translation option.
>
>
>
>
> Cheers
> Liangyou
>
>
>
>
>
>
> --
>
>
> Liangyou Li
> CNGL
> School of Computing
> Dublin City University
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 2
Date: Wed, 11 Mar 2015 17:50:53 +0000
From: Liangyou Li <llysuda@gmail.com>
Subject: Re: [Moses-support] Moses segmentation fault under
multi-thread
To: Matthias Huck <mhuck@inf.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CAHeunNXEdQkq=E5ay5twcAmq5NMG-Au06wCTP-x5VBHSDZnS2w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks Matthias.

This problem doesn't always happen.
Even on the same data, sometimes it can run without crash, sometimes it
doesn't.
But, by now, the error only happens when the decoder outputs features.
(-n-best-list or -output-search-graph-hypergraph)

Cheers
Liangyou


On Wed, Mar 11, 2015 at 5:38 PM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:

> Hi,
>
> I've recently been using these sparse feature functions without any
> issues in multi-threaded chart-based decoding. There might be a problem
> with thread safety, but I currently can't tell why you got the
> segmentation fault. You should investigate this in more detail.
>
> Cheers,
> Matthias
>
>
>
> On Wed, 2015-03-11 at 12:12 +0000, Liangyou Li wrote:
> > When I run moses with sparse features in multi-threads ( parameter
> -threads all), I got segmentation fault.
> >
> >
> > The three sparse features are:
> > SourceWordDeletionFeature factor=0
> > TargetWordInsertionFeature factor=0
> > WordTranslationFeature input-factor=0 output-factor=0
> >
> >
> > The exact command I used is:
> > moses_chart -threads all -f moses.ini -i input -n-best-list nbest
> 100 distinct
> >
> >
> > This error happens kind of randomly?. But it only happens when sparse
> features and multi-threads are used.
> >
> >
> > I've tried several times to use gdb to trace the error. Fortunately, I
> just get the back-trace info, as listed:
> >
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > [Switching to Thread 0x7fffdae77700 (LWP 32390)]
> > 0x00007ffff711f5e3 in std::basic_ostream<char, std::char_traits<char> >&
> std::operator<< <char, std::char_traits<char>, std::allocator<char>
> >(std::basic_ostream<char, std::char_traits<char> >&,
> std::basic_string<char, std::char_traits<char>, std::allocator<char> >
> const&) () from /usr/lib64/libstdc++.so.6
> > Missing separate debuginfos, use: zypper install
> glibc-debuginfo-2.15-22.17.1.x86_64
> libgcc47-debuginfo-4.7.1_20120723-1.1.1.x86_64
> liblzma5-debuginfo-5.0.3-12.2.2.x86_64
> libstdc++47-debuginfo-4.7.1_20120723-1.1.1.x86_64
> zlib-debuginfo-1.2.7-2.1.2.x86_64
> > (gdb) backtrace
> > #0 0x00007ffff711f5e3 in std::basic_ostream<char,
> std::char_traits<char> >& std::operator<< <char, std::char_traits<char>,
> std::allocator<char> >(std::basic_ostream<char, std::char_traits<char> >&,
> std::basic_string<char, std::char_traits<char>, std::allocator<char> >
> const&) () from /usr/lib64/libstdc++.so.6
> > #1 0x0000000000584711 in Moses::operator<< (out=..., name=...) at
> moses/FeatureVector.cpp:141
> > #2 0x0000000000440232 in
> Moses::ScoreComponentCollection::GetVectorForProducer (this=0x7fffad0e9380,
> sp=0xe2aab0) at moses/ScoreComponentCollection.cpp:293
> > #3 0x00000000004407da in
> Moses::ScoreComponentCollection::OutputFeatureScores (this=0x7fffad0e9380,
> out=..., ff=0xe2aab0, lastName="LM0") at
> moses/ScoreComponentCollection.cpp:351
> > #4 0x000000000044055a in
> Moses::ScoreComponentCollection::OutputAllFeatureScores
> (this=0x7fffad0e9380, out=...) at moses/ScoreComponentCollection.cpp:319
> > #5 0x0000000000561ce4 in Moses::ChartManager::OutputNBestList
> (this=0x7fffacbe0010, collector=0x9320d210, nBestList=std::vector of length
> 100, capacity 128 = {...}, translationId=569)
> > at moses/ChartManager.cpp:381
> > #6 0x00000000005619d6 in Moses::ChartManager::OutputNBest
> (this=0x7fffacbe0010, collector=0x9320d210) at moses/ChartManager.cpp:335
> > #7 0x0000000000467cd6 in Moses::TranslationTask::Run (this=0x214d90b0)
> at moses/TranslationTask.cpp:111
> > #8 0x0000000000530b69 in Moses::ThreadPool::Execute
> (this=0x7fffffffd710) at moses/ThreadPool.cpp:61
> > #9 0x0000000000534ecd in boost::_mfi::mf0<void,
> Moses::ThreadPool>::operator() (this=0xf263088, p=0x7fffffffd710) at
> /home/lly/plateform/boost_1_55_0/include/boost/bind/mem_fn_template.hpp:49
> > #10 0x0000000000534e30 in
> boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*>
> >::operator()<boost::_mfi::mf0<void, Moses::ThreadPool>, boost::_bi::list0>
> (this=0xf263098, f=..., a=...)
> > at /home/lly/plateform/boost_1_55_0/include/boost/bind/bind.hpp:253
> > #11 0x0000000000534dd5 in boost::_bi::bind_t<void,
> boost::_mfi::mf0<void, Moses::ThreadPool>,
> boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*> > >::operator()
> (this=0xf263088)
> > at
> /home/lly/plateform/boost_1_55_0/include/boost/bind/bind_template.hpp:20
> > #12 0x0000000000534d9a in
> boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void,
> Moses::ThreadPool>, boost::_bi::list1<boost::_bi::value<Moses::ThreadPool*>
> > > >::run (this=0xf262ed0)
> > at
> /home/lly/plateform/boost_1_55_0/include/boost/thread/detail/thread.hpp:117
> > #13 0x00000000008b4752 in thread_proxy ()
> > #14 0x00007ffff6965e0e in start_thread () from /lib64/libpthread.so.0
> > #15 0x00007ffff669d2cd in clone () from /lib64/libc.so.6
> >
> >
> > ?Has anyone ?had the problem before ? Any ideas on solving this ?
> > Many Thanks ?
> >
> >
> >
> >
> > PS: In my experiment, I found the function " void
> CompletedRuleCollection::Add(const TargetPhraseCollection &tpc, const
> StackVec &stackVec, const std::vector<float> &stackScores, const
> ChartParserCallback &outColl) " does not consider " m_ruleLimit " . So
> after adding parameter " -rule-limit 0 ", the decoder can only collect one
> translation option.
> >
> >
> >
> >
> > Cheers
> > Liangyou
> >
> >
> >
> >
> >
> >
> > --
> >
> >
> > Liangyou Li
> > CNGL
> > School of Computing
> > Dublin City University
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>


--

Liangyou Li
CNGL
School of Computing
Dublin City University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150311/d9300b86/attachment-0001.htm

------------------------------

Message: 3
Date: Wed, 11 Mar 2015 13:57:14 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Recasing on EMS
To: burak ayd?n <baydinx@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDuV3zZx2B2fFoyHR0bPq844CZ2qxZkMXZ7FqCrJcHVPQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

I updated the default config.toy and wrote a short set of instructions here:
http://www.statmt.org/moses/?n=Moses.SupportTools#ntoc10

-phi

On Wed, Mar 11, 2015 at 8:59 AM, burak ayd?n <baydinx@gmail.com> wrote:
> Hi,
>
> What should be changed in 'config.toy' to open recasing instead of
> truecasing while using EMS?
>
> Regards,
> Burak
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 101, Issue 36
**********************************************

0 Response to "Moses-support Digest, Vol 101, Issue 36"

Post a Comment

Subscribe to: Post Comments (Atom)