Home » Uncategories » Moses-support Digest, Vol 82, Issue 39

Moses-support Digest, Vol 82, Issue 39

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Segmentation fault on generateSequences while running
OSM-Train.sh (Nadir Durrani)


----------------------------------------------------------------------

Message: 1
Date: Tue, 27 Aug 2013 13:50:40 +0100
From: Nadir Durrani <nadir.durrani@nu.edu.pk>
Subject: [Moses-support] Segmentation fault on generateSequences while
running OSM-Train.sh
To: moses-support@mit.edu, jianzhang09@gmail.com
Message-ID:
<CAFDj2Q1NeXL1u1xYQf_HdjsOtv1R+TsOHbM8vc78VFXzJtYzvw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi Jian,

Try out

${MOSES_DIR}/scripts/OSM/OSM-Train.sh ${TARGET} ${SOURCE} ${ALIGN} ${ORDER}
${WORKDIR} ${MOSES_DIR} ${SRILM_BINDIR}

the script takes target before source. Btw we made some changes. There's a
new script OSM-Train.perl. So use that now.

I have also uploaded the source code for generating the operation
sequences. That should help you debug the problem.

Cheers,
Nadir




On Mon, Aug 26, 2013 at 9:43 PM, <moses-support-request@mit.edu> wrote:

> Send Moses-support mailing list submissions to
> moses-support@mit.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/moses-support
> or, via email, send a message with subject or body 'help' to
> moses-support-request@mit.edu
>
> You can reach the person managing the list at
> moses-support-owner@mit.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Moses-support digest..."
>
>
> Today's Topics:
>
> 1. Segmentation fault on generateSequences while running
> OSM-Train.sh (jian zhang)
> 2. bjam interaction of --with-boost and existing old boost
> installation (Lane Schwartz)
> 3. Re: bjam interaction of --with-boost and existing old boost
> installation (Lane Schwartz)
> 4. Re: Segmentation fault on generateSequences while running
> OSM-Train.sh (Philipp Koehn)
> 5. Re: Segmentation fault on generateSequences while running
> OSM-Train.sh (Kenneth Heafield)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 26 Aug 2013 19:26:59 +0100
> From: jian zhang <jianzhang09@gmail.com>
> Subject: [Moses-support] Segmentation fault on generateSequences while
> running OSM-Train.sh
> To: moses-support@mit.edu
> Message-ID:
> <CALA=z0CykwbsX08ZXrZ-L+XDoVCg4ZE4xQ88mOcJGvcxq=
> k4ag@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
> I got the Segmentation fault exception while running script OSM-Train.sh.
>
> *The output from OSM-Train.sh:*
>
> Training OSM - Start
> Mon Aug 26 19:14:47 IST 2013
> Extracting Singletons
> 3000Converting Bilingual Sentence Pair into Operation Corpus
> */home/user/moses-experiment/mosesdecoder-master/scripts/OSM/OSM-Train.sh:
> line 18: 25670 Segmentation fault $6/scripts/OSM/generateSequences
> $5/e $5/f $5/align $5/Singletons > $5/opCorpus*
> Learning Operation Sequence Translation Model
> one of required modified KneserNey count-of-counts is zero
> error in discount estimator for order 1
> Binarizing
> util/file.cc:63 in int util::OpenReadOrThrow(const char*) threw
> ErrnoException because `-1 == (ret = open(name, 00))'.
> No such file or directory while opening
> /home/user/osm-test/output/operationLM
> ERROR
> Training OSM - End
>
> The command I am using is:
>
> ${MOSES_DIR}/scripts/OSM/OSM-Train.sh ${SOURCE} ${TARGET} ${ALIGN} ${ORDER}
> ${WORKDIR} ${MOSES_DIR} ${SRILM_BINDIR}
>
> All variables are using absolute path.
>
> I was trying to look at the source code of generateSequences to debug, but
> seems like it is not in the moses distribution. So, what can cause this
> error?
>
>
> Thanks,
>
> Jian Zhang
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/fb8a9ee1/attachment-0001.htm
>
> ------------------------------
>
> Message: 2
> Date: Mon, 26 Aug 2013 15:09:37 -0400
> From: Lane Schwartz <dowobeha@gmail.com>
> Subject: [Moses-support] bjam interaction of --with-boost and existing
> old boost installation
> To: "moses-support@mit.edu" <moses-support@mit.edu>
> Message-ID:
> <CABv3vZnO+6GvAWyG78z3RK0bhPoyKVr=
> WqGB85BQOG3cDFQjxA@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Kenneth,
>
> I'm (finally) trying to install a modern version of boost (version 1.53) in
> a custom location. My Scientific Linux system has boost 1.41 installed.
>
> When I run Moses ./bjam --with-boost=/my/custom/boost, the moses compile
> finds the old system boost headers in /usr/include, and then gets confused
> when those headers don't match the boost 1.53 libraries it finds in
> /my/custom/boost.
>
> I'd like to modify the bjam files so that if --with-boost is provided, it
> will add a -I flag to g++ so that the boost headers in the provided path
> get searched before the system include paths. Unfortunately, I can't seem
> to figure out how to tell bjam to behave this way. Would you be able to
> help me figure out how to do this?
>
> Thanks,
> Lane
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/cc676215/attachment-0001.htm
>
> ------------------------------
>
> Message: 3
> Date: Mon, 26 Aug 2013 15:27:00 -0400
> From: Lane Schwartz <dowobeha@gmail.com>
> Subject: Re: [Moses-support] bjam interaction of --with-boost and
> existing old boost installation
> To: "moses-support@mit.edu" <moses-support@mit.edu>
> Message-ID:
> <
> CABv3vZkXTjto9W6Aat2VbsGSCZtJZPkaeuq3o-FtrmT4pfu+TA@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Well, turns out the correct answer is to RTFM.
>
> I reinstalled boost, using the instructions in the Moses
> BUILD-INSTRUCTIONS.txt. I'm not sure which change made the difference, but
> installing boost according to the instructions in there, coupled with
> adding the custom boost lib dir to LD_LIBRARY_PATH and LIBRARY_PATH appears
> to have made the difference.
>
> I have now successfully compiled Moses using a custom boost install.
>
> Thanks,
> Lane
>
>
> On Mon, Aug 26, 2013 at 3:09 PM, Lane Schwartz <dowobeha@gmail.com> wrote:
>
> > Kenneth,
> >
> > I'm (finally) trying to install a modern version of boost (version 1.53)
> > in a custom location. My Scientific Linux system has boost 1.41
> installed.
> >
> > When I run Moses ./bjam --with-boost=/my/custom/boost, the moses compile
> > finds the old system boost headers in /usr/include, and then gets
> confused
> > when those headers don't match the boost 1.53 libraries it finds in
> > /my/custom/boost.
> >
> > I'd like to modify the bjam files so that if --with-boost is provided, it
> > will add a -I flag to g++ so that the boost headers in the provided path
> > get searched before the system include paths. Unfortunately, I can't seem
> > to figure out how to tell bjam to behave this way. Would you be able to
> > help me figure out how to do this?
> >
> > Thanks,
> > Lane
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/f85c1970/attachment-0001.htm
>
> ------------------------------
>
> Message: 4
> Date: Mon, 26 Aug 2013 21:03:50 +0100
> From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
> Subject: Re: [Moses-support] Segmentation fault on generateSequences
> while running OSM-Train.sh
> To: jian zhang <jianzhang09@gmail.com>
> Cc: "moses-support@mit.edu" <moses-support@mit.edu>, Nadir Durrani
> <dnadir@inf.ed.ac.uk>
> Message-ID:
> <
> CAAFADDANr+CLk198fO+K8gV4-GV20JCOY94ioWo9c0fT_B+gmw@mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi,
>
> my guess would be this happening due to lack of RAM.
> Can you monitor memory usage while training runs?
>
> -phi
>
>
> On Mon, Aug 26, 2013 at 7:26 PM, jian zhang <jianzhang09@gmail.com> wrote:
>
> > Hi,
> >
> > I got the Segmentation fault exception while running script OSM-Train.sh.
> >
> > *The output from OSM-Train.sh:*
> >
> > Training OSM - Start
> > Mon Aug 26 19:14:47 IST 2013
> > Extracting Singletons
> > 3000Converting Bilingual Sentence Pair into Operation Corpus
> >
> */home/user/moses-experiment/mosesdecoder-master/scripts/OSM/OSM-Train.sh:
> > line 18: 25670 Segmentation fault $6/scripts/OSM/generateSequences
> > $5/e $5/f $5/align $5/Singletons > $5/opCorpus*
> > Learning Operation Sequence Translation Model
> > one of required modified KneserNey count-of-counts is zero
> > error in discount estimator for order 1
> > Binarizing
> > util/file.cc:63 in int util::OpenReadOrThrow(const char*) threw
> > ErrnoException because `-1 == (ret = open(name, 00))'.
> > No such file or directory while opening
> > /home/user/osm-test/output/operationLM
> > ERROR
> > Training OSM - End
> >
> > The command I am using is:
> >
> > ${MOSES_DIR}/scripts/OSM/OSM-Train.sh ${SOURCE} ${TARGET} ${ALIGN}
> > ${ORDER} ${WORKDIR} ${MOSES_DIR} ${SRILM_BINDIR}
> >
> > All variables are using absolute path.
> >
> > I was trying to look at the source code of generateSequences to debug,
> but
> > seems like it is not in the moses distribution. So, what can cause this
> > error?
> >
> >
> > Thanks,
> >
> > Jian Zhang
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/f81fdbb4/attachment-0001.htm
>
> ------------------------------
>
> Message: 5
> Date: Mon, 26 Aug 2013 21:43:37 +0100
> From: Kenneth Heafield <moses@kheafield.com>
> Subject: Re: [Moses-support] Segmentation fault on generateSequences
> while running OSM-Train.sh
> To: moses-support@mit.edu
> Message-ID: <521BBDF9.30501@kheafield.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> Didn't fix the error (it's not mine anyway), but I've added set -e to
> the script so that you'll see the real error message, a segfault in
> generateSequences, instead of compound errors from subsequent commands.
>
> On 08/26/13 21:03, Philipp Koehn wrote:
> > Hi,
> >
> > my guess would be this happening due to lack of RAM.
> > Can you monitor memory usage while training runs?
> >
> > -phi
> >
> >
> > On Mon, Aug 26, 2013 at 7:26 PM, jian zhang <jianzhang09@gmail.com
> > <mailto:jianzhang09@gmail.com>> wrote:
> >
> > Hi,
> >
> > I got the Segmentation fault exception while running script
> > OSM-Train.sh.
> >
> > *The output from OSM-Train.sh:*
> >
> > Training OSM - Start
> > Mon Aug 26 19:14:47 IST 2013
> > Extracting Singletons
> > 3000Converting Bilingual Sentence Pair into Operation Corpus
> >
> */home/user/moses-experiment/mosesdecoder-master/scripts/OSM/OSM-Train.sh:
> > line 18: 25670 Segmentation fault
> > $6/scripts/OSM/generateSequences $5/e $5/f $5/align $5/Singletons >
> > $5/opCorpus*
> > Learning Operation Sequence Translation Model
> > one of required modified KneserNey count-of-counts is zero
> > error in discount estimator for order 1
> > Binarizing
> > util/file.cc:63 in int util::OpenReadOrThrow(const char*) threw
> > ErrnoException because `-1 == (ret = open(name, 00))'.
> > No such file or directory while opening
> > /home/user/osm-test/output/operationLM
> > ERROR
> > Training OSM - End
> >
> > The command I am using is:
> >
> > ${MOSES_DIR}/scripts/OSM/OSM-Train.sh ${SOURCE} ${TARGET} ${ALIGN}
> > ${ORDER} ${WORKDIR} ${MOSES_DIR} ${SRILM_BINDIR}
> >
> > All variables are using absolute path.
> >
> > I was trying to look at the source code of generateSequences to
> > debug, but seems like it is not in the moses distribution. So, what
> > can cause this error?
> >
> >
> > Thanks,
> >
> > Jian Zhang
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
>
>
> ------------------------------
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> End of Moses-support Digest, Vol 82, Issue 35
> *********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130827/92e4ee6b/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 82, Issue 39
*********************************************

0 Response to "Moses-support Digest, Vol 82, Issue 39"

Post a Comment

Subscribe to: Post Comments (Atom)