Moses-support Digest, Vol 127, Issue 28

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Segfault from moses while tuning (Per Starb?ck )
2. Re: Segfault from moses while tuning (Hieu Hoang)
3. Re: Segfault from moses while tuning (Per Starb?ck )


----------------------------------------------------------------------

Message: 1
Date: Wed, 17 May 2017 15:07:08 +0200
From: starback@stp.lingfil.uu.se (Per Starb?ck )
Subject: Re: [Moses-support] Segfault from moses while tuning
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <sa537c3ws1v.fsf@numerus.lingfil.uu.se>
Content-Type: text/plain

Hieu Hoang writes:

> ah yes, I think the phrase-table was created in the version when [ and ] weren't reserved
> characters but now they are. So you have to use the executables in that version (v3?), or escape
> them

OK, so why did this go wrong? (Remember that I don't really am a Moses
user myself, which may be why I am confused here.)

That file phrase-table.0-0.1.1.gz was created when I ran
scripts/training/mert-moses.pl. That script comes from the same
version 3 linux-64bit.tgz as the rest. Is there a bug in mert-moses.pl,
or could there be anything wrong in any of the inputs I had to
mert-moses.pl?

Also, if I just want to escape the brackets after the fact, what is the
escape syntax there?


------------------------------

Message: 2
Date: Wed, 17 May 2017 14:46:04 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Segfault from moses while tuning
To: Per Starb?ck <starback@stp.lingfil.uu.se>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbibXqcuF0UH7Lk_WGn1dK0WXfbV_+_Ab8LZjmzg_3hy-A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

I don#t know but I would really rule out it's not RAM issues by eyeballing
'top' while moses is running.

Then depending on what you eventually want to do with Moses:
1. If you only want to run these models that can be downloaded from the
website, then use the executables from the website. Binarize to models with
the executables from the website. It worked when it was released, it should
work now.
2. If you want to eventually run Moses with your own data, then get the
code from github, training and tuning everything yourself.


* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 17 May 2017 at 14:07, Per Starb?ck <starback@stp.lingfil.uu.se> wrote:

> Hieu Hoang writes:
>
> > ah yes, I think the phrase-table was created in the version when [ and ]
> weren't reserved
> > characters but now they are. So you have to use the executables in that
> version (v3?), or escape
> > them
>
> OK, so why did this go wrong? (Remember that I don't really am a Moses
> user myself, which may be why I am confused here.)
>
> That file phrase-table.0-0.1.1.gz was created when I ran
> scripts/training/mert-moses.pl. That script comes from the same
> version 3 linux-64bit.tgz as the rest. Is there a bug in mert-moses.pl,
> or could there be anything wrong in any of the inputs I had to
> mert-moses.pl?
>
> Also, if I just want to escape the brackets after the fact, what is the
> escape syntax there?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170517/ee06d229/attachment-0001.html

------------------------------

Message: 3
Date: Wed, 17 May 2017 16:48:31 +0200
From: starback@stp.lingfil.uu.se (Per Starb?ck )
Subject: Re: [Moses-support] Segfault from moses while tuning
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <sa5inkzv8sg.fsf@numerus.lingfil.uu.se>
Content-Type: text/plain

> I don#t know but I would really rule out it's not RAM issues by eyeballing 'top' while moses is
> running.

That is ruled out. You helped me in identifying the problem. I can
reproducibly make the same command segfault or not by just adding or
removing a single line in the phrase-table file, and you said that
that line had a syntax that wasn't allowed (anymore). That mystery is
solved. (I did run with a memory checker as well. The moses process
peaked at "Maximum resident set size (kbytes): 280360" on a computer
that right then had > 2 GB Free.)

Even *if* there was some other problem *as well*, it is a problem that
the phrase table file I got from scripts/training/mert-moses.pl has an
old syntax with [], isn't it?

You answered that you don't know when I asked if this is a bug in
mert-moses.pl. OK, but surely you or someone else can answer my other
question, about what the escape syntax is? I ventured a guess that it
should be "\[" and "\]". Is that right? By just including the line

Konzert ||| [would] ||| 0.2 1 0.2 0.0833333 ||| 0-0 ||| 5 5 1 ||| |||

in the phrase table I got a segfault as I wrote earlier. When having
"\[would\]" there instead of "[would]" there was no crash. Is that a
good change?


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 127, Issue 28
**********************************************

Tweet