Moses-support Digest, Vol 97, Issue 91

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Problem at a training stage - linguistic model not
correctly trained ( Jo?o Luis A. d. C. Rosas)
2. Moses for Mere Mortals: Update (Jo?o Luis Rosas)
3. Re: moses-chart binary missing? (Eric Baucom)
4. Re: Format of binarized phrase tables (Marcin Junczys-Dowmunt)
5. Re: Format of binarized phrase tables (Raj Dabre)


----------------------------------------------------------------------

Message: 1
Date: Wed, 26 Nov 2014 21:59:08 +0000 (UTC)
From: " Jo?o Luis A. d. C. Rosas" <joao.rosas@skynet.be>
Subject: Re: [Moses-support] Problem at a training stage - linguistic
model not correctly trained
To: moses-support@mit.edu
Message-ID: <loom.20141126T192607-429@post.gmane.org>
Content-Type: text/plain; charset=utf-8


Radian Yazynin <info@...> writes:

>
>
> Dear Support staff members,
> I will appreciate if you help me with one issue.
> These are my first steps...
> I am trying out a Moses-For-Mere-Mortals package (Moses + IRSTLM + RandLM
+ ?MGIZA)
> in Ubuntu 14.04.1 64bit.
> Compilation resulted in one error only: MGIZA didn't install correctly.
> After that I downloaded MGIZA as suggested at http://www.statmt.org/moses/
and installed it separately
> into the folder where the erroneous MGIZA existed (in order to run the
scripts correctly).
> The preparatory step with test files in PT and EN (as included) was OK.
> Now that I try to use the test corpora with ./train-1.11 the training
procedure starts alright but
> after about 10 minutes terminates with the following:
> compile-lm: lmtable.cpp:237: int parseline(std: :istream&, int,
> ngram&, float&, float&): Assertion 'howmany == (Order+ 1) || ho
> wmany == (Order + )'' failed.
> Aborted (core dumped)
> *************** Writing training summary
> Linguistic model not correctly trained. Exiting...
> Thanks so much for you help!Kind regardsRadian Yazynin
> Tula, Russia
>
>

Hi Radian,

Thanks for having pointed this out. Here is what you were looking for :-) :
https://github.com/jladcr/Moses-for-Mere-Mortals .

This is in a new URL, supports Ubuntu 14.04 and has some improvements
(described in the documentation).

Greetings,

Jo?o





------------------------------

Message: 2
Date: Wed, 26 Nov 2014 21:59:46 +0000 (UTC)
From: Jo?o Luis Rosas <joao.rosas@skynet.be>
Subject: [Moses-support] Moses for Mere Mortals: Update
To: moses-support@mit.edu
Message-ID: <loom.20141126T192122-598@post.gmane.org>
Content-Type: text/plain; charset=utf-8

Hi,

There was recently a message about problems compiling Moses for Mere Mortals
in Ubuntu 14.04
(http://thread.gmane.org/gmane.comp.nlp.moses.user/11390/focus=11406).

We therefore have decided to review the whole package, added some slight
changes to it (in the end, a good deal of them), made it a bit more robust
and tested it in both Ubuntu 14.04 and Ubuntu 12.04. Thanks to Radian
Yazynin for having pointed us the problem. We will contact him separately
through e-mail.

You can find the result in https://github.com/jladcr/Moses-for-Mere-Mortals .

Greetings,

-- Jo?o Rosas



------------------------------

Message: 3
Date: Wed, 26 Nov 2014 17:22:06 -0500
From: Eric Baucom <eabaucom@umail.iu.edu>
Subject: Re: [Moses-support] moses-chart binary missing?
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CALtA6LvTpRimecb5su4a9KMtuF8EYu+QxMqWKM6xpvZYEHsKrw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

that works, thanks!

On Wed, Nov 26, 2014 at 4:20 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:

> There should be a softlink which points the file moses. Moses and
> Moses_chart are being merged. Everything in the tutorial should still work
> On 26 Nov 2014 21:16, "Eric Baucom" <eabaucom@umail.iu.edu> wrote:
>
>> I am interested in experimenting with tree-to-tree translations, so I
>> recently installed Moses according to the guidelines here:
>> http://www.statmt.org/moses/?n=Development.GetStarted .
>>
>> The installation completed successfully, and I am able to successfully
>> translate using the sample models as described in the same web page, with
>> the regular "moses" binary. However, my installation is missing the
>> "moses-chart" binary, which I believe is necessary to do any tree-to-tree
>> translation. Is this an addition step of the installation? I didn't see
>> any options about it in the documentation.
>>
>> Thanks,
>> Eric Baucom
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141126/0b48bd84/attachment-0001.htm

------------------------------

Message: 4
Date: Thu, 27 Nov 2014 11:43:35 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Format of binarized phrase tables
To: Raj Dabre <prajdabre@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <905d7115751fa193563d65467f96833a@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"



Hi,

I tried to port it to the newest Moses version, but I still get
segfaults. This used to work with Moses 1.* before the feature function
format was changed. However, since the binary format of the compact
phrase table has not changed since then, just use the old interface.

Checkout the "wipo" branch (not wipoNew!) and compile with

./bjam --with-cmph=/usr/include -j8
--with-java=/usr/lib/jvm/java-7-openjdk-amd64

You see you need to specifiy the location of your Java includes,
specifically the directories jni.h resides in. This should build the
stuff in misc/jni/* including java classes and stuff. You need ant for
that. Then you can run the query tool like this:

echo "test" | ./bin/JniQueryPt_example.sh
/some_path/phrase-table.0-0.minphr 4

You need to specify the path and the number of scores in the phrase
table. Look inside JniQueryPt_example.sh how to call the jar and inside
misc/jni/java/example.java how to call the code directly from Java. I
apologize for the code, I am not a Java programmer, so this may be
crude.

Best,

Marcin

W dniu 2014-11-26 12:00, Raj Dabre napisa?(a):

> Hello Marcin,
>
> Yes please. It would save me lots of time. Thanks.
>
> Regards.
>
> On Wed, Nov 26, 2014 at 6:50 PM, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:
>
> Hi,
>
> I have a JNI interface to my compact phrase table somewhere, I guess I can put that in contrib within a day or two if there is interest.
>
> best,
>
> Marcin
>
> W dniu 2014-11-26 10:45, Barry Haddow napisa?(a):
>
> Hi Raj
>
> The format of these tables is not described anywhere. You'd have to read
> the code in moses/TranslationModel/PhraseDictionaryTree.cpp, and then
> try to convert it it Java.
>
> A better plan would be to use JNI to call the C++ code -- a similar
> approach has been followed in the python interface in contrib/python.
> This would insulate you from the low-level details, and from changes in
> the format,
>
> cheers - Barry
>
> On 26/11/14 03:22, Raj Dabre wrote:
> Hello All, I know that Moses allows for binarization of a phrase table which can be read on demand at decoding time. We get 5 files named: phrase-table.binphr.* I want to write my own routine in Java to read phrase pairs from these on demand. Can anyone guide me ? PS: If an explanation of the same for binary reordering tables can be done then it would be great too. Thanks in advance. -- Raj Dabre. Research Student, Graduate School of Informatics, Kyoto University. CSE MTech, IITB., 2011-2014 _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

--

Raj Dabre.
Research Student, Graduate School of Informatics,
Kyoto University.

CSE MTech, IITB., 2011-2014



Links:
------
[1] http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141127/25288f12/attachment-0001.htm

------------------------------

Message: 5
Date: Thu, 27 Nov 2014 19:57:59 +0900
From: Raj Dabre <prajdabre@gmail.com>
Subject: Re: [Moses-support] Format of binarized phrase tables
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAB3gfjBFd0PBT0_-EH6dhUyYDnP6vaNZYrYT21QgBDfRscXgdQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello Marcin,

You just saved me a lot of time since I was planning to write this code
from scratch. Many thanks for that.
I will try to fix the reasons for the segfaults.

Many thanks again!

Regards.

On Thu, Nov 27, 2014 at 7:43 PM, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
wrote:

> Hi,
>
> I tried to port it to the newest Moses version, but I still get segfaults.
> This used to work with Moses 1.* before the feature function format was
> changed. However, since the binary format of the compact phrase table has
> not changed since then, just use the old interface.
>
> Checkout the "wipo" branch (not wipoNew!) and compile with
>
> ./bjam --with-cmph=/usr/include -j8
> --with-java=/usr/lib/jvm/java-7-openjdk-amd64
>
> You see you need to specifiy the location of your Java includes,
> specifically the directories jni.h resides in. This should build the stuff
> in misc/jni/* including java classes and stuff. You need ant for that. Then
> you can run the query tool like this:
>
> echo "test" | ./bin/JniQueryPt_example.sh
> /some_path/phrase-table.0-0.minphr 4
>
> You need to specify the path and the number of scores in the phrase table.
> Look inside JniQueryPt_example.sh how to call the jar and inside
> misc/jni/java/example.java how to call the code directly from Java. I
> apologize for the code, I am not a Java programmer, so this may be crude.
>
> Best,
>
> Marcin
>
> W dniu 2014-11-26 12:00, Raj Dabre napisa?(a):
>
> Hello Marcin,
>
> Yes please.
> It would save me lots of time.
> Thanks.
>
> Regards.
>
> On Wed, Nov 26, 2014 at 6:50 PM, Marcin Junczys-Dowmunt <
> junczys@amu.edu.pl> wrote:
>
>> Hi,
>>
>> I have a JNI interface to my compact phrase table somewhere, I guess I
>> can put that in contrib within a day or two if there is interest.
>>
>> best,
>>
>> Marcin
>>
>> W dniu 2014-11-26 10:45, Barry Haddow napisa?(a):
>>
>> Hi Raj
>>
>> The format of these tables is not described anywhere. You'd have to read
>> the code in moses/TranslationModel/PhraseDictionaryTree.cpp, and then
>> try to convert it it Java.
>>
>> A better plan would be to use JNI to call the C++ code -- a similar
>> approach has been followed in the python interface in contrib/python.
>> This would insulate you from the low-level details, and from changes in
>> the format,
>>
>> cheers - Barry
>>
>> On 26/11/14 03:22, Raj Dabre wrote:
>>
>> Hello All, I know that Moses allows for binarization of a phrase table
>> which can be read on demand at decoding time. We get 5 files named:
>> phrase-table.binphr.* I want to write my own routine in Java to read phrase
>> pairs from these on demand. Can anyone guide me ? PS: If an explanation of
>> the same for binary reordering tables can be done then it would be great
>> too. Thanks in advance. -- Raj Dabre. Research Student, Graduate School
>> of Informatics, Kyoto University. CSE MTech, IITB., 2011-2014
>> _______________________________________________ Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> Raj Dabre.
> Research Student,
> Graduate School of Informatics,
> Kyoto University.
> CSE MTech, IITB., 2011-2014
>
>
>
>



--
Raj Dabre.
Research Student,
Graduate School of Informatics,
Kyoto University.
CSE MTech, IITB., 2011-2014
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141127/fbde647b/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 97, Issue 91
*********************************************

Tweet