Moses-support Digest, Vol 92, Issue 54

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Python InterfaceSupport (Barry Haddow)
2. Re: Python InterfaceSupport (Judah Schvimer)


----------------------------------------------------------------------

Message: 1
Date: Mon, 30 Jun 2014 16:49:36 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Python InterfaceSupport
To: Judah Schvimer <judah.schvimer@mongodb.com>
Cc: moses-support@mit.edu
Message-ID: <53B18710.7050400@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=UTF-8; format=flowed

On 30/06/14 16:37, Judah Schvimer wrote:
> Hi,
>
> Thanks for the quick response! I have tried just link=shared and it
> does work. It's the combination of the two that doesn't work. It seems
> like IRSTLM is what needs to be compiled with -fPIC but I can't get
> that to work.
>
> And what do you mean by query the binarised phrase table? How is that
> different from decoding? I may be misunderstanding what the python
> interface was created for.

I just mean that you can use the python interface to find all possible
translations in the phrase table for a given segment. For me this was
useful because I wanted to implement a method of training the phrase
scores in python.

It can be useful for the Python interface to do more than just run
decoding, see here for example
https://ufal.mff.cuni.cz/pbml/98/art-chahuneau-smith-dyer.pdf


>
> Thanks,
> Judah
>
>
>
>
> On Mon, Jun 30, 2014 at 11:28 AM, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>
> Hi Judah
>
> Have you tried compiling with "link=shared" and without irstlm?
>
> The python interface to Moses is, afaik, incomplete and has not
> been heavily used. I have successfully used it to query a
> binarised phrase table but haven't tried its decoding interface.
>
> cheers - Barry
>
>
> On 30/06/14 16:04, Judah Schvimer wrote:
>
> Hi,
>
> I'm attempting to use the python interface to aid with a
> translation pipeline for longer pieces of text. Currently the
> pipeline is all written in python so this interface seemed
> perfect. I've run into a few problems however. The first
> problem is with compiling moses. Using "./bjam
> --libdir=/home/judah/mosesdecoder/lib link=shared -j12 -a" the
> compilation is successful, but using "./bjam
> --with-boost=/home/judah/boost_1_55_0
> --with-irstlm=/home/judah/irstlm-5.80.03
> -libdir=/home/judah/mosesdecoder/lib link=shared -j12 -a" the
> compilation fails. Compiling with only irstlm and no
> link=shared succeeds. Do you have any idea what's wrong? I can
> include a log if that would help. I have followed all of the
> documentation as well as I could. The error is:
>
>
> "/usr/bin/ld:
> /home/judah/irstlm-5.80.03/lib/libirstlm.a(dictionary.o):
> relocation R_X86_64_32 against `.rodata' can not be used when
> making a shared object; recompile with -fPIC
>
> /home/judah/irstlm-5.80.03/lib/libirstlm.a: error adding
> symbols: Bad value"
>
> I've tried recompiling both moses and irstlm with the -fPIC
> flag but nothing has worked so far.
>
> The second issue is with the interface itself. I may be
> misunderstanding the point of this interface but it seems to
> not translate anything longer than a 2 word phrase, is this
> something I'm doing wrong or is that all that it can do?
>
> Thanks,
>
> Judah
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 2
Date: Mon, 30 Jun 2014 12:14:31 -0400
From: Judah Schvimer <judah.schvimer@mongodb.com>
Subject: Re: [Moses-support] Python InterfaceSupport
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CALF9aB4y6J0NnXig8J7gyRJRc8q75MQfCtzWe6JK1QEs8+kZEA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

That looks perfect but I'm having trouble finding any method to take moses
phrase and reordering tables but use the cdec decoder. Is there a library
for doing that? I'd prefer not to recreate my system from the ground up
with cdec. I'm trying to translate .po files so I was hoping there would be
a way to translate sentence by sentence after extracting them with python.
My backup plan is to extract those all to a file, translate that file, and
then merge that file with the original .po file. That seems a bit hacky
however and it feels like there should be a better way.


On Mon, Jun 30, 2014 at 11:49 AM, Barry Haddow <bhaddow@staffmail.ed.ac.uk>
wrote:

> On 30/06/14 16:37, Judah Schvimer wrote:
>
>> Hi,
>>
>> Thanks for the quick response! I have tried just link=shared and it does
>> work. It's the combination of the two that doesn't work. It seems like
>> IRSTLM is what needs to be compiled with -fPIC but I can't get that to work.
>>
>> And what do you mean by query the binarised phrase table? How is that
>> different from decoding? I may be misunderstanding what the python
>> interface was created for.
>>
>
> I just mean that you can use the python interface to find all possible
> translations in the phrase table for a given segment. For me this was
> useful because I wanted to implement a method of training the phrase scores
> in python.
>
> It can be useful for the Python interface to do more than just run
> decoding, see here for example
> https://ufal.mff.cuni.cz/pbml/98/art-chahuneau-smith-dyer.pdf
>
>
>
>> Thanks,
>> Judah
>>
>>
>>
>>
>>
>> On Mon, Jun 30, 2014 at 11:28 AM, Barry Haddow <
>> bhaddow@staffmail.ed.ac.uk <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>>
>> Hi Judah
>>
>> Have you tried compiling with "link=shared" and without irstlm?
>>
>> The python interface to Moses is, afaik, incomplete and has not
>> been heavily used. I have successfully used it to query a
>> binarised phrase table but haven't tried its decoding interface.
>>
>> cheers - Barry
>>
>>
>> On 30/06/14 16:04, Judah Schvimer wrote:
>>
>> Hi,
>>
>> I'm attempting to use the python interface to aid with a
>> translation pipeline for longer pieces of text. Currently the
>> pipeline is all written in python so this interface seemed
>> perfect. I've run into a few problems however. The first
>> problem is with compiling moses. Using "./bjam
>> --libdir=/home/judah/mosesdecoder/lib link=shared -j12 -a" the
>> compilation is successful, but using "./bjam
>> --with-boost=/home/judah/boost_1_55_0
>> --with-irstlm=/home/judah/irstlm-5.80.03
>> -libdir=/home/judah/mosesdecoder/lib link=shared -j12 -a" the
>> compilation fails. Compiling with only irstlm and no
>> link=shared succeeds. Do you have any idea what's wrong? I can
>> include a log if that would help. I have followed all of the
>> documentation as well as I could. The error is:
>>
>>
>> "/usr/bin/ld:
>> /home/judah/irstlm-5.80.03/lib/libirstlm.a(dictionary.o):
>> relocation R_X86_64_32 against `.rodata' can not be used when
>> making a shared object; recompile with -fPIC
>>
>> /home/judah/irstlm-5.80.03/lib/libirstlm.a: error adding
>> symbols: Bad value"
>>
>> I've tried recompiling both moses and irstlm with the -fPIC
>> flag but nothing has worked so far.
>>
>> The second issue is with the interface itself. I may be
>> misunderstanding the point of this interface but it seems to
>> not translate anything longer than a 2 word phrase, is this
>> something I'm doing wrong or is that all that it can do?
>>
>> Thanks,
>>
>> Judah
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>> -- The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140630/e3295387/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 92, Issue 54
*********************************************

Tweet