Moses-support Digest, Vol 92, Issue 55

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Python InterfaceSupport (Barry Haddow)
2. Re: Python InterfaceSupport (Judah Schvimer)


----------------------------------------------------------------------

Message: 1
Date: Mon, 30 Jun 2014 20:07:43 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Python InterfaceSupport
To: Judah Schvimer <judah.schvimer@mongodb.com>
Cc: moses-support@mit.edu
Message-ID: <53B1B57F.1070403@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"

Hi Judah

The pycdec interface is specific to cdec - I was just using it to show
what a more complete python interface would do. The Moses python
interface was never really finished and to my knowledge there is no-one
working on it now. Anyone interested is welcome to pick it up ...

If you just want to translate from python though, then it may be
simplest just to run Moses using the subprocess module. If you're
interested in translating po files then there has been some work in
integrating Moses into localisation workflows - m4loc for example - but
I'm not really familiar with it,

cheers - Barry

On 30/06/14 17:14, Judah Schvimer wrote:
> That looks perfect but I'm having trouble finding any method to take
> moses phrase and reordering tables but use the cdec decoder. Is there
> a library for doing that? I'd prefer not to recreate my system from
> the ground up with cdec. I'm trying to translate .po files so I was
> hoping there would be a way to translate sentence by sentence after
> extracting them with python. My backup plan is to extract those all to
> a file, translate that file, and then merge that file with the
> original .po file. That seems a bit hacky however and it feels like
> there should be a better way.
>
>
> On Mon, Jun 30, 2014 at 11:49 AM, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>
> On 30/06/14 16:37, Judah Schvimer wrote:
>
> Hi,
>
> Thanks for the quick response! I have tried just link=shared
> and it does work. It's the combination of the two that doesn't
> work. It seems like IRSTLM is what needs to be compiled with
> -fPIC but I can't get that to work.
>
> And what do you mean by query the binarised phrase table? How
> is that different from decoding? I may be misunderstanding
> what the python interface was created for.
>
>
> I just mean that you can use the python interface to find all
> possible translations in the phrase table for a given segment. For
> me this was useful because I wanted to implement a method of
> training the phrase scores in python.
>
> It can be useful for the Python interface to do more than just
> run decoding, see here for example
> https://ufal.mff.cuni.cz/pbml/98/art-chahuneau-smith-dyer.pdf
>
>
>
> Thanks,
> Judah
>
>
>
>
>
> On Mon, Jun 30, 2014 at 11:28 AM, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk
> <mailto:bhaddow@staffmail.ed.ac.uk>
> <mailto:bhaddow@staffmail.ed.ac.uk
> <mailto:bhaddow@staffmail.ed.ac.uk>>> wrote:
>
> Hi Judah
>
> Have you tried compiling with "link=shared" and without
> irstlm?
>
> The python interface to Moses is, afaik, incomplete and
> has not
> been heavily used. I have successfully used it to query a
> binarised phrase table but haven't tried its decoding
> interface.
>
> cheers - Barry
>
>
> On 30/06/14 16:04, Judah Schvimer wrote:
>
> Hi,
>
> I'm attempting to use the python interface to aid with a
> translation pipeline for longer pieces of text.
> Currently the
> pipeline is all written in python so this interface seemed
> perfect. I've run into a few problems however. The first
> problem is with compiling moses. Using "./bjam
> --libdir=/home/judah/mosesdecoder/lib link=shared -j12
> -a" the
> compilation is successful, but using "./bjam
> --with-boost=/home/judah/boost_1_55_0
> --with-irstlm=/home/judah/irstlm-5.80.03
> -libdir=/home/judah/mosesdecoder/lib link=shared -j12
> -a" the
> compilation fails. Compiling with only irstlm and no
> link=shared succeeds. Do you have any idea what's
> wrong? I can
> include a log if that would help. I have followed all
> of the
> documentation as well as I could. The error is:
>
>
> "/usr/bin/ld:
> /home/judah/irstlm-5.80.03/lib/libirstlm.a(dictionary.o):
> relocation R_X86_64_32 against `.rodata' can not be
> used when
> making a shared object; recompile with -fPIC
>
> /home/judah/irstlm-5.80.03/lib/libirstlm.a: error adding
> symbols: Bad value"
>
> I've tried recompiling both moses and irstlm with the
> -fPIC
> flag but nothing has worked so far.
>
> The second issue is with the interface itself. I may be
> misunderstanding the point of this interface but it
> seems to
> not translate anything longer than a 2 word phrase, is
> this
> something I'm doing wrong or is that all that it can do?
>
> Thanks,
>
> Judah
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> <mailto:Moses-support@mit.edu <mailto:Moses-support@mit.edu>>
>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> -- The University of Edinburgh is a charitable body,
> registered in
> Scotland, with registration number SC005336.
>
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140630/bb6ad57c/attachment-0001.htm

------------------------------

Message: 2
Date: Mon, 30 Jun 2014 15:33:16 -0400
From: Judah Schvimer <judah.schvimer@mongodb.com>
Subject: Re: [Moses-support] Python InterfaceSupport
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CALF9aB4aFmKBkb7Gfwnjsu8C8538w4fa9QyrHu61Jk-fKsRXtA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks! I'll check out m4loc and if not use subprocess. I hadn't seen m4loc
and it might just do the trick!


On Mon, Jun 30, 2014 at 3:07 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk>
wrote:

> Hi Judah
>
> The pycdec interface is specific to cdec - I was just using it to show
> what a more complete python interface would do. The Moses python interface
> was never really finished and to my knowledge there is no-one working on it
> now. Anyone interested is welcome to pick it up ...
>
> If you just want to translate from python though, then it may be simplest
> just to run Moses using the subprocess module. If you're interested in
> translating po files then there has been some work in integrating Moses
> into localisation workflows - m4loc for example - but I'm not really
> familiar with it,
>
> cheers - Barry
>
>
> On 30/06/14 17:14, Judah Schvimer wrote:
>
> That looks perfect but I'm having trouble finding any method to take moses
> phrase and reordering tables but use the cdec decoder. Is there a library
> for doing that? I'd prefer not to recreate my system from the ground up
> with cdec. I'm trying to translate .po files so I was hoping there would be
> a way to translate sentence by sentence after extracting them with python.
> My backup plan is to extract those all to a file, translate that file, and
> then merge that file with the original .po file. That seems a bit hacky
> however and it feels like there should be a better way.
>
>
> On Mon, Jun 30, 2014 at 11:49 AM, Barry Haddow <bhaddow@staffmail.ed.ac.uk
> > wrote:
>
>> On 30/06/14 16:37, Judah Schvimer wrote:
>>
>>> Hi,
>>>
>>> Thanks for the quick response! I have tried just link=shared and it does
>>> work. It's the combination of the two that doesn't work. It seems like
>>> IRSTLM is what needs to be compiled with -fPIC but I can't get that to work.
>>>
>>> And what do you mean by query the binarised phrase table? How is that
>>> different from decoding? I may be misunderstanding what the python
>>> interface was created for.
>>>
>>
>> I just mean that you can use the python interface to find all possible
>> translations in the phrase table for a given segment. For me this was
>> useful because I wanted to implement a method of training the phrase scores
>> in python.
>>
>> It can be useful for the Python interface to do more than just run
>> decoding, see here for example
>> https://ufal.mff.cuni.cz/pbml/98/art-chahuneau-smith-dyer.pdf
>>
>>
>>
>>> Thanks,
>>> Judah
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Jun 30, 2014 at 11:28 AM, Barry Haddow <
>>> bhaddow@staffmail.ed.ac.uk <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>>>
>>> Hi Judah
>>>
>>> Have you tried compiling with "link=shared" and without irstlm?
>>>
>>> The python interface to Moses is, afaik, incomplete and has not
>>> been heavily used. I have successfully used it to query a
>>> binarised phrase table but haven't tried its decoding interface.
>>>
>>> cheers - Barry
>>>
>>>
>>> On 30/06/14 16:04, Judah Schvimer wrote:
>>>
>>> Hi,
>>>
>>> I'm attempting to use the python interface to aid with a
>>> translation pipeline for longer pieces of text. Currently the
>>> pipeline is all written in python so this interface seemed
>>> perfect. I've run into a few problems however. The first
>>> problem is with compiling moses. Using "./bjam
>>> --libdir=/home/judah/mosesdecoder/lib link=shared -j12 -a" the
>>> compilation is successful, but using "./bjam
>>> --with-boost=/home/judah/boost_1_55_0
>>> --with-irstlm=/home/judah/irstlm-5.80.03
>>> -libdir=/home/judah/mosesdecoder/lib link=shared -j12 -a" the
>>> compilation fails. Compiling with only irstlm and no
>>> link=shared succeeds. Do you have any idea what's wrong? I can
>>> include a log if that would help. I have followed all of the
>>> documentation as well as I could. The error is:
>>>
>>>
>>> "/usr/bin/ld:
>>> /home/judah/irstlm-5.80.03/lib/libirstlm.a(dictionary.o):
>>> relocation R_X86_64_32 against `.rodata' can not be used when
>>> making a shared object; recompile with -fPIC
>>>
>>> /home/judah/irstlm-5.80.03/lib/libirstlm.a: error adding
>>> symbols: Bad value"
>>>
>>> I've tried recompiling both moses and irstlm with the -fPIC
>>> flag but nothing has worked so far.
>>>
>>> The second issue is with the interface itself. I may be
>>> misunderstanding the point of this interface but it seems to
>>> not translate anything longer than a 2 word phrase, is this
>>> something I'm doing wrong or is that all that it can do?
>>>
>>> Thanks,
>>>
>>> Judah
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> -- The University of Edinburgh is a charitable body, registered
>>> in
>>> Scotland, with registration number SC005336.
>>>
>>>
>>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140630/e3a802f6/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 92, Issue 55
*********************************************

Tweet