Moses-support Digest, Vol 91, Issue 19

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Fwd: Regarding an issue during Translation (Chitra C)
2. Re: Installing moses on my mac with SRILM: undefined symbol
error (Catherine Gasnier)


----------------------------------------------------------------------

Message: 1
Date: Wed, 14 May 2014 13:02:15 +0530
From: Chitra C <chitrac03@gmail.com>
Subject: [Moses-support] Fwd: Regarding an issue during Translation
To: moses-support@mit.edu
Message-ID:
<CACDiBTE7XOXo+t45nJMUP7nT9yHSztgOkJrm5ZceLQ6zRg7Tqg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello Barry,

Thank you for your immediate response.

I am using the same version as specified in the webpage,it is irstlm-5.80.03


On Tue, May 13, 2014 at 6:35 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk>wrote:

> Hi Chitra
>
> Can you check that you are using the version of IRSTLM specified in the
> "baseline system" (http://www.statmt.org/moses/?n=Moses.Baseline) page?
> The "--text yes" argument should work.
>
> Also, the "baseline system" page also explains how to create a kenlm
> binary file from an ARPA file. It looks like you are passing an irstlm
> binary file to kenlm, and that is why Moses is failing.
>
> cheers - Barry
>
>
> On 13/05/14 11:40, Chitra C wrote:
>
>> Hello Moses-Support team,
>>
>> My name is Chitra, I am working on SMT[ Statistical Machine Translation ]
>> from English to Kannada Language. I am using Giza,IRSTLM and Moses tools.
>>
>> I am following Oses manual for installation and training.Installation of
>> all the tools are successful,but during the training of language model i
>> have a problem.
>>
>> when i execute the following command
>>
>> ~/irstlm/bin/compile-lm \
>> --text yes \
>> kannada.lm.kn.gz \
>> kannada.arpa.kn <http://kannada.arpa.kn>
>>
>>
>> I am getting a warning called-Too many arguments
>> In order to avoid this, i gave following command
>>
>> ~/irstlm/bin/compile-lm \
>> --text \
>> kannada.lm.kn.gz \
>> kannada.arpa.kn <http://kannada.arpa.kn>
>>
>>
>> arpa file got created,but i am getting error when i try to do
>> tuning,testing ..etc.
>>
>> The error was like
>>
>> Loading the LM will be faster if you build a binary file.
>> Reading kannada.blm.kn <http://kannada.blm.kn>
>>
>> ----5---10---15---20---25---30---35---40---45---50---55---
>> 60---65---70---75---80---85---90---95--100
>> ************************************************************
>> ****************************************
>> lm/read_arpa.cc:63 in void lm::ReadARPACounts(util::FilePiece&,
>> std::vector<long long unsigned int, std::allocator<long long unsigned int>
>> >&) threw FormatLoadException because `line.size() >= 4 &&
>> StringPiece(line.data(), 4) == "blmt"'.
>> This looks like an IRSTLM binary file. Did you forget to pass --text
>> yes to compile-lm? Byte: 40 File: kannada.blm.kn <http://kannada.blm.kn>
>>
>>
>> Please help me out to solve this problem,
>> Thank you in advance.
>>
>> Regards,
>> Chitra C
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140514/d2f9b8e0/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 14 May 2014 14:40:21 +0200
From: Catherine Gasnier <cath.gasnier@gmail.com>
Subject: Re: [Moses-support] Installing moses on my mac with SRILM:
undefined symbol error
To: moses-support@MIT.EDU
Message-ID:
<CAKpbGS1OxEMU+xYn8E=hEBjD_660jwhvhJBT1n5BzSJ0T33LNQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

So, I finally figured out the whole how and why of things.
While compiling SRILM, it compiles with option '-I/usr/include -L/usr/lib'
because this is where it wants to look for TCL. Therefore, it will also use
the iconv header which is in those directories, which is the one with
symbols starting with '_iconv...'.
However, when I compiled Moses, I compiled with option
'--with-macports=/opt/local', so when dynamically linking, it will find the
iconv library which is in /opt/local, i.e. the MacPorts one, with symbols
starting with '_libiconv...', and complain.
There are several workarounds for this, but what I would simply advise is
not to use MacPorts, nor even use the boost library installed with MacPorts
(because it will have the same effect).
In fact, I will personally uninstall MacPorts and never use it again:
having duplicate libraries is a source of big mess. Use Homebrew instead.
But that's just personal taste.

Hope that will help anyone having the same issue.

Catherine Gasnier
Rue du pont 22
1003 Lausanne
Switzerland
+41 78 914 38 48


On 14 May 2014 14:27, Hieu Hoang <hieuhoang@gmail.com> wrote:

> cool. share your pearls of wisdom with other mac users on the mailing list
>
>
> On 14/05/2014 12:55, Catherine Gasnier wrote:
>
> mmh, just because irstlm does not compile either :D
>
> So, I finally figured out the whole how and why of things.
> While compiling SRILM, it compiles with option '-I/usr/include -L/usr/lib'
> because this is where it wants to look for TCL. Therefore, it will also use
> the iconv header which is in those directories, which is the one with
> symbols starting with '_iconv...'.
> However, when I compiled Moses, I compiled with option
> '--with-macports=/opt/local', so when dynamically linking, it will find the
> iconv library which is in /opt/local, i.e. the MacPorts one, with symbols
> starting with '_libiconv...', and complain.
> There are several workarounds for this, but what I would simply advise is
> not to use MacPorts, nor even use the boost library installed with MacPorts
> (because it will have the same effect), and even uninstall MacPorts and
> never use it again: having duplicate library is a source of big mess. Use
> Homebrew instead.
>
> Hope that will help anyone having the same issue.
>
> Catherine Gasnier
> Rue du pont 22
> 1003 Lausanne
> Switzerland
> +41 78 914 38 48
>
>
> On 13 May 2014 17:18, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
>> it was easy for me but i can't remember how i set up my laptop.
>>
>> why do you need srilm? these days, there's far better tools for training
>> & querying lm. The only thing that i know it has that noone else does is lm
>> interpolation
>>
>>
>> On 13 May 2014 15:54, Catherine Gasnier <cath.gasnier@gmail.com> wrote:
>>
>>> You told me you have had this issue, but didn't remember what you did
>>> exactly.
>>>
>>> Would you know exactly how you compiled SRILM?
>>>
>>> Catherine Gasnier
>>> Rue du pont 22
>>> 1003 Lausanne
>>> Switzerland
>>> +41 78 914 38 48 <%2B41%2078%20914%2038%2048>
>>>
>>>
>>> On 13 May 2014 12:11, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>>>
>>>> dunno, it always worked for me without any special handling. Maybe i'm
>>>> just lucky
>>>>
>>>>
>>>> On 13 May 2014 11:05, Catherine Gasnier <cath.gasnier@gmail.com> wrote:
>>>>
>>>>> I know, I have the same, and it is precisely the source of the error
>>>>> for me: the compiler will fetch libiconv in /opt/local/lib, which is the
>>>>> Macports one and which has the wrong symbols (_libiconv_open for example),
>>>>> whereas it would be nice if it could by itself (configuring the Jamroot for
>>>>> example) fetch libiconv in /usr/lib, which is the libiconv which has the
>>>>> right symbols (_iconv_open for example).
>>>>>
>>>>>
>>>>> Catherine Gasnier
>>>>> Rue du pont 22
>>>>> 1003 Lausanne
>>>>> Switzerland
>>>>> +41 78 914 38 48 <%2B41%2078%20914%2038%2048>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Hieu Hoang
>>>> Research Associate
>>>> University of Edinburgh
>>>> http://www.hoang.co.uk/hieu
>>>>
>>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140514/6137decd/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 91, Issue 19
*********************************************

Tweet