Moses-support Digest, Vol 88, Issue 5

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Factored SMT Language Models Segmentation Fault (Hieu Hoang)
2. Re: Factored SMT Language Models Segmentation Fault
(Kenneth Heafield)
3. Re: Building problem (Lane Schwartz)
4. Re: Get word-to-word alignments from mosesserver (Tomas Fulajtar)


----------------------------------------------------------------------

Message: 1
Date: Mon, 3 Feb 2014 17:32:53 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Factored SMT Language Models Segmentation
Fault
To: Jeffrey Micher <jmicher@cs.cmu.edu>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhVG6nQCVd+D8=mH4HMoMh6XdH9fCYYuAv0kGSz=h-1ZA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

can you please let us look at your moses.ini file, a few lines of your
phrase-table and each of the LM (the text files, not the binarized files)


On 3 February 2014 16:43, Jeffrey Micher <jmicher@cs.cmu.edu> wrote:

> Hi,
>
>
>
> I'm trying to build a factored SMT system with 3 output lms. I used KenLM
> for a surface form model and lemma model. When I tried to use it for my
> POS model, I get a Kneser-Ney smoothing error. So for this model I use
> SRILM with witten bell smoothing. For tuning, the models load into memory
> just fine, but when the first word in the tuning file is encountered, I get
> a seg fault. I tried using both binarized LMs with lmplz, and just regular
> arpa format. I also tried making sure lazyken was set to false, but I'm
> still getting a seg fault.
>
>
>
> Any help with this would be greatly appreciated.
>
>
>
> Jeffrey
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140203/ef93ccc9/attachment-0001.htm

------------------------------

Message: 2
Date: Mon, 03 Feb 2014 09:37:51 -0800
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Factored SMT Language Models Segmentation
Fault
To: moses-support@mit.edu
Message-ID: <52EFD3EF.20504@kheafield.com>
Content-Type: text/plain; charset=windows-1252

Hi,

I second what Hieu said. My first guess is some number disagreement
with factor indices. To be clear, you're loading all three models with
KenLM which were binarized with build_binary?

It would help if you could run inside gdb and provide a backtrace.
Debugging symbols (-g) are enabled by default but adding "debug" to your
bjam arguments then running with gdb will provide more useful information.

Kenneth


On 02/03/14 09:32, Hieu Hoang wrote:
> can you please let us look at your moses.ini file, a few lines of your
> phrase-table and each of the LM (the text files, not the binarized files)
>
>
> On 3 February 2014 16:43, Jeffrey Micher <jmicher@cs.cmu.edu
> <mailto:jmicher@cs.cmu.edu>> wrote:
>
> Hi,____
>
> __ __
>
> I?m trying to build a factored SMT system with 3 output lms. I used
> KenLM for a surface form model and lemma model. When I tried to use
> it for my POS model, I get a Kneser-Ney smoothing error. So for
> this model I use SRILM with witten bell smoothing. For tuning, the
> models load into memory just fine, but when the first word in the
> tuning file is encountered, I get a seg fault. I tried using both
> binarized LMs with lmplz, and just regular arpa format. I also
> tried making sure lazyken was set to false, but I?m still getting a
> seg fault. ____
>
> __ __
>
> Any help with this would be greatly appreciated.____
>
> __ __
>
> Jeffrey____
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 3
Date: Mon, 3 Feb 2014 21:45:01 +0000
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] Building problem
To: Sehrob Ibrohimov <isehrob@gmail.com>, "moses-support@mit.edu"
<moses-support@mit.edu>
Message-ID:
<CABv3vZkjCKDzwGy=GpUhxVMWnFERTN6ED-wi+kX0sqEQL6Q9Aw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Sehrob,

Thanks for the info. I believe this is fixed in the newer releases of
Moses. Try the version 2.1:

http://www.statmt.org/moses/?n=Moses.Releases

Let me know if the problem still exists in v2.1.

Thanks,
Lane


On Mon, Feb 3, 2014 at 5:34 PM, Sehrob Ibrohimov <isehrob@gmail.com> wrote:
> Lane,
>
> I got it, thanks! I found out what release is moses, it is 0.91. And I
> thought, maybe, it will be useful to tell you the version of OS and
> gcc etc. too, so my OS - Ubuntu 12.04 and gcc 4.6.
> Thank you for your support, any way! )
>
> Sehrob.
>
> On 2/3/14, Lane Schwartz <dowobeha@gmail.com> wrote:
>> No, I did not implement cross-lingual triggers.
>>
>> On Mon, Feb 3, 2014 at 3:11 PM, Prashant Mathur <prashant@fbk.eu> wrote:
>>> Hi Lane,
>>>
>>> Since you mentioned this paper, did you also implement MI triggers for
>>> LM?
>>> I tried using cross-lingual triggers in Moses as mentioned in [1], but it
>>> didn't seem to work for me.
>>>
>>> [1] Using inter-lingual triggers for machine translation. Lavecchia et.
>>> al.
>>> 07.
>>> link : http://www.loria.fr/~smaili/interspeechCaro07.pdf
>>>
>>> Thanks,
>>> --
>>> Prashant
>>>
>>>
>>> On 02/03/2014 03:41 PM, Lane Schwartz wrote:
>>>
>>> Tom,
>>>
>>> The original paper is Xiong et al (2011)
>>> "Enhancing Language Models in Statistical Machine Translation with
>>> Backward N-grams and Mutual Information Triggers"
>>> http://aclweb.org/anthology//P/P11/P11-1129.pdf
>>>
>>> I implemented the backward n-gram functionality in Moses, beginning
>>> with the following commit:
>>> https://github.com/moses-smt/mosesdecoder/commit/8459a86137f7d15834009557a62c4278479da088
>>>
>>> You can browse my subsequent related commits:
>>> https://github.com/moses-smt/mosesdecoder/commits?author=dowobeha
>>>
>>> Lane
>>>
>>>
>>> On Mon, Feb 3, 2014 at 2:13 PM, Tom Hoar
>>> <tahoar@precisiontranslationtools.com> wrote:
>>>
>>> I've seen several references to "backward LM" on the list. I just
>>> searched
>>> statmt.org/moses but didn't find any details. What is a good source for a
>>> description?
>>> Thanks.
>>>
>>>
>>>
>>>
>>> On 02/03/2014 09:06 PM, Hieu Hoang wrote:
>>>
>>> If the programs
>>> bin/moses
>>> bin/moses_chart
>>> and others have been created, maybe you can ignore the error.
>>>
>>> The error is from a unit test for the backward LM. If may to be specific
>>> to
>>> a gcc version or OS. If you don't use backward LM, you may not need to
>>> worry
>>> about it
>>>
>>>
>>>
>>> On 2 February 2014 14:09, Sehrob Ibrohimov <isehrob@gmail.com> wrote:
>>>
>>> Hello everybody!
>>> I am having building problems. when I try to build the moses I get
>>> this error message:
>>>
>>> ##########################################################################
>>> ====== BEGIN OUTPUT ======
>>> Running 1 test case...
>>> moses/LM/BackwardTest.cpp(162): error in "ProbingAll":
>>> difference{100%} between static_cast<double>(TransformLMScore( 0.0
>>> )){0} and static_cast<double>(ngramScore){1.3724476843890443e-07}
>>> exceeds 0.01%
>>>
>>> *** 1 failure detected in test suite "BackwardTest"
>>>
>>> EXIT STATUS: 201
>>> ====== END OUTPUT ======
>>> ...failed testing.capture-output
>>>
>>> moses/LM/bin/BackwardTest.test/gcc-4.6/release/debug-symbols-on/link-static/threading-multi/BackwardTest.run...
>>> ...failed updating 1 target...
>>> ...skipped 1 target...
>>> ...updated 4 targets...
>>> The build failed with command line:
>>> ./jam-files/bjam --debug-configuration -d2
>>>
>>> ############################################################################
>>>
>>> I've attached the build.log. please help me fix the problem!
>>> thanks a lot!
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Research Associate
>>> University of Edinburgh
>>> http://www.hoang.co.uk/hieu
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> When a place gets crowded enough to require ID's, social collapse is not
>> far away. It is time to go elsewhere. The best thing about space travel
>> is that it made it possible to go elsewhere.
>> -- R.A. Heinlein, "Time Enough For Love"
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>



--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"


------------------------------

Message: 4
Date: Tue, 4 Feb 2014 11:11:13 +0000
From: Tomas Fulajtar <TomasFu@moravia.com>
Subject: Re: [Moses-support] Get word-to-word alignments from
mosesserver
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>, Jyotesh Choudhari
<jyoteshrc@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<0F546C639409F6479DF2CE6DD5267B16A0404A74@dag-cz-1.CZ.moravia-it.com>
Content-Type: text/plain; charset="iso-8859-2"

Hi Hieu, Jyotesh,

I am also interested about this feature - it would be useful for the M4loc project and its reinsert_wordalign.pm<https://code.google.com/p/m4loc/wiki/Pod_reinsert_wordalign> tool.

Could you please let us know about the progress? Can I help you somehow?

Thank you,


Tom?? Fulajt?r | Reseacher
T: +420-545-552-340
tomasfu@moravia.com<mailto:tomasfu@moravia.com> | moravia.com<http://www.moravia.com/>

From: Hieu Hoang [mailto:Hieu.Hoang@ed.ac.uk]
Sent: Wednesday, January 15, 2014 12:52 PM
To: Jyotesh Choudhari
Cc: moses-support
Subject: Re: [Moses-support] Get word-to-word alignments from mosesserver

I don't think it's been implemented. However, if you want to implement it, I can help you. Please email me offline and I'll show you how

On 13 January 2014 13:44, Jyotesh Choudhari <jyoteshrc@gmail.com<mailto:jyoteshrc@gmail.com>> wrote:
Hi,
According to Moses tutorial
(http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc30), mosesserver
returns only phrase alignments. I want word alignments too. How can I get these?

Thanks and regards,
Jyotesh Choudhari

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140204/c110d3d8/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 88, Issue 5
********************************************

Tweet