Moses-support Digest, Vol 98, Issue 35

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Free corpus for evaluation of Word Confidence Estimation
(WCE) in Spoken Language Translation (Laurent Besacier)
2. Re: When does this error message occur and how to fix it
(Raj Dabre)


----------------------------------------------------------------------

Message: 1
Date: Fri, 12 Dec 2014 10:28:07 +0100
From: Laurent Besacier <laurent.besacier@imag.fr>
Subject: [Moses-support] Free corpus for evaluation of Word Confidence
Estimation (WCE) in Spoken Language Translation
To: "'moses-support'" <moses-support@mit.edu>
Message-ID: <1A5AA4E9-BCC0-4F52-B180-3E004A1BC8D5@imag.fr>
Content-Type: text/plain; charset="iso-8859-1"

Dear all,

Some of you might be interested by a free downlable corpus on Word Confidence Estimation (WCE) in Spoken Language Translation
This corpus contains 2643 speech utterances for which a quintuplet containing: ASR output (src-asr), verbatim transcript (src-ref), text translation output (tgt-mt), speech translation output
(tgt-slt) and post-edition of translation (tgt-pe), is made available.

Link to the corpus (as well as IWSLT 2014 describing it) :
https://github.com/besacier/WCE-SLT-LIG

best regards

Laurent Besacier


------------------------
Laurent Besacier
Professeur ? l'Universit? Joseph Fourier (Grenoble 1)
Laboratoire d'Informatique de Grenoble (LIG)
Membre Junior de l'Institut Universitaire de France (IUF 2012-2017)
laurent.besacier@imag.fr
-------------------------
Pour venir me voir: LIG - Batiment B
41, rue des Math?matiques - Domaine Universitaire
Appeler le 35695 en bas du batiment B (car l'entr?e est badg?e)
--------------------------





-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141212/8f4aad60/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1879 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20141212/8f4aad60/attachment-0001.bin

------------------------------

Message: 2
Date: Fri, 12 Dec 2014 19:48:22 +0900
From: Raj Dabre <prajdabre@gmail.com>
Subject: Re: [Moses-support] When does this error message occur and
how to fix it
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAB3gfjAZqXtEbuyiu_RH87sG07ZfAq505bMp_oFVjcXZ8yir_g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hey,

This is actually quite close to what I need.
There is one limitation though.... this works only for phrase tables and
not for reordering tables.
I can do a workround by treating the reordering table as a phrase table....
but this code is designed to accept the only phrase table format.

So I decided to change the code a little.
More specifically: I tried to append a dummy alignment and count string
component like "0-0 ||| 1 3 1 ||| |||"
This makes the line resemble the standard phrase table format.

But I get errors like: std::bad_alloc
Since I am using hardcoded strings... this might be the issue.
Kindly advise me on how I can do this.

Regards.


On Thu, Dec 11, 2014 at 8:17 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
> there's also is the 'KenPT' that was create by Nikolay Bogoychev. We've
> been meaning to see if it could be incorporated into the KenLM main
> repository
> https://github.com/XapaJIaMnu
>
> On 10 December 2014 at 20:27, Kenneth Heafield <moses@kheafield.com>
> wrote:
>
>> I for one would like a version of phrase table querying that doesn't
>> depend on the rest of Moses. For cleanliness and for portability
>> (there's no reason this shouldn't kill cdec's phrase table for example).
>>
>> Kenneth
>>
>> On 12/10/14 13:39, Marcin Junczys-Dowmunt wrote:
>> > I am on it, sort of. I am waiting for Raj, to give me a stack trace. I
>> > admit this code is basically abandoned, but let's see what we can do.
>> >
>> > W dniu 10.12.2014 o 19:35, Hieu Hoang pisze:
>> >> I see. I'm not sure why the querying depends on StaticData. There may
>> >> be some weights is wants to read. Only Marcin will know about that.
>> >> You could try to debug yourself.
>> >>
>> >> If you intend to develop with Moses, I would urge you to update to the
>> >> master branch of Moses. People forget how old code works and often
>> >> can't answer your questions. The new code has also been cleaned up and
>> >> rationalised. For instance, the 'translation system' functionality was
>> >> problematic and has been deleted
>> >>
>> >> On 10 December 2014 at 15:53, Raj Dabre <prajdabre@gmail.com
>> >> <mailto:prajdabre@gmail.com>> wrote:
>> >>
>> >> I am not using any moses.ini because I am not translating anything.
>> >> My focus is on reading the phrase tables using a querying program
>> >> written by Marcin.
>> >>
>> >> On Thu, Dec 11, 2014 at 12:50 AM, Hieu Hoang <Hieu.Hoang@ed.ac.uk
>> >> <mailto:Hieu.Hoang@ed.ac.uk>> wrote:
>> >>
>> >> Are you using the translation system functionality in v1? That
>> >> seems to be the problem. Can I have a look @ the moses.ini.
>> >>
>> >> On 10 December 2014 at 15:44, Raj Dabre <prajdabre@gmail.com
>> >> <mailto:prajdabre@gmail.com>> wrote:
>> >>
>> >> I am using an older version (V1).
>> >> I am not running a translation experiment but using Java
>> >> Native Interface to call C functions.
>> >> Can you tell me what might be the problem ?
>> >> I can surely say that the root of the problem is in the
>> >> StaticData.cpp file.
>> >> It seems that I am creating 2 instances of a class trying
>> >> to access a background moses process.
>> >> Does this sound about right ?
>> >>
>> >>
>> >> On Thu, Dec 11, 2014 at 12:37 AM, Hieu Hoang
>> >> <Hieu.Hoang@ed.ac.uk <mailto:Hieu.Hoang@ed.ac.uk>> wrote:
>> >>
>> >> what version of Moses are you using? The current
>> >> master branch on github, or an older version (version
>> >> 1 etc)?
>> >>
>> >> Can I please see the moses.ini file you're using too
>> >>
>> >> On 10 December 2014 at 12:34, Raj Dabre
>> >> <prajdabre@gmail.com <mailto:prajdabre@gmail.com>>
>> wrote:
>> >>
>> >> Dear all,
>> >>
>> >> I am using a JNI interface to the CPP code that
>> >> reads from binary phrase tables.
>> >>
>> >> I get an error like: Mismatch between number of
>> >> word penalties and number of translation systems
>> >>
>> >> This happens when I create 2 objects for reading
>> >> from 2 different tables.
>> >>
>> >> My understanding is that there is some problem
>> >> with Static data but I am not sure.
>> >>
>> >> Can anyone help me ?
>> >>
>> >> --
>> >> Raj Dabre.
>> >> Research Student,
>> >> Graduate School of Informatics,
>> >> Kyoto University.
>> >> CSE MTech, IITB., 2011-2014
>> >>
>> >>
>> >> _______________________________________________
>> >> Moses-support mailing list
>> >> Moses-support@mit.edu <mailto:
>> Moses-support@mit.edu>
>> >>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Hieu Hoang
>> >> Research Associate
>> >> University of Edinburgh
>> >> http://www.hoang.co.uk/hieu
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Raj Dabre.
>> >> Research Student,
>> >> Graduate School of Informatics,
>> >> Kyoto University.
>> >> CSE MTech, IITB., 2011-2014
>> >>
>> >>
>> >> _______________________________________________
>> >> Moses-support mailing list
>> >> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Hieu Hoang
>> >> Research Associate
>> >> University of Edinburgh
>> >> http://www.hoang.co.uk/hieu
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Raj Dabre.
>> >> Research Student,
>> >> Graduate School of Informatics,
>> >> Kyoto University.
>> >> CSE MTech, IITB., 2011-2014
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Hieu Hoang
>> >> Research Associate
>> >> University of Edinburgh
>> >> http://www.hoang.co.uk/hieu
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Moses-support mailing list
>> >> Moses-support@mit.edu
>> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> >
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>

--
Raj Dabre.
Research Student,
Graduate School of Informatics,
Kyoto University.
CSE MTech, IITB., 2011-2014
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141212/89547bba/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 98, Issue 35
*********************************************

Tweet