Moses-support Digest, Vol 98, Issue 33

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: When does this error message occur and how to fix it
(Raj Dabre)
2. Tree-to-Tree rule table column definition (Ben Ao)


----------------------------------------------------------------------

Message: 1
Date: Thu, 11 Dec 2014 12:06:58 +0900
From: Raj Dabre <prajdabre@gmail.com>
Subject: Re: [Moses-support] When does this error message occur and
how to fix it
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAB3gfjAma+9zro0r6D+R3kjHr_gkq-ntC1eYQM-1troxRL9dBg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Also I want to report something odd.

These segmentation faults seem to be occurring quite randomly.

For example: I wrote a standalone program which just creates 2 objects of
the JNI class and queries using both.
This runs sometimes and sometimes gives the segmentation fault.

Secondly: When I run my program using NETBEANS the segfaults almost never
occur.


This is confusing.

On Thu, Dec 11, 2014 at 11:56 AM, Raj Dabre <prajdabre@gmail.com> wrote:

> Hello,
>
> I am attaching the jni error message I get when I run my Java program.
> Thanks in advance for the help.
>
>
> On Thu, Dec 11, 2014 at 5:48 AM, Marcin Junczys-Dowmunt <
> junczys@amu.edu.pl> wrote:
>
>> I am waiting for some free time around Christmas. Let's see what I can
>> cook up then. A critical mass of missing/half-implemented features and
>> heisenbugs has accumulated in the compact phrase table. Before that
>> hoewever, I would like to find some time to finally fix LM pruning,
>> maybe next week. The phrase table is next on my list.
>>
>> W dniu 10.12.2014 o 21:37, Christian Hardmeier pisze:
>> > I'm also a potential user of a standalone phrase table. I currently
>> link my decoder against moses mostly because of the phrase table, and it's
>> not always easy to do so.
>> >
>> > /Christian
>> >
>> > On 10.12.2014, at 15:27, Kenneth Heafield wrote:
>> >
>> >> I for one would like a version of phrase table querying that doesn't
>> >> depend on the rest of Moses. For cleanliness and for portability
>> >> (there's no reason this shouldn't kill cdec's phrase table for
>> example).
>> >>
>> >> Kenneth
>> >>
>> >> On 12/10/14 13:39, Marcin Junczys-Dowmunt wrote:
>> >>> I am on it, sort of. I am waiting for Raj, to give me a stack trace. I
>> >>> admit this code is basically abandoned, but let's see what we can do.
>> >>>
>> >>> W dniu 10.12.2014 o 19:35, Hieu Hoang pisze:
>> >>>> I see. I'm not sure why the querying depends on StaticData. There may
>> >>>> be some weights is wants to read. Only Marcin will know about that.
>> >>>> You could try to debug yourself.
>> >>>>
>> >>>> If you intend to develop with Moses, I would urge you to update to
>> the
>> >>>> master branch of Moses. People forget how old code works and often
>> >>>> can't answer your questions. The new code has also been cleaned up
>> and
>> >>>> rationalised. For instance, the 'translation system' functionality
>> was
>> >>>> problematic and has been deleted
>> >>>>
>> >>>> On 10 December 2014 at 15:53, Raj Dabre <prajdabre@gmail.com
>> >>>> <mailto:prajdabre@gmail.com>> wrote:
>> >>>>
>> >>>> I am not using any moses.ini because I am not translating
>> anything.
>> >>>> My focus is on reading the phrase tables using a querying program
>> >>>> written by Marcin.
>> >>>>
>> >>>> On Thu, Dec 11, 2014 at 12:50 AM, Hieu Hoang <
>> Hieu.Hoang@ed.ac.uk
>> >>>> <mailto:Hieu.Hoang@ed.ac.uk>> wrote:
>> >>>>
>> >>>> Are you using the translation system functionality in v1?
>> That
>> >>>> seems to be the problem. Can I have a look @ the moses.ini.
>> >>>>
>> >>>> On 10 December 2014 at 15:44, Raj Dabre <prajdabre@gmail.com
>> >>>> <mailto:prajdabre@gmail.com>> wrote:
>> >>>>
>> >>>> I am using an older version (V1).
>> >>>> I am not running a translation experiment but using Java
>> >>>> Native Interface to call C functions.
>> >>>> Can you tell me what might be the problem ?
>> >>>> I can surely say that the root of the problem is in the
>> >>>> StaticData.cpp file.
>> >>>> It seems that I am creating 2 instances of a class trying
>> >>>> to access a background moses process.
>> >>>> Does this sound about right ?
>> >>>>
>> >>>>
>> >>>> On Thu, Dec 11, 2014 at 12:37 AM, Hieu Hoang
>> >>>> <Hieu.Hoang@ed.ac.uk <mailto:Hieu.Hoang@ed.ac.uk>>
>> wrote:
>> >>>>
>> >>>> what version of Moses are you using? The current
>> >>>> master branch on github, or an older version (version
>> >>>> 1 etc)?
>> >>>>
>> >>>> Can I please see the moses.ini file you're using too
>> >>>>
>> >>>> On 10 December 2014 at 12:34, Raj Dabre
>> >>>> <prajdabre@gmail.com <mailto:prajdabre@gmail.com>>
>> wrote:
>> >>>>
>> >>>> Dear all,
>> >>>>
>> >>>> I am using a JNI interface to the CPP code that
>> >>>> reads from binary phrase tables.
>> >>>>
>> >>>> I get an error like: Mismatch between number of
>> >>>> word penalties and number of translation systems
>> >>>>
>> >>>> This happens when I create 2 objects for reading
>> >>>> from 2 different tables.
>> >>>>
>> >>>> My understanding is that there is some problem
>> >>>> with Static data but I am not sure.
>> >>>>
>> >>>> Can anyone help me ?
>> >>>>
>> >>>> --
>> >>>> Raj Dabre.
>> >>>> Research Student,
>> >>>> Graduate School of Informatics,
>> >>>> Kyoto University.
>> >>>> CSE MTech, IITB., 2011-2014
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Moses-support mailing list
>> >>>> Moses-support@mit.edu <mailto:
>> Moses-support@mit.edu>
>> >>>>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Hieu Hoang
>> >>>> Research Associate
>> >>>> University of Edinburgh
>> >>>> http://www.hoang.co.uk/hieu
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Raj Dabre.
>> >>>> Research Student,
>> >>>> Graduate School of Informatics,
>> >>>> Kyoto University.
>> >>>> CSE MTech, IITB., 2011-2014
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Moses-support mailing list
>> >>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> >>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Hieu Hoang
>> >>>> Research Associate
>> >>>> University of Edinburgh
>> >>>> http://www.hoang.co.uk/hieu
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Raj Dabre.
>> >>>> Research Student,
>> >>>> Graduate School of Informatics,
>> >>>> Kyoto University.
>> >>>> CSE MTech, IITB., 2011-2014
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Hieu Hoang
>> >>>> Research Associate
>> >>>> University of Edinburgh
>> >>>> http://www.hoang.co.uk/hieu
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> Moses-support mailing list
>> >>>> Moses-support@mit.edu
>> >>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> Moses-support mailing list
>> >>> Moses-support@mit.edu
>> >>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >>>
>> >> _______________________________________________
>> >> Moses-support mailing list
>> >> Moses-support@mit.edu
>> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
>
> --
> Raj Dabre.
> Research Student,
> Graduate School of Informatics,
> Kyoto University.
> CSE MTech, IITB., 2011-2014
>
>


--
Raj Dabre.
Research Student,
Graduate School of Informatics,
Kyoto University.
CSE MTech, IITB., 2011-2014
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141211/841ee24e/attachment-0001.htm

------------------------------

Message: 2
Date: Thu, 11 Dec 2014 16:52:39 +0800
From: Ben Ao <lobichai@hotmail.com>
Subject: [Moses-support] Tree-to-Tree rule table column definition
To: <moses-support@mit.edu>
Cc: lobichai@hotmail.com
Message-ID: <COL129-DS2354411290D15EAF3901B6C4630@phx.gbl>
Content-Type: text/plain; charset="iso-2022-jp"

Dear all,



I was trained the tree-to-tree rule-table from Chinese to English.

The command I used to train the model is:



~/mosesdecoder/scripts/training/train-model.perl \

-external-bin-dir /share/apps/moses/gcc/giza-bin-dir \

-root-dir zct2t_en \

-corpus test.10.moses -f cn -e en \

-alignment grow-diag-final-and -source-syntax -target-syntax -glue-grammar \

-lm 0:3:/home/ben/data/zc.blm.en:1





However, I find out the rule-table is very strange. Such as:

Brunel ?? [VP][VP] ? [IP] ||| now brunel may have [VP][VP] , [TOP] ||| 0.
499999 7.48437e-05 0.499999 0.0159349 2.718 ||| 0-0 0-1 1-2 1-3 2-4 3-5 |||
0.166667 0.166667 0.0833333

CA? ? [VP][VBD] ? [IP] ||| ca : i [VP][VBD] . [TOP] ||| 0.5 0.15398 0.5
0.0413438 2.718 ||| 0-0 0-1 1-2 2-3 3-4 ||| 0.222222 0.222222 0.111111



Question 1: There contain 5 probabilities in the middle:

0.499999 7.48437e-05 0.499999 0.0159349 2.718

0.5 0.15398 0.5 0.0413438 2.
718

The last part ?2.718? is same in all rules of the rule-table, but what it
means?



Question 2: What is meaning for the last part in the rule?

0.166667 0.166667 0.0833333

0.222222 0.222222 0.111111

It look like some probabilities, but what it means?





I worked on this problem for a few weeks, and no idea of the definition for
the rule-table generated by tree-to-tree training.



Many and many thanks for solving those problem!!



Regards,

Ben

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141211/0be347be/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 98, Issue 33
*********************************************

Tweet