Moses-support Digest, Vol 122, Issue 15

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Conversion of phrase model to PhraseDictionaryCompact
(Hieu Hoang)
2. Fwd: (G/her G/libanos)


----------------------------------------------------------------------

Message: 1
Date: Fri, 9 Dec 2016 10:57:19 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Conversion of phrase model to
PhraseDictionaryCompact
To: Shubham Khandelwal <skhlnmiit@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjpVBdTJf=FWRrfoEPUuiqvLhrSLQqKu090UONZuuO6fg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

This is a hierarchical model. You must binarize with CreateOnDiskPt for
this model

Hieu Hoang
http://www.hoang.co.uk/hieu

On 9 December 2016 at 08:18, Shubham Khandelwal <skhlnmiit@gmail.com> wrote:

> Hello,
>
> Thanks. It worked now. I have created compact phrase table.
> Now, when I am running it using following command:
>
> *~/mosesdecoder/bin/moses
> -f ~/Translate/models/de-en/model/moses.ini.2 -threads all*
>
> Then, after creating input-output object, it gave following segmentation
> fault error:
>
> Created input-output object : [14.796] seconds
> Ich bin ein Student
> Line 0: Initialize search took 0.000 seconds total
> Translating: <s> Ich bin ein Student </s> ||| [0,0]=X (1) [0,1]=X (1)
> [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1) [1,2]=X (1)
> [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1) [2,4]=X (1)
> [2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1) [4,5]=X (1)
> [5,5]=X (1)
>
> Segmentation fault (core dumped)
>
> In my machine, I have 40GB RAM but still I am confused why it gave this
> error.
> Can you please help me in this regard. I have attached moses.ini.2 for
> your reference.
>
> Thanks.
>
> Regards,
> Shubham
>
> On Fri, Dec 9, 2016 at 2:02 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> maybe try
>>
>> -encoding None
>>
>> On 08/12/2016 19:44, Shubham Khandelwal wrote:
>>
>> Hi Hieu,
>>
>> Thanks for your reply.
>> Yes, I have used the absolute path and also I tried with -T but it did
>> not work.
>> Is there any other solution to this problem.
>>
>> Btw, Can anybody please upload the compact model of all pre-made models
>> as this will take less space and also it will be very fast during decoding.
>>
>> Thanks.
>>
>> On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> the previous email you referred to says that the directory
>>>
>>> * binarised-model/ *
>>>
>>> *must exist before you run it, otherwise it will segfault. I would also
>>> use absolute path to make sure, ie. not *
>>> *binarised-model/phrase-table *
>>>
>>> *but *
>>>
>>>
>>> * /home/shubham/moses/binarised-model/phrase-table *
>>>
>>> *The previous email exchange also says you should try to add the
>>> argument *
>>>
>>>
>>> * -T . *
>>>
>>> Hieu Hoang
>>> http://www.hoang.co.uk/hieu
>>>
>>> On 8 December 2016 at 15:52, Shubham Khandelwal <skhlnmiit@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> This is just the reminder of my previous email.
>>>>
>>>> Thanking You.
>>>>
>>>> Regards,
>>>> Shubham
>>>>
>>>> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal <skhlnmiit@gmail.com
>>>> > wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I have just downloaded phrase-table.2.gz (18GB) de-en model
>>>>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made
>>>>> models.
>>>>> Now, I am converting them to PhraseDictionaryCompact using following
>>>>> command (for exmaple):
>>>>>
>>>>>
>>>>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in
>>>>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table *
>>>>>
>>>>> But after passing 1/3, it gave following segementation fault error:
>>>>>
>>>>> *Pass 1/3: Creating hash function for rank assignment*
>>>>> *Segmentation fault (core dumped)*
>>>>>
>>>>> I have found almost same issue on this thread:
>>>>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033
>>>>> However, I have provided the existing *binarised-model *folder in the
>>>>> command. Also, I have the write-access in /tmp but still it gave
>>>>> sementation fault.
>>>>>
>>>>> Can you please tell me what could be wrong here ?
>>>>>
>>>>> Thanking You.
>>>>>
>>>>> Regards,
>>>>> Shubham
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161209/d9681b32/attachment-0001.html

------------------------------

Message: 2
Date: Fri, 9 Dec 2016 06:17:49 -0800
From: "G/her G/libanos" <gerizaba@gmail.com>
Subject: [Moses-support] Fwd:
To: moses-support@mit.edu
Message-ID:
<CALR6RoTVs=hZfzedL8rkKzK5F3ZVeguVupDJkBzUenrrzypZrg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

hello there....
1. we done our work using baseline and the system work for the train data
but we need to localized the source code in the tokinization to leave
abrivated word ?/? as in english Adm. as Notbreaking_prefix.en that is use
the dot(.)

2. where we to change the code in c++ or in the perl part of the codes



3. we need to uses the translation system in window, since the end users
are not expert in ubuntu and we see the how to change the window see the
Moses GUI and which part of the our file will be load to the model only or
all the train including the train data and can we modify the GUI of the
Moses.



10qs for every thing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161209/e099b1ff/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 122, Issue 15
**********************************************

Tweet