Moses-support Digest, Vol 122, Issue 18

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Complete phrase-based SMT suite as native Windows
application suite. (Tom Hoar)
2. Re: Conversion of phrase model to PhraseDictionaryCompact
(Shubham Khandelwal)


----------------------------------------------------------------------

Message: 1
Date: Sat, 10 Dec 2016 20:31:26 +0700
From: Tom Hoar <tahoar@pttools.net>
Subject: Re: [Moses-support] Complete phrase-based SMT suite as native
Windows application suite.
To: Gher Glibanos <gerizaba@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <0eb35c6f-ac43-6664-2f3f-a69f240e2255@pttools.net>
Content-Type: text/plain; charset="windows-1252"

Hi Gher,

We sell an entire/turn-key phrase-based SMT system called Slate Desktop.
It includes corpus management, engine management, connectors to popular
CAT tools and the entire Moses stack, all running as a native Windows
application suite.

You'll find all you need at http://pttools.net. No need to reply. You
can purchase via our online e-commerce store. Our customers report being
up and running with their installed system within 20 minutes. Depending
on the size of your TMs, engines typically complete overnight. Good luck.


Best regards,

Tom Hoar
Chief Executive Officer
*/Precision Translation Tools Pte Ltd/*
Singapore/Thailand
Web: www.precisiontranslationtools.com
<http://www.precisiontranslationtools.com>
Thailand Mobile: +66 87 345-1875
Skype call: tahoar <skype:tahoar?call>
Skype chat: tahoar <skype:tahoar>


On 12/10/2016 6:30 PM, moses-support-request@mit.edu wrote:
> On Fri, Dec 9, 2016 at 9:17 AM, G/her G/libanos<gerizaba@gmail.com> wrote:
>
>> hello there....
>> 1. we done our work using baseline and the system work for the train data
>> but we need to localized the source code in the tokinization to leave
>> abrivated word ?/? as in english Adm. as Notbreaking_prefix.en that is use
>> the dot(.)
>>
>> 2. where we to change the code in c++ or in the perl part of the codes
>>
>>
>>
>> 3. we need to uses the translation system in window, since the end users
>> are not expert in ubuntu and we see the how to change the window see the
>> Moses GUI and which part of the our file will be load to the model only or
>> all the train including the train data and can we modify the GUI of the
>> Moses.
>>
>>
>>
>> 10qs for every thing
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:http://mailman.mit.edu/mailman/private/moses-support/attachments/20161209/b68d0dd5/attachment-0001.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161210/812be24a/attachment-0001.html

------------------------------

Message: 2
Date: Sat, 10 Dec 2016 19:36:08 +0530
From: Shubham Khandelwal <skhlnmiit@gmail.com>
Subject: Re: [Moses-support] Conversion of phrase model to
PhraseDictionaryCompact
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAHweNTtj5OeNQNw+Yvy5GCdMAYVqr5b_+73jD96gfmMUR0D_4w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Yes, CreateOnDiskPt command executed without any error.

There are 5 files in this phrase-table.3.folder: Misc.dat , Source.dat,
TargetColl.dat, TargetInd.dat, Vocab.dat
*Misc.dat and Vocab.dat files are empty. *
I just checked that my hard-disk memory is full as this folder took 165G
already. So may be, due to this reason those 2 files are empty. But
CreateOnDiskPt
command should throw an error of *No space left on machine *when it stopped.

Let me know if no space on my machine, is the issue or not so that I can go
for better device having more hard-disk memory.

Also May I know that How much memory phrase-table.3.folder has in general
when CreateOnDiskPt command executes completely as phrase-table.3.gz size
is only 23GB.

Thanking You.


On Sat, Dec 10, 2016 at 6:53 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:

> strange, did the CreateOnDiskPt command execute ok, ie. with no error?
>
> Does this file exist:
> /home/shubham/models/fr-en/phrase-table.3.folder/Misc.dat
> If you do
> cat Misc.dat
> what does it say?
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
> On 10 December 2016 at 11:30, Shubham Khandelwal <skhlnmiit@gmail.com>
> wrote:
>
>> Hi,
>>
>> Thanks Hieu for your reply.
>> I have used *CreateOnDiskPt* to binarize the model and stored in
>> *phrase-table.3.folder *using following command:
>>
>>
>> *~/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2 phrase-table.3.gz
>> phrase-table.3.folder*
>>
>> Also I have made changes in *moses.ini.3 (*i.e. I have converted
>> *PhraseDictionaryMemory* to *PhraseDictionaryCompact* as follows*) *
>>
>> PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
>> path=/home/shubham/models/fr-en/phrase-table.3.folder input-factor=0
>> output-factor=0
>>
>> Now, when I run it using :* ~/mosesdecoder/bin/moses -f moses.ini.3 * ,
>> it gave following error after *Created input-output object*:
>>
>> *terminate called after throwing an instance of 'util::Exception'*
>> * what(): OnDiskPt/OnDiskWrapper.cpp:217 in uint64_t
>> OnDiskPt::OnDiskWrapper::GetMisc(const string&) const threw util::Exception
>> because `iter == m_miscInfo.end()'.*
>> *Couldn't find value for key NumSourceFactors*
>> *Aborted (core dumped)*
>>
>> Here, I do not know that what key value should I pass and how ? Can you
>> please help me in this regard.
>>
>> Thank you so much for your help.
>>
>> Regards,
>> Shubham
>>
>> On Fri, Dec 9, 2016 at 4:27 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> This is a hierarchical model. You must binarize with CreateOnDiskPt for
>>> this model
>>>
>>> Hieu Hoang
>>> http://www.hoang.co.uk/hieu
>>>
>>> On 9 December 2016 at 08:18, Shubham Khandelwal <skhlnmiit@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Thanks. It worked now. I have created compact phrase table.
>>>> Now, when I am running it using following command:
>>>>
>>>> *~/mosesdecoder/bin/moses
>>>> -f ~/Translate/models/de-en/model/moses.ini.2 -threads all*
>>>>
>>>> Then, after creating input-output object, it gave following
>>>> segmentation fault error:
>>>>
>>>> Created input-output object : [14.796] seconds
>>>> Ich bin ein Student
>>>> Line 0: Initialize search took 0.000 seconds total
>>>> Translating: <s> Ich bin ein Student </s> ||| [0,0]=X (1) [0,1]=X (1)
>>>> [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1) [1,2]=X (1)
>>>> [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1) [2,4]=X (1)
>>>> [2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1) [4,5]=X (1)
>>>> [5,5]=X (1)
>>>>
>>>> Segmentation fault (core dumped)
>>>>
>>>> In my machine, I have 40GB RAM but still I am confused why it gave this
>>>> error.
>>>> Can you please help me in this regard. I have attached moses.ini.2 for
>>>> your reference.
>>>>
>>>> Thanks.
>>>>
>>>> Regards,
>>>> Shubham
>>>>
>>>> On Fri, Dec 9, 2016 at 2:02 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>>
>>>>> maybe try
>>>>>
>>>>> -encoding None
>>>>>
>>>>> On 08/12/2016 19:44, Shubham Khandelwal wrote:
>>>>>
>>>>> Hi Hieu,
>>>>>
>>>>> Thanks for your reply.
>>>>> Yes, I have used the absolute path and also I tried with -T but it did
>>>>> not work.
>>>>> Is there any other solution to this problem.
>>>>>
>>>>> Btw, Can anybody please upload the compact model of all pre-made
>>>>> models as this will take less space and also it will be very fast during
>>>>> decoding.
>>>>>
>>>>> Thanks.
>>>>>
>>>>> On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <hieuhoang@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> the previous email you referred to says that the directory
>>>>>>
>>>>>> * binarised-model/ *
>>>>>>
>>>>>> *must exist before you run it, otherwise it will segfault. I would
>>>>>> also use absolute path to make sure, ie. not *
>>>>>> *binarised-model/phrase-table *
>>>>>>
>>>>>> *but *
>>>>>>
>>>>>>
>>>>>> * /home/shubham/moses/binarised-model/phrase-table *
>>>>>>
>>>>>> *The previous email exchange also says you should try to add the
>>>>>> argument *
>>>>>>
>>>>>>
>>>>>> * -T . *
>>>>>>
>>>>>> Hieu Hoang
>>>>>> http://www.hoang.co.uk/hieu
>>>>>>
>>>>>> On 8 December 2016 at 15:52, Shubham Khandelwal <skhlnmiit@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> This is just the reminder of my previous email.
>>>>>>>
>>>>>>> Thanking You.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shubham
>>>>>>>
>>>>>>> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal <
>>>>>>> skhlnmiit@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I have just downloaded phrase-table.2.gz (18GB) de-en model
>>>>>>>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made
>>>>>>>> models.
>>>>>>>> Now, I am converting them to PhraseDictionaryCompact using
>>>>>>>> following command (for exmaple):
>>>>>>>>
>>>>>>>>
>>>>>>>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in
>>>>>>>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table *
>>>>>>>>
>>>>>>>> But after passing 1/3, it gave following segementation fault error:
>>>>>>>>
>>>>>>>> *Pass 1/3: Creating hash function for rank assignment*
>>>>>>>> *Segmentation fault (core dumped)*
>>>>>>>>
>>>>>>>> I have found almost same issue on this thread:
>>>>>>>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033
>>>>>>>> However, I have provided the existing *binarised-model *folder in
>>>>>>>> the command. Also, I have the write-access in /tmp but still it
>>>>>>>> gave sementation fault.
>>>>>>>>
>>>>>>>> Can you please tell me what could be wrong here ?
>>>>>>>>
>>>>>>>> Thanking You.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Shubham
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Moses-support mailing list
>>>>>>> Moses-support@mit.edu
>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161210/bb126b28/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 122, Issue 18
**********************************************

Tweet