Moses-support Digest, Vol 122, Issue 13

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: RNN-based features in Moses (Rico Sennrich)
2. Re: Conversion of phrase model to PhraseDictionaryCompact
(Hieu Hoang)
3. Re: Conversion of phrase model to PhraseDictionaryCompact
(Shubham Khandelwal)


----------------------------------------------------------------------

Message: 1
Date: Thu, 8 Dec 2016 17:39:39 +0000
From: Rico Sennrich <rico.sennrich@gmx.ch>
Subject: Re: [Moses-support] RNN-based features in Moses
To: moses-support@mit.edu
Message-ID: <1cf42d05-359f-fed8-32b3-0c71306d761e@gmx.ch>
Content-Type: text/plain; charset="windows-1252"

Hello Hakimeh,

the branch https://github.com/moses-smt/mosesdecoder/tree/nmt-hybrid
supports NMT (which is basically an RNN conditioned on the source text)
as a feature function in Moses. It is described in this paper:

@inproceedings{junczysdowmunt-dwojak-sennrich:2016:WMT,
address = "Berlin, Germany",
author = "Junczys-Dowmunt, Marcin and Dwojak, Tomasz and Sennrich,
Rico",
booktitle = "{Proceedings of the First Conference on Machine
Translation}",
month = "August",
pages = "319--325",
publisher = "Association for Computational Linguistics",
title = "{The AMU-UEDIN Submission to the WMT16 News Translation
Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT}",
url = "http://www.aclweb.org/anthology/W/W16/W16-2316",
year = "2016"
}

I'm afraid that documentation is currently lacking though.

best wishes,
Rico

On 08/12/16 08:34, Shafagh Fadaee wrote:
> Dear Moses Community,
> I'm searching for some RNN-based features added to Moses decoder or used in reranking Moses' n-best list.
> It seems Moses have some (LM) features based on Feedforward neural networks but I haven't find any implementation for recurrent neural networks in Moses.
> Apparently there are many efforts in this field but I have trouble finding released codes to use as guideline in my work.
> I would be happy to hear your suggestions.
> Best regards,
> Hakimeh Fadaei
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161208/0787fb13/attachment-0001.html

------------------------------

Message: 2
Date: Thu, 8 Dec 2016 19:20:10 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Conversion of phrase model to
PhraseDictionaryCompact
To: Shubham Khandelwal <skhlnmiit@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjY5JawXaThRiUwCPco0=U_SoGEeRkeMDWOWa+Su1d9dA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

the previous email you referred to says that the directory

* binarised-model/*

*must exist before you run it, otherwise it will segfault. I would also use
absolute path to make sure, ie. not *
*binarised-model/phrase-table *

*but*


* /home/shubham/moses/binarised-model/phrase-table *

*The previous email exchange also says you should try to add the argument*


* -T .*

Hieu Hoang
http://www.hoang.co.uk/hieu

On 8 December 2016 at 15:52, Shubham Khandelwal <skhlnmiit@gmail.com> wrote:

> Hello,
>
> This is just the reminder of my previous email.
>
> Thanking You.
>
> Regards,
> Shubham
>
> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal <skhlnmiit@gmail.com>
> wrote:
>
>> Hello,
>>
>> I have just downloaded phrase-table.2.gz (18GB) de-en model
>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made
>> models.
>> Now, I am converting them to PhraseDictionaryCompact using following
>> command (for exmaple):
>>
>>
>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in
>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table *
>>
>> But after passing 1/3, it gave following segementation fault error:
>>
>> *Pass 1/3: Creating hash function for rank assignment*
>> *Segmentation fault (core dumped)*
>>
>> I have found almost same issue on this thread:
>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033
>> However, I have provided the existing *binarised-model *folder in the
>> command. Also, I have the write-access in /tmp but still it gave
>> sementation fault.
>>
>> Can you please tell me what could be wrong here ?
>>
>> Thanking You.
>>
>> Regards,
>> Shubham
>>
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161208/f8b4f138/attachment-0001.html

------------------------------

Message: 3
Date: Fri, 9 Dec 2016 01:14:12 +0530
From: Shubham Khandelwal <skhlnmiit@gmail.com>
Subject: Re: [Moses-support] Conversion of phrase model to
PhraseDictionaryCompact
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAHweNTtB9afDxpjhXZmbZzDgytGizQXAXHqijERKodXXMs_vUA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Hieu,

Thanks for your reply.
Yes, I have used the absolute path and also I tried with -T but it did not
work.
Is there any other solution to this problem.

Btw, Can anybody please upload the compact model of all pre-made models as
this will take less space and also it will be very fast during decoding.

Thanks.

On Fri, Dec 9, 2016 at 12:50 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:

> the previous email you referred to says that the directory
>
> * binarised-model/*
>
> *must exist before you run it, otherwise it will segfault. I would also
> use absolute path to make sure, ie. not *
> *binarised-model/phrase-table *
>
> *but*
>
>
> * /home/shubham/moses/binarised-model/phrase-table *
>
> *The previous email exchange also says you should try to add the argument*
>
>
> * -T .*
>
> Hieu Hoang
> http://www.hoang.co.uk/hieu
>
> On 8 December 2016 at 15:52, Shubham Khandelwal <skhlnmiit@gmail.com>
> wrote:
>
>> Hello,
>>
>> This is just the reminder of my previous email.
>>
>> Thanking You.
>>
>> Regards,
>> Shubham
>>
>> On Thu, Dec 8, 2016 at 9:04 AM, Shubham Khandelwal <skhlnmiit@gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> I have just downloaded phrase-table.2.gz (18GB) de-en model
>>> and phrase-table.3.gz (22GB) fr-en model from the available pre-made
>>> models.
>>> Now, I am converting them to PhraseDictionaryCompact using following
>>> command (for exmaple):
>>>
>>>
>>> *~/mosesdecoder/bin/processPhraseTableMin -threads all -in
>>> ~/model/phrase-table.3.gz -nscores 4 -out binarised-model/phrase-table *
>>>
>>> But after passing 1/3, it gave following segementation fault error:
>>>
>>> *Pass 1/3: Creating hash function for rank assignment*
>>> *Segmentation fault (core dumped)*
>>>
>>> I have found almost same issue on this thread:
>>> http://comments.gmane.org/gmane.comp.nlp.moses.user/13033
>>> However, I have provided the existing *binarised-model *folder in the
>>> command. Also, I have the write-access in /tmp but still it gave
>>> sementation fault.
>>>
>>> Can you please tell me what could be wrong here ?
>>>
>>> Thanking You.
>>>
>>> Regards,
>>> Shubham
>>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161208/f2e9a3cc/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 122, Issue 13
**********************************************

Tweet