Moses-support Digest, Vol 95, Issue 28

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: On-Disk phrase table underpeforms as compared to filtered
phrase table (Hieu Hoang)
2. Re: dictionary in moses (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 17 Sep 2014 16:06:45 +0200
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] On-Disk phrase table underpeforms as
compared to filtered phrase table
To: Eleftherios Avramidis <Eleftherios.Avramidis@dfki.de>,
Moses-support <moses-support@mit.edu>
Message-ID: <54199575.6000309@gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed

the binary/on-disk phrase-tables created by
processPhraseTable
and
CreateOnDiskPt
work on slightly different ways so they won't give exactly the same
answer. However, the resulting BLEU scores should be similar.

The ways they are different:
1. CreateOnDiskPt only keeps the n-best p(e|f) for every source
phrase. n is the tableLimit argument when you run it. processPhraseTable
will keep all translation rules
2. The phrase-table created by processPhraseTable will sort and
prune the phrase-table during decoding according to the weighted scores.
The translation rules for each source phrase created by CreateOnDiskPt
will always be sorted and pruned according to p(e|f)

On 17/09/14 12:40, Eleftherios Avramidis wrote:
> Hi,
>
> the filtered phrase table is processed by filter-model-given-input.pl,
> with a parameter to use "processPhraseTable" for binarization
>
> best
> Lefteris
>
> PS: sorry for the delay, you were very fast, but Moses e-mails go to
> separate folder and I forgot to check it :-/
>
>
> On 15/09/14 21:14, Hieu Hoang wrote:
>> is the filtered phrase table also binarized with on-disk, or is it
>> text, or was it binarized with as PhraseDictionaryBinary?
>>
>> On 15/09/14 20:57, Eleftherios Avramidis wrote:
>>> Hi,
>>>
>>> I have trained a simple phrase-based system.
>>> First, I filtered the phrase table based on my test-set, and then I
>>> decoded the test-set (as typically EMS does).
>>> Secondly, I converted the original full phrase table to an On-Disk
>>> phrase table, and then sent the same sentences for decoding.
>>> My problem is that the sentences decoded with the on-disk full phrase
>>> table get different decoding than the ones from the filtered phrase
>>> table.
>>>
>>> I have checked that the moses ini weights, the language model, the
>>> re-ordering table, and the pre-processing scripts are the same.
>>>
>>> Does anybody know why this is happening?
>>>
>>> cheers
>>> Lefteris
>>>
>>>
>



------------------------------

Message: 2
Date: Wed, 17 Sep 2014 17:06:21 +0200
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] dictionary in moses
To: i.ramadan@saudisoft.com
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhXpZV5LuOCwkhrx3s6=LzG-a4CBEBr5P805pKF=7d+dA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

You can use the dictionary to
1. Create a primary primary phrase-table.
2. Add it to the parallel data

Your last question on the mailing list - 50 parallel sentences for the
tuning set is too small in most circumstances. I usually use around
1000-2000 sentences


On 17 September 2014 11:02, Ihab Ramadan <i.ramadan@saudisoft.com> wrote:

> Dear All,
>
> I tried the parameter ??dictionary dic.txt ? in the training process with
> no effect at all, how can I use my dictionary to get exact translation for
> the words
>
> thanks
>
>
>
> Best Regards
>
> *Ihab Ramadan*| Senior Developer| Saudisoft <http://www.saudisoft.com/> -
> Egypt | *Tel * +2 02 330 320 37 Ext- 0 | Mob+201007570826 | Fax
> +20233032036 | *Follow us on *[image: linked]
> <http://www.linkedin.com/company/77017?trk=vsrp_companies_res_name&trkInfo=VSRPsearchId%3A1489659901402995947155%2CVSRPtargetId%3A77017%2CVSRPcmpt%3Aprimary>* |
> **[image: ZA102637861]*
> <https://www.facebook.com/pages/Saudisoft-Co-Ltd/289968997768973?ref_type=bookmark>* |
> **[image: ZA102637858]* <https://twitter.com/Saudisoft>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140917/54e3de6e/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.gif
Type: image/gif
Size: 1351 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140917/54e3de6e/attachment-0003.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.gif
Type: image/gif
Size: 1317 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140917/54e3de6e/attachment-0004.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 1314 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140917/54e3de6e/attachment-0005.gif

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 95, Issue 28
*********************************************

Tweet