Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Recompile your moses for tcmalloc (Hieu Hoang)
2. Re: Binarising the phrase table (Per Tunedal)
----------------------------------------------------------------------
Message: 1
Date: Sun, 2 Mar 2014 08:05:28 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Recompile your moses for tcmalloc
To: "Aaron L.-F. Han" <hanlifengaaron@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID:
<CAEKMkbivtvE9t-q2ngk0A9=k=txZitt8ry+no2gPCaeADvzUxg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
I think that's correct. Inputtype can be 0 if your input is plain text and
you're using a hiero or string-to-tree model.
It doesn't matter if you set input type =3. Moses just see it as a tree
with no non terminals
On 2 Mar 2014 05:55, "Aaron L.-F. Han" <hanlifengaaron@gmail.com> wrote:
> Hi Hieu,
>
> For the tree-based translation based on Moses, what should be the
> parameter value of "search-algorithm" and "inputtype" in the generated
> Moses.ini file? the default values are 3 for both the two parameters.
>
>
> Bests, Aaron
>
>
> On Sat, Mar 1, 2014 at 6:46 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
>> ah, this is the source of the 46% slowdown in Moses v 2.1.
>>
>> cs-en Europarl (filtered), 50 input sentencesv 1.0 v 2.1tcmalloc fixPhrase-BasedBinary
>> 20.13229.334 (-46%) 20.618 (-2%)Compact 19.98526.422 17.847 (11%)On-Disk26.83117.945
>> (11%) Hierarchical On-Disk39.43353.943 39.492 (0%)
>>
>> What the Lord giveth, the Lord taketh away
>>
>>
>>
>> On 1 March 2014 08:12, Kenneth Heafield <moses@kheafield.com> wrote:
>>
>>> Hi,
>>>
>>> If you got Moses since
>>>
>>> Thu Aug 22 10:16:12 2013 +0100
>>>
>>> namely commit 6b5c1a0 and installed tcmalloc to make threading faster,
>>> I'm sorry to say it wasn't being linked. It's entirely my fault and has
>>> been fixed in 952bb76.
>>>
>>> Kenneth
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
>
> Master Degree Candidate, Research Assistant,
>
> Natural Language Processing & Portuguese-Chinese Machine Translation
> Laboratory
>
> Address: Room 108, Research Building, University of Macau, Av. Padre Tom?s
> Pereira Taipa, Macau, China
>
> Homepage: http://www.linkedin.com/in/aaronhan
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140302/c71ba1be/attachment-0001.htm
------------------------------
Message: 2
Date: Sun, 02 Mar 2014 14:09:24 +0100
From: Per Tunedal <per.tunedal@operamail.com>
Subject: Re: [Moses-support] Binarising the phrase table
To: moses-support@mit.edu
Message-ID: <1393765748.25640.484030AD@webmail.messagingengine.com>
Content-Type: text/plain; charset="utf-8"
Hi Massinissa,
thank you very much. It works like a charm.
I'm really impressed how fast the model is loaded now.
Yours,
Per Tunedal
On Fri, Feb 28, 2014, at 10:54, Massinissa Ahmim wrote:
Hi Per,
Yes, you have change the 0 into 1 to use binary files. Actually you
should do that only for the phrase-table, meaning :
1 0 0
5 /home/per/working/train/binarised-model/Kryptering1.sv-fr.phrase-tabl
e
0-0 wbe-msd-bidirectional-fe-allff
6 /home/per/working/train/binarised-model/Kryptering1.sv-fr.reordering-
table
My last comment could be kinda misleading but by the 'same thing' I
meant changing the paths.
It should work now assuming the paths are accurate
Regards
Massinissa
2014-02-28 10:42 GMT+01:00 Per Tunedal <[1]per.tunedal@operamail.com>:
Hi Massinissa,
my commands:
~/mosesdecoder/bin/processPhraseTable \
-ttable 0 0 train/model/Kryptering1.sv-fr.phrase-table.pruned \
-nscores 5 -out
train/binarised-model/Kryptering1.sv-fr.phrase-table
~/mosesdecoder/bin/processLexicalTable \
-in train/model/reordering-table.wbe-msd-bidirectional-fe.gz \
-out train/binarised-model/Kryptering1.sv-fr.reordering-table
I tried the following changes in moses.ini:
# OLD FORMAT a binary table type (1) is assumed
[ttable-file]
#0 0 0 5 /home/per/working/train/model/phrase-table.gz
0 0 0 5
/home/per/working/train/binarised-model/Kryptering1.sv-fr.phrase-tab
le
# distortion (reordering) files
[distortion-file]
#0-0 wbe-msd-bidirectional-fe-allff 6
/home/per/working/train/model/reordering-table.wbe-msd-bidirectional
-fe.gz
0-0 wbe-msd-bidirectional-fe-allff 6
/home/per/working/train/binarised-model/Kryptering1.sv-fr.reordering
-table
But it doesn't work.
Maybe changing the zeros to ones would do the trick?
1 0 0 5
/home/per/working/train/binarised-model/Kryptering1.sv-fr.phrase-tab
le
1-0 wbe-msd-bidirectional-fe-allff 6
/home/per/working/train/binarised-model/Kryptering1.sv-fr.reordering
-table
Yours,
Per Tunedal
On Fri, Feb 28, 2014, at 9:55, Massinissa Ahmim wrote:
> Hi Per,
>
> Could you please past here the command your ran to binarise the
> Phrase-table?
>
> The current paths in your moses.ini should be to the out-put of
> processPhraseTable,
> for instance, if the output was
> /home/per/working/train/model/whatever/phrase-table/
> , then you have to change your current path by this one making sure
to
> use
> the stemming (taking the .gz off as I did above) and switching the
first
> moses parameter (from 0 to 1) as follows :
>
>
> 1 0 0 5 /home/per/working/train/model/whatever/phrase-table/
>
>
> Same thing with the reordering table
>
> Regards
>
> Massinissa
>
>
> 2014-02-28 7:47 GMT+01:00 Per Tunedal <[2]per.tunedal@operamail.com>:
>
> >
> > Hi,
> > tried to binarise the phrase table and got in to trouble.
> >
> > 1. Error messages, as below. What's that?
> > distinct source phrases: 439511 distinct first words of source
phrases:
> > 67600 number of phrase pairs (line count): 940438
> > Count of lines with missing alignments: 0/940438
> > WARNING: there are src voc entries with no phrase translation:
count
> > 2168
> > There exists phrase translations for 65432 entries
> >
> > 2. Modify the moses.ini file. I've found this on the Moses/Baseline
> > page:
> > 1. Change PhraseDictionaryMemory to PhraseDictionaryBinary
> > 2. Set the path of the PhraseDictionary feature to point to
> >
$HOME/working/train/binarised-model/Kryptering1.sv-fr.phrase-table
> > 3. Set the path of the LexicalReordering feature to point to
> >
$HOME/working/train/binarised-model/Kryptering1.sv-fr.reordering-table
> >
> > But I cannot find any such entries in my moses.ini - maybe because
I'm
> > running a somewhat older version of Moses. I've found e.g. the
following
> > lines in my ini-file:
> >
> > [ttable-file]
> > 0 0 0 5 /home/per/working/train/model/phrase-table.gz
> >
> > # distortion (reordering) files
> > [distortion-file]
> > 0-0 wbe-msd-bidirectional-fe-allff 6
> >
/home/per/working/train/model/reordering-table.wbe-msd-bidirectional-fe
.gz
> >
> > How should I change the entries to use my binarised model?
> >
> > Yours,
> > Per Tunedal
> >
> > _______________________________________________
> > Moses-support mailing list
> > [3]Moses-support@mit.edu
> > [4]http://mailman.mit.edu/mailman/listinfo/moses-support
> >
>
>
>
> --
>
> [image: Description : Description : lingua_custodia_final full logo]
>
> *The Translation Trustee*
>
> *1, Place Charles de Gaulle*
>
> *78180 Montigny-le-Bretonneux*
>
> *Tel : [5]+33 1 30 44 04 23 Mobile : [6]+33 7 61 44 40 84*
>
> *Email :* *[7]massinissa.ahmim@linguacustodia.com
> <[8]massinissa.ahmim@linguacustodia.com>*
>
> *Website :* *[9]www.linguacustodia.com
<[10]http://www.linguacustodia.com/> -
> [11]www.thetranslationtrustee.com
<[12]http://www.thetranslationtrustee.com>*
>
> ? Pensez ? l'environnement, n'imprimez ce courriel que si n?cessaire.
>
> Please do not print this email unless it is absolutely necessary.
Spread
> environmental awareness.
> Email had 1 attachment:
> + image001.jpg
> 6k (image/jpeg)
--
Description : Description : lingua_custodia_final full logo
The Translation Trustee
1, Place Charles de Gaulle
78180 Montigny-le-Bretonneux
Tel : +33 1 30 44 04 23 Mobile : +33 7 61 44 40 84
Email : [13]massinissa.ahmim@linguacustodia.com
Website : [14]www.linguacustodia.com -
[15]www.thetranslationtrustee.com
? Pensez ? l?environnement, n?imprimez ce courriel que si n?cessaire.
Please do not print this email unless it is absolutely
necessary. Spread environmental awareness.
Email had 1 attachment:
* image001.jpg
6k (image/jpeg)
References
1. mailto:per.tunedal@operamail.com
2. mailto:per.tunedal@operamail.com
3. mailto:Moses-support@mit.edu
4. http://mailman.mit.edu/mailman/listinfo/moses-support
5. tel:%2B33%201%2030%2044%2004%2023
6. tel:%2B33%207%2061%2044%2040%2084
7. mailto:massinissa.ahmim@linguacustodia.com
8. mailto:massinissa.ahmim@linguacustodia.com
9. http://www.linguacustodia.com/
10. http://www.linguacustodia.com/
11. http://www.thetranslationtrustee.com/
12. http://www.thetranslationtrustee.com/
13. mailto:massinissa.ahmim@linguacustodia.com
14. http://www.linguacustodia.com/
15. http://www.thetranslationtrustee.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140302/fc4fbd0a/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140302/fc4fbd0a/attachment.jpg
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 89, Issue 2
********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 89, Issue 2"
Post a Comment