Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. unformatted biconcor output (Alex Martinez)
2. Reading phrase table taking very long (rohit dholakia)
3. Re: Reading phrase table taking very long (rohit dholakia)
4. Re: Reading phrase table taking very long (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Mon, 09 Sep 2013 16:55:41 +0000 (GMT)
From: Alex Martinez <cmxela@me.com>
Subject: [Moses-support] unformatted biconcor output
To: moses-support@mit.edu
Message-ID: <b8c61d89-8a42-4316-a78d-fcadc04e8801@me.com>
Content-Type: text/plain; charset="utf-8"
Hi,
i have succeeded installing the Web interface to the EMS and I'm playing with the config.toy. I still have a small problem... when I click over a source word to get the concordance (biconcor) the output I get is unformatted like....
finally(15) enfin , ceux qui s' opposent ? une r?glementation sur les r?mun?rations seront s?rement ceux qui nous mettront en garde contre les ? cons?quences fortuites ? . ||| finally , those opposing pay regulation are certain to warn us about " unintended consequences . " ||| 0 0 ||| 0 0 ||| 0-0 1-1 2-2 19-3 10-4 11-4 7-5 8-5 5-6 5-7 6-8 3-9 4-9 5-9 17-10 9-11 22-12 24-13 23-14 26-15 enfin , d' autres politiques des pays riches pourraient ?tre plus efficaces que l' aide internationale . ||| finally , other rich country policies may be much more beneficial than aid . ||| 0 0 ||| 0 0 ||| 0-0 1-1 2-2 3-2 5-3 7-3 6-4 4-5 8-6 9-7 11-8 10-9 11-10 12-11 13-12 14-12 15-12 enfin , dans le cadre de l' appr?ciation de soci?t?s contr?l?es , il importe de ne pas juger l' ind?pendance des administrateurs essentiellement sur leur ind?pendance vis @-@ ?-vis du conseil de la soci?t? o? ils si?gent , mais plut?t vis @-@ ?-vis de son actionnaire majoritaire . ||| finally , when assessing con!
trolled companies , the independence of directors should not be judged largely by looking at the extent to which they are independent of the company on whose board they serve . rather , considerable attention should be given to their independence from the controlling shareholder . ||| 0 0 ||| 0 0 ||| 0-0 1-1 2-2 7-3 10-4 9-5 11-6 18-7 18-8 19-8 20-9 21-10 13-11 15-12 16-12 17-13 17-14 17-15 17-16 17-17 30-18 29-19 28-20 31-21 32-27 33-28 23-29 34-31 35-32 36-33 38-34 39-35 37-36 38-37 42-38 43-42 24-43 25-44 26-45 27-46 27-47 44-47 45-48 46-48 ............
I have moved biconcor and the web directory to the root of my apache web server, have I missed something?
Thanks in advance
Alex.?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130909/07919a69/attachment-0001.htm
------------------------------
Message: 2
Date: Mon, 9 Sep 2013 13:24:00 -0700
From: rohit dholakia <rdholaki@sfu.ca>
Subject: [Moses-support] Reading phrase table taking very long
To: moses-support@mit.edu
Message-ID:
<CAA==LgvEfKCvabqQvZyavPcLe2czMF+awGbTkY2PmvTYPkdoRg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi
I have recently managed to install moses from the master branch on
github. The file format has changed. The training and mert has gone
without incident. But, i have observed that reading the phrase table is
taking very very long compared to the previous moses. From mert.out and
from translating test data log,
*Loading table into memory...done.*
*Start loading text SCFG phrase table. Moses format : [51.000] seconds*
*Reading /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz*
*
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
*
Now, this step, this step is taking really long. Am I the only one who is
observing this ? My phrase table is trained on only 110K sentences, quite
noisy ones at that. Has only 4.8 million rules.
*[rd@seawolf1 TestingMoses]$ zcat train/model/phrase-table.gz | wc -l*
*4838810*
To explain the time taken, its been more than 5 minutes and not even a
single star has come up below that last line ( ---- 5 --- 10 ) .
Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130909/cb28f59c/attachment-0001.htm
------------------------------
Message: 3
Date: Mon, 9 Sep 2013 13:34:48 -0700
From: rohit dholakia <rdholaki@sfu.ca>
Subject: Re: [Moses-support] Reading phrase table taking very long
To: moses-support@mit.edu
Message-ID:
<CAA==LgtoPs2ryph5VpNhcSy7C=woh8MofhNFw-iSYXKZLO4qGg@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi,
Alright, my phrase table is just 100MB compressed. And after taking quite
long to read the phrase table, i got an exception bad_alloc. I have quite a
bit of RAM, so, i am not sure what I am doing wrong .
Start loading text SCFG phrase table. Moses format : [51.000] seconds
Reading /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Exception: std::bad_alloc
Are the changes in the master branch mostly related to an updated moses
file format ? Are there any changes in the algorithms(mert/ibm models)
that warrant a re-run of tests done using the previous version ?
On Mon, Sep 9, 2013 at 1:24 PM, rohit dholakia <rdholaki@sfu.ca> wrote:
> Hi
>
> I have recently managed to install moses from the master branch on
> github. The file format has changed. The training and mert has gone
> without incident. But, i have observed that reading the phrase table is
> taking very very long compared to the previous moses. From mert.out and
> from translating test data log,
>
> *Loading table into memory...done.*
> *Start loading text SCFG phrase table. Moses format : [51.000] seconds*
> *Reading
> /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz*
> *
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> *
>
>
> Now, this step, this step is taking really long. Am I the only one who is
> observing this ? My phrase table is trained on only 110K sentences, quite
> noisy ones at that. Has only 4.8 million rules.
>
> *[rd@seawolf1 TestingMoses]$ zcat train/model/phrase-table.gz | wc -l*
> *4838810*
>
> To explain the time taken, its been more than 5 minutes and not even a
> single star has come up below that last line ( ---- 5 --- 10 ) .
>
> Thanks.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130909/71990d4f/attachment-0001.htm
------------------------------
Message: 4
Date: Mon, 09 Sep 2013 23:10:06 +0200
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Reading phrase table taking very long
To: moses-support@mit.edu
Message-ID: <522E392E.2010109@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
you're running out of memory. Binarize your phrase table
On 09/09/2013 22:34, rohit dholakia wrote:
> Hi,
>
> Alright, my phrase table is just 100MB compressed. And after taking
> quite long to read the phrase table, i got an exception bad_alloc. I
> have quite a bit of RAM, so, i am not sure what I am doing wrong .
>
> Start loading text SCFG phrase table. Moses format : [51.000] seconds
> Reading /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> ****************************************************************************************************
> Exception: std::bad_alloc
>
> Are the changes in the master branch mostly related to an updated
> moses file format ? Are there any changes in the algorithms(mert/ibm
> models) that warrant a re-run of tests done using the previous version ?
>
>
>
>
> On Mon, Sep 9, 2013 at 1:24 PM, rohit dholakia <rdholaki@sfu.ca
> <mailto:rdholaki@sfu.ca>> wrote:
>
> Hi
>
> I have recently managed to install moses from the master branch
> on github. The file format has changed. The training and mert has
> gone without incident. But, i have observed that reading the
> phrase table is taking very very long compared to the previous
> moses. From mert.out and from translating test data log,
>
> /Loading table into memory...done./
> /Start loading text SCFG phrase table. Moses format : [51.000]
> seconds/
> /Reading
> /global/scratch/rd/Thesis/TestingMoses/train/model/phrase-table.gz/
> /----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100/
>
>
> Now, this step, this step is taking really long. Am I the only one
> who is observing this ? My phrase table is trained on only 110K
> sentences, quite noisy ones at that. Has only 4.8 million rules.
>
> /[rd@seawolf1 TestingMoses]$ zcat train/model/phrase-table.gz | wc -l/
> /4838810/
>
> To explain the time taken, its been more than 5 minutes and not
> even a single star has come up below that last line ( ---- 5 ---
> 10 ) .
>
> Thanks.
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130909/8efe5c40/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 83, Issue 19
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 83, Issue 19"
Post a Comment