Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Need help in resolving an error in moses installation
(Hieu Hoang)
2. Re: Moses in server mode (Hieu Hoang)
3. Re: Moses training performance (Andrzej Zydron)
4. Re: Moses training performance (Marcin Junczys-Dowmunt)
5. Re: Moses training performance (Barry Haddow)
----------------------------------------------------------------------
Message: 1
Date: Thu, 27 Feb 2014 19:12:34 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Need help in resolving an error in moses
installation
To: Sampada Dehankar <dehankarsampada1510@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhi71JVW9yEbbLteUA1gd10tB6GnSiFgCbXwdDYd5wLZA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
If you are new to Moses, please follow this webpage.
http://www.statmt.org/moses/?n=Development.GetStarted
It teaches you step-by-step how to compile Moses
On 26 February 2014 17:53, Sampada Dehankar
<dehankarsampada1510@gmail.com>wrote:
> EXACT command executed :
>
> ./bjam
> --with-srilm=/home/sampada/Documents/Semester_2/CS_712_ADV_NLP/moses1/srilm
> --with-giza=/home/sampada/Documents/Semester_2/CS_712_ADV_NLP/moses1/bin
>
>
> Please find attached build.log.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140227/34a758d7/attachment-0001.htm
------------------------------
Message: 2
Date: Thu, 27 Feb 2014 19:17:08 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Moses in server mode
To: Sarah Schulz <sarah.schulz@ugent.be>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbizmUOUU3MrM6Zbu3u+nmmYM5nAXhuXaG=nb+MXRNpeEQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Is the problem reproduceable with a particular sentence, and how long is
the sentence? Can you reproduce the error with a different sentence with
the same number of words or characters?
If it's reproduceable, perhaps you can make your files available for
download and I'll try and debug it
On 26 February 2014 08:16, Sarah Schulz <sarah.schulz@ugent.be> wrote:
> Hello,
>
> I am running Moses in server mode using the python lib xmlrpclib. When I
> send long messages to Moses, it crashes. I guess it has to do with a
> limitation of the message size. I could find a possibility to increase
> the allowed message size for the XML-RPC implementation for C/C++ (which
> shows me that there is something like a maximum length) but not for the
> python implementation.
>
> That is the error I get:
>
> File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
> return self.__send(self.__name, args)
> File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
> verbose=self.__verbose
> File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
> return self.single_request(host, handler, request_body, verbose)
> File "/usr/lib/python2.7/xmlrpclib.py", line 1294, in single_request
> response = h.getresponse(buffering=True)
> File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
> response.begin()
> File "/usr/lib/python2.7/httplib.py", line 407, in begin
> version, status, reason = self._read_status()
> File "/usr/lib/python2.7/httplib.py", line 371, in _read_status
> raise BadStatusLine(line)
> httplib.BadStatusLine: ''
>
>
> Does anybody have experience with that?
>
> Cheers,
>
> Sarah
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140227/1d5add4f/attachment-0001.htm
------------------------------
Message: 3
Date: Thu, 27 Feb 2014 20:05:12 +0000
From: Andrzej Zydron <azydron@xtm-intl.com>
Subject: Re: [Moses-support] Moses training performance
To: undisclosed-recipients:;
Cc: moses-support <moses-support@mit.edu>
Message-ID: <530F9A78.8030409@xtm-intl.com>
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
Hi Hieu, Barry and Marcin,
Thank you for your replies and suggestions.
The Xeon server is completely dedicated to Moses and is running
absolutely nothing else, as opposed to my Mac which is running the usual
laptop background stuff like mail etc., as well as having Eclipse doing
various Java stuff in the background.
I re-ran the tests as Barry advised with only 4 cores and the results were
training: 41:56
tuning: 28:16
decoding: 01:36
Total 01:08:17
Therefore 18 minutes slower than the best time on the Xeon with 6 cores
(50:01 minutes).
Regarding Marcin's suggestion, here are the individual moses-training
process' timings:
MacBook Pro 4 Threads i7 3720QM 8Gb RAM SSD
mkls 17:39:16 17:39:49 00:00:33
snt2cooc.out 17:39:50 17:39:52 00:00:02
mgiza 17:39:52 17:45:47 00:05:55
snt2cooc.out 17:45:47 17:45:50 00:00:03
mgiza 17:45:50 17:53:20 00:07:30
giza2bal.pl 17:53:21 17:53:23 00:00:02
extract 17:53:25 17:53:31 00:00:06
score 17:53:31 17:53:41 00:00:10
lexical-reordering 17:53:41 17:53:45 00:00:04
Total 00:14:25
Using 6 Threads Xeon E5-1650v2 128GB RAM SATA using 28GB RAMDISK
mkls 19:31:07 19:35:43 00:04:36
snt2cooc.out 19:35:43 19:36:05 00:00:22
mgiza 19:36:05 19:49:47 00:13:42
snt2cooc.out 19:49:47 19:50:12 00:00:25
mgiza 19:50:12 20:04:09 00:13:57
giza2bal.pl 20:04:09 20:04:31 00:00:22
extract 20:04:31 20:05:19 00:00:48
score 20:05:19 20:07:00 00:01:41
lexical-reordering 20:07:00 20:07:01 00:00:01
Total 00:35:54
As you can see the culprit are mkls and mgiza.
Email signature standard
Best Regards,
Andrzej Zydro?
---------------------------------------
CTO
*XTM International Ltd.*
PO Box 2167, Gerrards Cross, SL9 8XF, UK
email: azydron@xtm-intl.com <mailto:azydron@xtm-intl.com>
Tel: +44 (0) 1753 480 479
Mob: +44 (0) 7966 477 181
skype: Zydron
www.xtm-intl.com <http://www.xtm-intl.com/>
------------------------------
Message: 4
Date: Thu, 27 Feb 2014 21:18:43 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Moses training performance
To: moses-support@mit.edu
Message-ID: <530F9DA3.80903@amu.edu.pl>
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
Are you absolutely sure that you are using exactly the same training
data on both machines and that nothing is being truncated in the
process? Are the resulting phrase tables on both machines of the same size?
W dniu 27.02.2014 21:05, Andrzej Zydron pisze:
> Hi Hieu, Barry and Marcin,
>
> Thank you for your replies and suggestions.
>
> The Xeon server is completely dedicated to Moses and is running
> absolutely nothing else, as opposed to my Mac which is running the usual
> laptop background stuff like mail etc., as well as having Eclipse doing
> various Java stuff in the background.
>
> I re-ran the tests as Barry advised with only 4 cores and the results were
>
> training: 41:56
> tuning: 28:16
> decoding: 01:36
>
> Total 01:08:17
>
> Therefore 18 minutes slower than the best time on the Xeon with 6 cores
> (50:01 minutes).
>
> Regarding Marcin's suggestion, here are the individual moses-training
> process' timings:
>
> MacBook Pro 4 Threads i7 3720QM 8Gb RAM SSD
> mkls 17:39:16 17:39:49 00:00:33
> snt2cooc.out 17:39:50 17:39:52 00:00:02
> mgiza 17:39:52 17:45:47 00:05:55
> snt2cooc.out 17:45:47 17:45:50 00:00:03
> mgiza 17:45:50 17:53:20 00:07:30
> giza2bal.pl 17:53:21 17:53:23 00:00:02
> extract 17:53:25 17:53:31 00:00:06
> score 17:53:31 17:53:41 00:00:10
> lexical-reordering 17:53:41 17:53:45 00:00:04
>
> Total 00:14:25
>
> Using 6 Threads Xeon E5-1650v2 128GB RAM SATA using 28GB RAMDISK
> mkls 19:31:07 19:35:43 00:04:36
> snt2cooc.out 19:35:43 19:36:05 00:00:22
> mgiza 19:36:05 19:49:47 00:13:42
> snt2cooc.out 19:49:47 19:50:12 00:00:25
> mgiza 19:50:12 20:04:09 00:13:57
> giza2bal.pl 20:04:09 20:04:31 00:00:22
> extract 20:04:31 20:05:19 00:00:48
> score 20:05:19 20:07:00 00:01:41
> lexical-reordering 20:07:00 20:07:01 00:00:01
>
> Total 00:35:54
>
> As you can see the culprit are mkls and mgiza.
>
> Email signature standard
>
> Best Regards,
>
>
> Andrzej Zydro?
>
> ---------------------------------------
>
> CTO
>
> *XTM International Ltd.*
>
> PO Box 2167, Gerrards Cross, SL9 8XF, UK
>
> email: azydron@xtm-intl.com <mailto:azydron@xtm-intl.com>
>
> Tel: +44 (0) 1753 480 479
>
> Mob: +44 (0) 7966 477 181
>
> skype: Zydron
>
> www.xtm-intl.com <http://www.xtm-intl.com/>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
------------------------------
Message: 5
Date: Thu, 27 Feb 2014 20:43:19 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Moses training performance
To: Andrzej Zydron <azydron@xtm-intl.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <530FA367.7020008@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
Hi Andrzej
Whilst mgiza is the time hog in the training, I find it surprising that
score takes 10 seconds on the mac and nearly 2 minutes on the xeon. Most
of its work is sorting and reading and writing compressed files. I
wonder if there is some difference in the sort? Is it using disk on the
xeon, and doing everything in ram on the mac? Is it using a temporary
directory outside the ram disk - although I think it should put its tmp
directory inside the Moses training directory.
cheers - Barry
On 27/02/14 20:05, Andrzej Zydron wrote:
> Hi Hieu, Barry and Marcin,
>
> Thank you for your replies and suggestions.
>
> The Xeon server is completely dedicated to Moses and is running
> absolutely nothing else, as opposed to my Mac which is running the usual
> laptop background stuff like mail etc., as well as having Eclipse doing
> various Java stuff in the background.
>
> I re-ran the tests as Barry advised with only 4 cores and the results were
>
> training: 41:56
> tuning: 28:16
> decoding: 01:36
>
> Total 01:08:17
>
> Therefore 18 minutes slower than the best time on the Xeon with 6 cores
> (50:01 minutes).
>
> Regarding Marcin's suggestion, here are the individual moses-training
> process' timings:
>
> MacBook Pro 4 Threads i7 3720QM 8Gb RAM SSD
> mkls 17:39:16 17:39:49 00:00:33
> snt2cooc.out 17:39:50 17:39:52 00:00:02
> mgiza 17:39:52 17:45:47 00:05:55
> snt2cooc.out 17:45:47 17:45:50 00:00:03
> mgiza 17:45:50 17:53:20 00:07:30
> giza2bal.pl 17:53:21 17:53:23 00:00:02
> extract 17:53:25 17:53:31 00:00:06
> score 17:53:31 17:53:41 00:00:10
> lexical-reordering 17:53:41 17:53:45 00:00:04
>
> Total 00:14:25
>
> Using 6 Threads Xeon E5-1650v2 128GB RAM SATA using 28GB RAMDISK
> mkls 19:31:07 19:35:43 00:04:36
> snt2cooc.out 19:35:43 19:36:05 00:00:22
> mgiza 19:36:05 19:49:47 00:13:42
> snt2cooc.out 19:49:47 19:50:12 00:00:25
> mgiza 19:50:12 20:04:09 00:13:57
> giza2bal.pl 20:04:09 20:04:31 00:00:22
> extract 20:04:31 20:05:19 00:00:48
> score 20:05:19 20:07:00 00:01:41
> lexical-reordering 20:07:00 20:07:01 00:00:01
>
> Total 00:35:54
>
> As you can see the culprit are mkls and mgiza.
>
> Email signature standard
>
> Best Regards,
>
>
> Andrzej Zydro?
>
> ---------------------------------------
>
> CTO
>
> *XTM International Ltd.*
>
> PO Box 2167, Gerrards Cross, SL9 8XF, UK
>
> email: azydron@xtm-intl.com <mailto:azydron@xtm-intl.com>
>
> Tel: +44 (0) 1753 480 479
>
> Mob: +44 (0) 7966 477 181
>
> skype: Zydron
>
> www.xtm-intl.com <http://www.xtm-intl.com/>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 88, Issue 65
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 88, Issue 65"
Post a Comment