Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: MGIZA is slower than GIZA (hxshi)
2. Re: MGIZA is slower than GIZA (Christophe Servan)
----------------------------------------------------------------------
Message: 1
Date: Thu, 22 Jan 2015 00:29:35 +0800
From: hxshi <hxshi@mtlab.hit.edu.cn>
Subject: Re: [Moses-support] MGIZA is slower than GIZA
To: "Hieu Hoang" <hieuhoang@gmail.com>, xiangli <xiangli@me.com>,
moses-support <moses-support@mit.edu>
Message-ID: <2015012200293474078431@mtlab.hit.edu.cn>
Content-Type: text/plain; charset="utf-8"
When I try to cmake the new mgiza, it appears errors as following:
Could you please tell me what maybe the reason?
cmake .
-- The C compiler identification is GNU 4.4.7
-- The CXX compiler identification is GNU 4.4.7
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- You have not set the install dir, default to './inst', if
you want to set it, use cmake -DCMAKE_INSTALL_PREFIX to do so
-- Performing Test TR1_SHARED_PTR_USE_TR1_MEMORY
-- Performing Test TR1_SHARED_PTR_USE_TR1_MEMORY - Success
-- Performing Test TR1_SHARED_PTR_USE_MEMORY
-- Performing Test TR1_SHARED_PTR_USE_MEMORY - Failed
-- Performing Test TR1_UNORDERED_MAP_USE_TR1_UNORDERED_MAP
-- Performing Test TR1_UNORDERED_MAP_USE_TR1_UNORDERED_MAP - Success
-- Performing Test TR1_UNORDERED_MAP_USE_UNORDERED_MAP
-- Performing Test TR1_UNORDERED_MAP_USE_UNORDERED_MAP - Failed
CMake Error at /usr/lib64/boost/Boost.cmake:536 (message):
The imported target "boost_date_time-static-debug" references the file
"/usr/lib64/lib64/libboost_date_time-d.a"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/usr/lib64/boost/Boost.cmake"
but not all the files it references.
Call Stack (most recent call first):
/usr/lib64/boost/BoostConfig.cmake:28 (include)
/usr/share/cmake/Modules/FindBoost.cmake:177 (find_package)
CMakeLists.txt:55 (FIND_PACKAGE)
Shi Huaxing
MI&T Lab
School of Computer Science and Technology
Harbin Institute of Technology
From: Hieu Hoang
Date: 2015-01-21 22:39
To: xiangli; moses-support
Subject: Re: [Moses-support] MGIZA is slower than GIZA
it turns out that the build system for mgiza has changed from automake to cmake a few years ago but the compile flags wasn't properly set in the new cmake build. Rico and I have just added the flags to cmake and the manual-compile.sh script.
https://github.com/moses-smt/mgiza/commit/403a56ebc03da4970905195cbd5b4decf58c6686
https://github.com/moses-smt/mgiza/commit/a42e5541b2e321d4cca13f86b3f12141190e7384
This makes mgiza x4 faster:
Before:
420.76user 13.63system 1:42.12elapsed 425%CPU (0avgtext+0avgdata 181616maxresident)k
After:
101.93user 9.34system 0:45.92elapsed 242%CPU (0avgtext+0avgdata 181184maxresident)k
On 19/01/15 19:52, Hieu Hoang wrote:
Hi Li
You're absolutely right, mgiza has gotten slower than giza++! I have mgiza from 2 years ago which is x2 faster on 3 cores, but now it's x2 slower.
Currently rolling back to find the offending commit. Will get back to you when I find it
These are the timings:
CURRENT MGIZA
1. 25722.74user 904.54system 1:26:41elapsed 511%CPU (0avgtext+0avgdata 1906128maxresident)k
2. 24095.06user 978.64system 1:20:57elapsed 516%CPU (0avgtext+0avgdata 1906176maxresident)k
GIZA++
4902.41user 21.95system 43:54.45elapsed 186%CPU (0avgtext+0avgdata 1906144maxresident)k
OLD MGIZA
6576.71user 570.62system 24:09.90elapsed 492%CPU (0avgtext+0avgdata 1906144maxresident)k
On 17/01/15 08:41, Li Xiang wrote:
Hi,
GIZA:
${mosesScript}/training/train-model.perl \
--external-bin-dir "${binDir}" \
--root-dir "${trainDir}" \
--corpus train \
--f src \
--e ref \
--alignment grow-diag-final-and \
--parallel \
--first-step 1 \
--last-step 3
MGIZA
${mosesScript}/training/train-model.perl \
--external-bin-dir "${binDir}" \
--root-dir "${trainDir}" \
--corpus train \
--f src \
--e ref \
--alignment grow-diag-final-and \
--parallel \
--first-step 1 \
--last-step 3 \
--mgiza --mgiza-cpus 3
? 2015?1?17??16:39?Hieu Hoang <Hieu.Hoang@ed.ac.uk> ???
ok, can u tell me what u ran for giza++ and mgiza
On 17 January 2015 at 08:29, Li Xiang <xiangli@me.com> wrote:
Hi Hieu,
I give you 5K training data for evaluate the performance. And I get similar result that mgiza is slower than giza on the data.
? 2015?1?17??00:34?Hieu Hoang <Hieu.Hoang@ed.ac.uk> ???
can you provide the training corpus so I can verify your results?
On 16 January 2015 at 15:53, Li Xiang <lixiang.ict@gmail.com> wrote:
Hi all,
I trained the alignment model on the same data with the same parameters using GIZA and MGIZA respectively. The training corpus includes 200K sentences. My server has an Intel Quad CPU i4790K which has 4 cores and each core has 2 threads. It costs 2905 seconds for GIZA. But it costs 5259 seconds for MGIZA with 3 threads. I think MGIZA is much faster than GIZA. But I got bad result. I do not know the reason is the compile way or others.
Does anyone has relative experience? Thanks.
The following is the training command for MGIZA. And the training data is the FBIS zh-en data. But I can not public the data because of copyright.
${mosesScript}/training/train-model.perl \
--external-bin-dir "${binDir}" \
--root-dir "${trainDir}" \
--corpus train \
--f src \
--e ref \
--alignment grow-diag-final-and \
--parallel \
--first-step 1 \
--last-step 3 \
--mgiza --mgiza-cpus 3
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150122/550f3bdf/attachment-0001.htm
------------------------------
Message: 2
Date: Wed, 21 Jan 2015 17:56:10 +0100
From: Christophe Servan <christophe.servan@gmail.com>
Subject: Re: [Moses-support] MGIZA is slower than GIZA
To: hxshi <hxshi@mtlab.hit.edu.cn>, Christophe Servan
<christophe.servan@gmail.com>
Cc: moses-support <moses-support@mit.edu>, xiangli <xiangli@me.com>
Message-ID:
<CAAsGDkrA9J1L_ouR91a-YX=WccZMAeCqFu_vUOXTyyCqHjZK9w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
I had the same problem, I solve it by setting the right version of boost
(1.55 for me) in the ROOT_DIR/CMakeLists.txt.
To do this I replace this line:
FIND_PACKAGE( Boost 1.44 COMPONENTS thread system)
by this line:
FIND_PACKAGE( Boost 1.55 COMPONENTS thread system)
Cheers,
Christophe
2015-01-21 17:29 GMT+01:00 hxshi <hxshi@mtlab.hit.edu.cn>:
> When I try to cmake the new mgiza, it appears errors as following:
> Could you please tell me what maybe the reason?
> cmake .
>
> -- The C compiler identification is GNU 4.4.7
> -- The CXX compiler identification is GNU 4.4.7
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- You have not set the install dir, default to './inst', if
> you want to set it, use cmake -DCMAKE_INSTALL_PREFIX to do so
> -- Performing Test TR1_SHARED_PTR_USE_TR1_MEMORY
> -- Performing Test TR1_SHARED_PTR_USE_TR1_MEMORY - Success
> -- Performing Test TR1_SHARED_PTR_USE_MEMORY
> -- Performing Test TR1_SHARED_PTR_USE_MEMORY - Failed
> -- Performing Test TR1_UNORDERED_MAP_USE_TR1_UNORDERED_MAP
> -- Performing Test TR1_UNORDERED_MAP_USE_TR1_UNORDERED_MAP - Success
> -- Performing Test TR1_UNORDERED_MAP_USE_UNORDERED_MAP
> -- Performing Test TR1_UNORDERED_MAP_USE_UNORDERED_MAP - Failed
> CMake Error at /usr/lib64/boost/Boost.cmake:536 (message):
> The imported target "boost_date_time-static-debug" references the file
>
> "/usr/lib64/lib64/libboost_date_time-d.a"
>
> but this file does not exist. Possible reasons include:
>
> * The file was deleted, renamed, or moved to another location.
>
> * An install or uninstall procedure did not complete successfully.
>
> * The installation package was faulty and contained
>
> "/usr/lib64/boost/Boost.cmake"
>
> but not all the files it references.
>
> Call Stack (most recent call first):
> /usr/lib64/boost/BoostConfig.cmake:28 (include)
> /usr/share/cmake/Modules/FindBoost.cmake:177 (find_package)
> CMakeLists.txt:55 (FIND_PACKAGE)
>
>
>
> ------------------------------
>
> Shi Huaxing
>
> MI&T Lab
> School of Computer Science and Technology
> Harbin Institute of Technology
>
> *From:* Hieu Hoang <hieuhoang@gmail.com>
> *Date:* 2015-01-21 22:39
> *To:* xiangli <xiangli@me.com>; moses-support <moses-support@mit.edu>
> *Subject:* Re: [Moses-support] MGIZA is slower than GIZA
> it turns out that the build system for mgiza has changed from automake
> to cmake a few years ago but the compile flags wasn't properly set in the
> new cmake build. Rico and I have just added the flags to cmake and the
> manual-compile.sh script.
>
> https://github.com/moses-smt/mgiza/commit/403a56ebc03da4970905195cbd5b4decf58c6686
>
> https://github.com/moses-smt/mgiza/commit/a42e5541b2e321d4cca13f86b3f12141190e7384
>
> This makes mgiza x4 faster:
>
> Before:
> 420.76user 13.63system 1:42.12elapsed 425%CPU (0avgtext+0avgdata
> 181616maxresident)k
>
> After:
> 101.93user 9.34system 0:45.92elapsed 242%CPU (0avgtext+0avgdata
> 181184maxresident)k
>
> On 19/01/15 19:52, Hieu Hoang wrote:
>
>
> Hi Li
>
> You're absolutely right, mgiza has gotten slower than giza++! I have mgiza
> from 2 years ago which is x2 faster on 3 cores, but now it's x2 slower.
>
> Currently rolling back to find the offending commit. Will get back to you
> when I find it
>
> These are the timings:
> *CURRENT MGIZA*
> 1. 25722.74user 904.54system 1:26:41elapsed 511%CPU (0avgtext+0avgdata
> 1906128maxresident)k
> 2. 24095.06user 978.64system 1:20:57elapsed 516%CPU (0avgtext+0avgdata
> 1906176maxresident)k
>
> *GIZA++*
> 4902.41user 21.95system 43:54.45elapsed 186%CPU (0avgtext+0avgdata
> 1906144maxresident)k
>
>
> *OLD **MGIZA*
> 6576.71user 570.62system 24:09.90elapsed 492%CPU (0avgtext+0avgdata
> 1906144maxresident)k
>
>
> On 17/01/15 08:41, Li Xiang wrote:
>
> Hi,
>
> GIZA:
>
> ${mosesScript}/training/train-model.perl \
> --external-bin-dir "${binDir}" \
> --root-dir "${trainDir}" \
> --corpus train \
> --f src \
> --e ref \
> --alignment grow-diag-final-and \
> --parallel \
> --first-step 1 \
> --last-step 3
>
> MGIZA
>
> ${mosesScript}/training/train-model.perl \
> --external-bin-dir "${binDir}" \
> --root-dir "${trainDir}" \
> --corpus train \
> --f src \
> --e ref \
> --alignment grow-diag-final-and \
> --parallel \
> --first-step 1 \
> --last-step 3 \
> --mgiza --mgiza-cpus 3
>
>
>
> ? 2015?1?17??16:39?Hieu Hoang <Hieu.Hoang@ed.ac.uk> ???
>
> ok, can u tell me what u ran for giza++ and mgiza
>
> On 17 January 2015 at 08:29, Li Xiang <xiangli@me.com> wrote:
>
>> Hi Hieu,
>>
>> I give you 5K training data for evaluate the performance. And I get
>> similar result that mgiza is slower than giza on the data.
>>
>>
>> ? 2015?1?17??00:34?Hieu Hoang <Hieu.Hoang@ed.ac.uk> ???
>>
>> can you provide the training corpus so I can verify your results?
>>
>> On 16 January 2015 at 15:53, Li Xiang <lixiang.ict@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I trained the alignment model on the same data with the same parameters
>>> using GIZA and MGIZA respectively. The training corpus includes 200K
>>> sentences. My server has an Intel Quad CPU i4790K which has 4 cores and
>>> each core has 2 threads. It costs 2905 seconds for GIZA. But it costs 5259
>>> seconds for MGIZA with 3 threads. I think MGIZA is much faster than GIZA.
>>> But I got bad result. I do not know the reason is the compile way or others.
>>>
>>> Does anyone has relative experience? Thanks.
>>>
>>> The following is the training command for MGIZA. And the training data
>>> is the FBIS zh-en data. But I can not public the data because of copyright.
>>>
>>>
>>> ${mosesScript}/training/train-model.perl \
>>> --external-bin-dir "${binDir}" \
>>> --root-dir "${trainDir}" \
>>> --corpus train \
>>> --f src \
>>> --e ref \
>>> --alignment grow-diag-final-and \
>>> --parallel \
>>> --first-step 1 \
>>> --last-step 3 \
>>> --mgiza --mgiza-cpus 3
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150121/72557075/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 99, Issue 48
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 99, Issue 48"
Post a Comment