Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Building Moses with statically linked libraries?
(Lane Schwartz)
----------------------------------------------------------------------
Message: 1
Date: Mon, 26 Aug 2013 16:59:07 -0400
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] Building Moses with statically linked
libraries?
To: Kenneth Heafield <moses@kheafield.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZmxBc=SYKMJKyzV_BQy23ZXE1EJwfrze3Zp2+OGyQ-3VQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I'm attempting to compile Moses in such a way that at least the boost
libraries are statically compiled. I'm fine if other libraries are shared.
My interpretation of this thread (and BUILD-INSTRUCTIONS.txt) is that if I
compile boost (following instructions in BUILD-INSTRUCTIONS.txt) so that it
makes available static versions of its libraries, then compile moses using
link=static, boost should be statically linked with moses. Unfortunately,
that doesn't appear to be happening for me:
./bjam link=static -j8 --with-cmph=/tools/moses/cmph-2.0
--with-srilm=/tools/SRILM/SRILM-1.7.0
--with-boost=/tools/moses/boost_1_53_0 -q --debug-configuration -d2
The log for the above is attached. If I try running moses after this
compile (without adding /tools/moses/boost_1_53_0 to LIBRARY_PATH and
LD_LIBRARY_PATH), I get the following error:
./bin/moses: error while loading shared libraries:
libboost_system-mt.so.1.53.0: cannot open shared object file: No such file
or directory
Running ldd on bin/moses confirms that there is a dynamic boost library
linking:
linux-vdso.so.1 => (0x00007fff8adff000)
libz.so.1 => /lib64/libz.so.1 (0x0000003949200000)
librt.so.1 => /lib64/librt.so.1 (0x0000003949600000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003955a00000)
libboost_system-mt.so.1.53.0 => not found
libSegFault.so => /lib64/libSegFault.so (0x00007f1671726000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000394fa00000)
libm.so.6 => /lib64/libm.so.6 (0x0000003948600000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000394d600000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003948e00000)
libc.so.6 => /lib64/libc.so.6 (0x0000003948200000)
/lib64/ld-linux-x86-64.so.2 (0x0000003947e00000)
Looking in /tools/moses/boost_1_53_0/lib I can verify that the static .a
file exists:
libboost_system-mt.a
libboost_system-mt.so
libboost_system-mt.so.1.53.0
Any help would be appreciated. I'm attaching the compile log and a listing
of the boost files installed in /tools/moses/boost_1_53_0/lib.
Thanks,
Lane
On Mon, Aug 6, 2012 at 12:39 PM, Kenneth Heafield <moses@kheafield.com>wrote:
> Ok, committed. Here's how the build system now behaves:
>
> link=shared: Everything linked dynamically.
>
> Default: internal libraries are statically linked. Boost and zlib
> statically linked if possible. libSegFault dynamically linked.
> Dynamically linked executable.
>
> --without-libsegfault: Same as default but no libSegFault. Still a
> dynamically linked executable, even if you have static boost and zlib.
> It's complicated to do detect then be automatic.
>
> --static: No libSegFault. Print warning messages if you're missing
> static libraries, but keep building anyway. Static executable.
>
> Kenneth
>
> On 08/06/2012 12:00 PM, Kenneth Heafield wrote:
> > D'oh, it's a feature, not a bug. Add runtime-link=static and you'll get
> > a fully-static executable.
> >
> > Plan to add this to the Moses build system. Testing now.
> >
> > Kenneth
> >
> > On 08/06/2012 11:31 AM, Tom Hoar wrote:
> >> FYI, this build is on U-10.04.4 using the bjam shipped with Moses. I can
> >> send you the complete log if it helps. Let me know if you need anything
> >> else.
> >>
> >> Tom
> >>
> >>
> >> On Mon, 06 Aug 2012 11:27:27 -0400, Kenneth Heafield
> >> <moses@kheafield.com> wrote:
> >>> Hi,
> >>>
> >>> There appears to be a bug in boost-build that prevents it from making
> >>> fully-static binaries with g++. This might take a while to sort out.
> >>>
> >>> Kenneth
> >>>
> >>> On 08/06/2012 10:48 AM, Tom Hoar wrote:
> >>>> Thanks, Ken. I think this is easier than adding another option to
> Moses
> >>>> if we can understand the output. The ldd manpage says "ldd prints the
> >>>> shared libraries required by each program or shared library specified
> on
> >>>> the command line." I assume "shared libraries" means not statically
> >>>> linked?
> >>>>
> >>>> Here's the output on the moses I just built with today's github
> updates.
> >>>> The log output reported exit code 0 for all lines with "g++ -static".
> >>>>
> >>>> tahoar@library1:~$ ldd `which moses`
> >>>> linux-vdso.so.1 => (0x00007fffa2954000)
> >>>> librt.so.1 => /lib/librt.so.1 (0x00007f818aa7b000)
> >>>> libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f818a767000)
> >>>> libm.so.6 => /lib/libm.so.6 (0x00007f818a4e3000)
> >>>> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f818a2cc000)
> >>>> libpthread.so.0 => /lib/libpthread.so.0 (0x00007f818a0af000)
> >>>> libc.so.6 => /lib/libc.so.6 (0x00007f8189d2b000)
> >>>> /lib64/ld-linux-x86-64.so.2 (0x00007f818aca7000)
> >>>>
> >>>> Is this what I should expect if all of the libboost libraries are
> >>>> statically linked? I think so because there are no references to the
> >>>> lboost_* or lz libraries in the log file.
> >>>>
> >>>> Tom
> >>>>
> >>>>
> >>>> On Mon, 06 Aug 2012 10:14:35 -0400, Kenneth Heafield
> >>>> <moses@kheafield.com> wrote:
> >>>>> On linux,
> >>>>>
> >>>>> ldd bin/moses
> >>>>>
> >>>>> On 08/06/2012 10:03 AM, Tom Hoar wrote:
> >>>>>> Thanks Ken. I downloaded/compiled with the latest changes up to
> >>>>>> BOOST_CHECK_CLOSE. The exit code 1 disappeared and everything seems
> >>>>>> okay.
> >>>>>>
> >>>>>> One more question. Other than the log output at compile time, is
> there
> >>>>>> any way to query the Moses binary to see which libraries are
> >>>>>> statically
> >>>>>> vs dynamically linked?
> >>>>>>
> >>>>>> For example, a while back, someone on the list gave this command to
> >>>>>> test
> >>>>>> if the binary was compiled --with-srilm, and there are others for
> >>>>>> IRSTLM, RANDLM and KENLM.
> >>>>>>
> >>>>>> nm -C "/path/to/moses" | grep "vtable for Moses::LanguageModelSRI"`
> >>>>>>
> >>>>>> I tried the obvious two grep searches "static" and "dynamic"
> (below),
> >>>>>> but they don't seem to relate to the libraries. Does anyone know a
> way
> >>>>>> to find/test if libraries are dynamically or statically linked?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Tom
> >>>>>>
> >>>>>>
> >>>>>> tahoar@library1:~/domy-2.5$ nm -C `which moses` | grep "static"
> >>>>>> 00000000005ddbc0 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005de7c1 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005ded37 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005e0067 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005e2fc6 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005ecae3 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005ef140 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005f475f t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005f4cd9 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005f5e53 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005f6f8e t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 00000000005f7183 t __static_initialization_and_destruction_0(int,
> int)
> >>>>>> 0000000000883cc0 d static_bl_desc
> >>>>>> 0000000000883ca0 d static_d_desc
> >>>>>> 0000000000610680 r static_dtree
> >>>>>> 0000000000883c80 d static_l_desc
> >>>>>> 0000000000610200 r static_ltree
> >>>>>>
> >>>>>> tahoar@library1:~/domy-2.5$ nm -C `which moses` | grep "dynamic"
> >>>>>> 0000000000570c50 W
> >>>>>> boost::unordered_detail::hash_table_unique_keys<std::pair<float
> const,
> >>>>>> boost::dynamic_bitset<unsigned long, std::allocator<unsigned long>
> >>>>>>>> ,
> >>>>>> float, boost::hash<float>, std::equal_to<float>,
> >>>>>> std::allocator<std::pair<float const, boost::dynamic_bitset<unsigned
> >>>>>> long, std::allocator<unsigned long> > > > >::operator[](float
> const&)
> >>>>>> 000000000057baf0 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<std::pair<unsigned
> >>>>>>
> >>>>>>
> >>>>>> char, unsigned char> const, boost::dynamic_bitset<unsigned long,
> >>>>>> std::allocator<unsigned long> > > >
> >::~hash_table_data_unique_keys()
> >>>>>> 0000000000570af0 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long, std::allocator<unsigned
> >>>>>> long> > > > >::create_buckets()
> >>>>>> 0000000000570c00 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long, std::allocator<unsigned
> >>>>>> long> > > > >::node_constructor::~node_constructor()
> >>>>>> 0000000000570b70 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long, std::allocator<unsigned
> >>>>>> long> > > > >::~hash_table_data_unique_keys()
> >>>>>> 000000000057ba60 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<unsigned
> >>>>>>
> >>>>>>
> >>>>>> int const, boost::dynamic_bitset<unsigned long,
> >>>>>> std::allocator<unsigned
> >>>>>> long> > > > >::~hash_table_data_unique_keys()
> >>>>>> U __dynamic_cast@@CXXABI_1.3
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, 06 Aug 2012 08:20:00 -0400, Kenneth Heafield
> >>>>>> <moses@kheafield.com> wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> You're correct. There doesn't seem to be a static version of this
> >>>>>>> library. I've added the --nosegfault option (which isn't as cool as
> >>>>>>> it sounds) to skip this library.
> >>>>>>>
> >>>>>>> Kenneth
> >>>>>>>
> >>>>>>> On 08/06/2012 02:08 AM, Tom Hoar wrote:
> >>>>>>>> I read the comment "In order to obtain a fully static Moses,
> >>>>>>>> every g++
> >>>>>>>> command line that includes "-static" must pass with exit code 0."
> >>>>>>>> with
> >>>>>>>> interest.
> >>>>>>>>
> >>>>>>>> When we compile moses, this log output line shows an exit code 1.
> >>>>>>>> All
> >>>>>>>> the others are 0.
> >>>>>>>>
> >>>>>>>> bash -c "g++ -static -lSegFault -x c++ -<<<'int main() {}' -o
> >>>>>>>> /dev/null
> >>>>>>>>> /dev/null 2>/dev/null"
> >>>>>>>> 1
> >>>>>>>>
> >>>>>>>> Any suggestions as to what's missing/how to correct so we have a
> >>>>>>>> fully-static Moses?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Tom
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, 03 Aug 2012 18:13:47 -0400, Kenneth Heafield
> >>>>>>>> <moses@kheafield.com> wrote:
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Moses attempts to link statically by default but falls back to
> >>>>>>>>> dynamic
> >>>>>>>>> links. You must have static versions of all the dependencies
> >>>>>>>>> installed
> >>>>>>>>> as well. Run
> >>>>>>>>>
> >>>>>>>>> bjam --debug-configuration
> >>>>>>>>>
> >>>>>>>>> and, near the top, it will show you some command lines followed
> by
> >>>>>>>>> their
> >>>>>>>>> exit code. In order to obtain a fully static Moses, every g++
> >>>>>>>>> command
> >>>>>>>>> line that includes "-static" must pass with exit code 0.
> >>>>>>>>>
> >>>>>>>>> Kenneth
> >>>>>>
> >>>>
> >>
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/281029c7/attachment.htm
-------------- next part --------------
libboost_atomic-mt.a
libboost_atomic-mt.so
libboost_atomic-mt.so.1.53.0
libboost_chrono-mt.a
libboost_chrono-mt.so
libboost_chrono-mt.so.1.53.0
libboost_chrono.a
libboost_chrono.so
libboost_chrono.so.1.53.0
libboost_context-mt.a
libboost_context-mt.so
libboost_context-mt.so.1.53.0
libboost_context.a
libboost_context.so
libboost_context.so.1.53.0
libboost_date_time-mt.a
libboost_date_time-mt.so
libboost_date_time-mt.so.1.53.0
libboost_date_time.a
libboost_date_time.so
libboost_date_time.so.1.53.0
libboost_exception-mt.a
libboost_exception.a
libboost_filesystem-mt.a
libboost_filesystem-mt.so
libboost_filesystem-mt.so.1.53.0
libboost_filesystem.a
libboost_filesystem.so
libboost_filesystem.so.1.53.0
libboost_graph-mt.a
libboost_graph-mt.so
libboost_graph-mt.so.1.53.0
libboost_graph.a
libboost_graph.so
libboost_graph.so.1.53.0
libboost_iostreams-mt.a
libboost_iostreams-mt.so
libboost_iostreams-mt.so.1.53.0
libboost_iostreams.a
libboost_iostreams.so
libboost_iostreams.so.1.53.0
libboost_locale-mt.a
libboost_locale-mt.so
libboost_locale-mt.so.1.53.0
libboost_math_c99-mt.a
libboost_math_c99-mt.so
libboost_math_c99-mt.so.1.53.0
libboost_math_c99.a
libboost_math_c99.so
libboost_math_c99.so.1.53.0
libboost_math_c99f-mt.a
libboost_math_c99f-mt.so
libboost_math_c99f-mt.so.1.53.0
libboost_math_c99f.a
libboost_math_c99f.so
libboost_math_c99f.so.1.53.0
libboost_math_c99l-mt.a
libboost_math_c99l-mt.so
libboost_math_c99l-mt.so.1.53.0
libboost_math_c99l.a
libboost_math_c99l.so
libboost_math_c99l.so.1.53.0
libboost_math_tr1-mt.a
libboost_math_tr1-mt.so
libboost_math_tr1-mt.so.1.53.0
libboost_math_tr1.a
libboost_math_tr1.so
libboost_math_tr1.so.1.53.0
libboost_math_tr1f-mt.a
libboost_math_tr1f-mt.so
libboost_math_tr1f-mt.so.1.53.0
libboost_math_tr1f.a
libboost_math_tr1f.so
libboost_math_tr1f.so.1.53.0
libboost_math_tr1l-mt.a
libboost_math_tr1l-mt.so
libboost_math_tr1l-mt.so.1.53.0
libboost_math_tr1l.a
libboost_math_tr1l.so
libboost_math_tr1l.so.1.53.0
libboost_prg_exec_monitor-mt.a
libboost_prg_exec_monitor-mt.so
libboost_prg_exec_monitor-mt.so.1.53.0
libboost_prg_exec_monitor.a
libboost_prg_exec_monitor.so
libboost_prg_exec_monitor.so.1.53.0
libboost_program_options-mt.a
libboost_program_options-mt.so
libboost_program_options-mt.so.1.53.0
libboost_program_options.a
libboost_program_options.so
libboost_program_options.so.1.53.0
libboost_python-mt.a
libboost_python-mt.so
libboost_python-mt.so.1.53.0
libboost_python.a
libboost_python.so
libboost_python.so.1.53.0
libboost_random-mt.a
libboost_random-mt.so
libboost_random-mt.so.1.53.0
libboost_random.a
libboost_random.so
libboost_random.so.1.53.0
libboost_regex-mt.a
libboost_regex-mt.so
libboost_regex-mt.so.1.53.0
libboost_regex.a
libboost_regex.so
libboost_regex.so.1.53.0
libboost_serialization-mt.a
libboost_serialization-mt.so
libboost_serialization-mt.so.1.53.0
libboost_serialization.a
libboost_serialization.so
libboost_serialization.so.1.53.0
libboost_signals-mt.a
libboost_signals-mt.so
libboost_signals-mt.so.1.53.0
libboost_signals.a
libboost_signals.so
libboost_signals.so.1.53.0
libboost_system-mt.a
libboost_system-mt.so
libboost_system-mt.so.1.53.0
libboost_system.a
libboost_system.so
libboost_system.so.1.53.0
libboost_test_exec_monitor-mt.a
libboost_test_exec_monitor.a
libboost_thread-mt.a
libboost_thread-mt.so
libboost_thread-mt.so.1.53.0
libboost_timer-mt.a
libboost_timer-mt.so
libboost_timer-mt.so.1.53.0
libboost_timer.a
libboost_timer.so
libboost_timer.so.1.53.0
libboost_unit_test_framework-mt.a
libboost_unit_test_framework-mt.so
libboost_unit_test_framework-mt.so.1.53.0
libboost_unit_test_framework.a
libboost_unit_test_framework.so
libboost_unit_test_framework.so.1.53.0
libboost_wave-mt.a
libboost_wave-mt.so
libboost_wave-mt.so.1.53.0
libboost_wave.a
libboost_wave.so
libboost_wave.so.1.53.0
libboost_wserialization-mt.a
libboost_wserialization-mt.so
libboost_wserialization-mt.so.1.53.0
libboost_wserialization.a
libboost_wserialization.so
libboost_wserialization.so.1.53.0
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.log.zip
Type: application/zip
Size: 29756 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/281029c7/attachment.zip
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 82, Issue 36
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 82, Issue 36"
Post a Comment