Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Did the configuration files change (Jo?o Gra?a)
2. Re: Building Moses with statically linked libraries?
(Kenneth Heafield)
----------------------------------------------------------------------
Message: 1
Date: Mon, 26 Aug 2013 22:34:30 +0100
From: Jo?o Gra?a <gracaninja@gmail.com>
Subject: Re: [Moses-support] Did the configuration files change
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CAGfH6a6f4wbsmsfmzbRsYTVsY+JZRniMkrqp8VpT81BTDagS9g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Hieu,
I am trying to use the new version of moses with the old format moses.ini
from the pre-trained models.
In attach is the configuration file I am currently trying to use which is a
modified copy from the tunning file taken from
http://www.statmt.org/moses/RELEASE-1.0/models/en-es/tuning/moses.tuned.ini.1
However when I run moses I get this error:
vagrant@precise64:/vagrant/mt-models/en-es$ ~/mosesdecoder/bin/moses -f
moses.ini
Defined parameters (per moses.ini or switch):
: q
config: moses.ini
distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6
/vagrant/mt-models/en-es/reordering-table.1.wbe-msd-bidirectional-fe.gz
distortion-limit: 6
input-factors: 0
lmodel-file: 9 0 5 /vagrant/mt-models/en-es/europarl.binlm.1
mapping: 0 T 0
ttable-file: 0 0 0 5 /vagrant/mt-models/en-es/phrase-table.1
ttable-limit: 20
weight-d: 0.048861 0.0882949 0.0561228 0.0794274 0.152473 -0.00619951
0.0546715
weight-l: 0.0772417
weight-t: 0.0617933 0.0247665 0.0492006 0.0490465 0.0571993
weight-w: -0.194702
ERROR:Unknown parameter
Do you know what I am setting wrong?
Thanks for you help,
Jo?o
On Tue, Aug 6, 2013 at 12:10 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
> The old format.
>
>
> On 6 August 2013 12:07, Jo?o Gra?a <gracaninja@gmail.com> wrote:
>
>> Hi Hieu,
>>
>> Thanks a lot for your help.
>>
>> Which format do the pre-made models use?
>>
>> Thanks,
>>
>> Joao
>>
>>
>> On Tue, Aug 6, 2013 at 12:03 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>>
>>> ah, the moses.ini file format has recently changed. The sample files are
>>> in the new format.
>>>
>>> Older versions of moses don't understand the new format. (New Moses can
>>> understand both old and new format)
>>>
>>> To get the source code for the new Moses version:
>>> git@github.com:moses-smt/mosesdecoder.git
>>>
>>> If you really want to stick with the old moses, the old sample files are
>>> here:
>>> http://www.statmt.org/moses/download/sample-models.old.tgz
>>>
>>> However, the sample models are unrealistically small. These premade
>>> models are more realistic:
>>> http://www.statmt.org/moses/RELEASE-1.0/models/
>>>
>>>
>>> On 6 August 2013 11:42, Jo?o Gra?a <gracaninja@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I am trying to install moses on my system and run it with the sample
>>>> models.
>>>> I am using the moses linux package from
>>>> http://www.statmt.org/~jie/linux/moses_1.0-1_amd64.deb
>>>>
>>>> and then I try to run the sample models available from the web site.
>>>>
>>>> http://www.statmt.org/moses/download/sample-models.tgz
>>>>
>>>> but when i try to run the decode I get the following error:
>>>>
>>>> vagrant@precise64:/vagrant/sample-models$ /opt/moses/bin/moses -f
>>>> phrase-model/moses.ini < phrase-model/in
>>>> Defined parameters (per moses.ini or switch):
>>>> config: phrase-model/moses.ini
>>>> feature: KENLM name=LM factor=0 order=3 num-features=1
>>>> path=lm/europarl.srilm.gz Distortion WordPenalty UnknownWordPenalty
>>>> PhraseDictionaryMemory input-factor=0 output-factor=0
>>>> path=phrase-model/phrase-table num-features=1 table-limit=10
>>>> input-factors: 0
>>>> mapping: T 0
>>>> n-best-list: nbest.txt 100
>>>> weight: WordPenalty0= 0 LM= 1 Distortion0= 1 PhraseDictionaryMemory0=
>>>> 1
>>>> ERROR:Unknown parameter feature
>>>> ERROR:Unknown parameter weight
>>>> ERROR:No phrase translation table (ttable-file)
>>>>
>>>>
>>>> Was there any major changes on the configuration file moses.ini that
>>>> renders the previous releases of moses unusable with the sample models?
>>>>
>>>> Thanks you for all your help,
>>>>
>>>> Jo?o
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Research Associate
>>> University of Edinburgh
>>> http://www.hoang.co.uk/hieu
>>>
>>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/fea1a239/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini
Type: application/octet-stream
Size: 1352 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20130826/fea1a239/attachment-0001.obj
------------------------------
Message: 2
Date: Mon, 26 Aug 2013 23:41:45 +0100
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Building Moses with statically linked
libraries?
To: Lane Schwartz <dowobeha@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <521BD9A9.40005@kheafield.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
Ugh sorry something was weird with bjam. I've put a kludge in that
forces static linkage unless link=shared appears on the command line.
Kenneth
On 08/26/13 21:59, Lane Schwartz wrote:
> I'm attempting to compile Moses in such a way that at least the boost
> libraries are statically compiled. I'm fine if other libraries are shared.
>
> My interpretation of this thread (and BUILD-INSTRUCTIONS.txt) is that if
> I compile boost (following instructions in BUILD-INSTRUCTIONS.txt) so
> that it makes available static versions of its libraries, then compile
> moses using link=static, boost should be statically linked with moses.
> Unfortunately, that doesn't appear to be happening for me:
>
> ./bjam link=static -j8 --with-cmph=/tools/moses/cmph-2.0
> --with-srilm=/tools/SRILM/SRILM-1.7.0
> --with-boost=/tools/moses/boost_1_53_0 -q --debug-configuration -d2
>
> The log for the above is attached. If I try running moses after this
> compile (without adding /tools/moses/boost_1_53_0 to LIBRARY_PATH and
> LD_LIBRARY_PATH), I get the following error:
>
> ./bin/moses: error while loading shared libraries:
> libboost_system-mt.so.1.53.0: cannot open shared object file: No such
> file or directory
>
> Running ldd on bin/moses confirms that there is a dynamic boost library
> linking:
>
> linux-vdso.so.1 => (0x00007fff8adff000)
>
> libz.so.1 => /lib64/libz.so.1 (0x0000003949200000)
>
> librt.so.1 => /lib64/librt.so.1 (0x0000003949600000)
>
> libbz2.so.1 => /lib64/libbz2.so.1 (0x0000003955a00000)
>
> libboost_system-mt.so.1.53.0 => not found
>
> libSegFault.so => /lib64/libSegFault.so (0x00007f1671726000)
>
> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x000000394fa00000)
>
> libm.so.6 => /lib64/libm.so.6 (0x0000003948600000)
>
> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000394d600000)
>
> libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003948e00000)
>
> libc.so.6 => /lib64/libc.so.6 (0x0000003948200000)
>
> /lib64/ld-linux-x86-64.so.2 (0x0000003947e00000)
>
>
> Looking in /tools/moses/boost_1_53_0/lib I can verify that the static .a
> file exists:
>
>
> libboost_system-mt.a
>
> libboost_system-mt.so
>
> libboost_system-mt.so.1.53.0
>
>
> Any help would be appreciated. I'm attaching the compile log and a
> listing of the boost files installed in /tools/moses/boost_1_53_0/lib.
>
>
> Thanks,
>
> Lane
>
>
>
>
> On Mon, Aug 6, 2012 at 12:39 PM, Kenneth Heafield <moses@kheafield.com
> <mailto:moses@kheafield.com>> wrote:
>
> Ok, committed. Here's how the build system now behaves:
>
> link=shared: Everything linked dynamically.
>
> Default: internal libraries are statically linked. Boost and zlib
> statically linked if possible. libSegFault dynamically linked.
> Dynamically linked executable.
>
> --without-libsegfault: Same as default but no libSegFault. Still a
> dynamically linked executable, even if you have static boost and zlib.
> It's complicated to do detect then be automatic.
>
> --static: No libSegFault. Print warning messages if you're missing
> static libraries, but keep building anyway. Static executable.
>
> Kenneth
>
> On 08/06/2012 12:00 PM, Kenneth Heafield wrote:
> > D'oh, it's a feature, not a bug. Add runtime-link=static and
> you'll get
> > a fully-static executable.
> >
> > Plan to add this to the Moses build system. Testing now.
> >
> > Kenneth
> >
> > On 08/06/2012 11:31 AM, Tom Hoar wrote:
> >> FYI, this build is on U-10.04.4 using the bjam shipped with
> Moses. I can
> >> send you the complete log if it helps. Let me know if you need
> anything
> >> else.
> >>
> >> Tom
> >>
> >>
> >> On Mon, 06 Aug 2012 11:27:27 -0400, Kenneth Heafield
> >> <moses@kheafield.com <mailto:moses@kheafield.com>> wrote:
> >>> Hi,
> >>>
> >>> There appears to be a bug in boost-build that prevents it from
> making
> >>> fully-static binaries with g++. This might take a while to sort out.
> >>>
> >>> Kenneth
> >>>
> >>> On 08/06/2012 10:48 AM, Tom Hoar wrote:
> >>>> Thanks, Ken. I think this is easier than adding another option
> to Moses
> >>>> if we can understand the output. The ldd manpage says "ldd
> prints the
> >>>> shared libraries required by each program or shared library
> specified on
> >>>> the command line." I assume "shared libraries" means not statically
> >>>> linked?
> >>>>
> >>>> Here's the output on the moses I just built with today's github
> updates.
> >>>> The log output reported exit code 0 for all lines with "g++
> -static".
> >>>>
> >>>> tahoar@library1:~$ ldd `which moses`
> >>>> linux-vdso.so.1 => (0x00007fffa2954000)
> >>>> librt.so.1 => /lib/librt.so.1 (0x00007f818aa7b000)
> >>>> libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f818a767000)
> >>>> libm.so.6 => /lib/libm.so.6 (0x00007f818a4e3000)
> >>>> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f818a2cc000)
> >>>> libpthread.so.0 => /lib/libpthread.so.0 (0x00007f818a0af000)
> >>>> libc.so.6 => /lib/libc.so.6 (0x00007f8189d2b000)
> >>>> /lib64/ld-linux-x86-64.so.2 (0x00007f818aca7000)
> >>>>
> >>>> Is this what I should expect if all of the libboost libraries are
> >>>> statically linked? I think so because there are no references
> to the
> >>>> lboost_* or lz libraries in the log file.
> >>>>
> >>>> Tom
> >>>>
> >>>>
> >>>> On Mon, 06 Aug 2012 10:14:35 -0400, Kenneth Heafield
> >>>> <moses@kheafield.com <mailto:moses@kheafield.com>> wrote:
> >>>>> On linux,
> >>>>>
> >>>>> ldd bin/moses
> >>>>>
> >>>>> On 08/06/2012 10:03 AM, Tom Hoar wrote:
> >>>>>> Thanks Ken. I downloaded/compiled with the latest changes up to
> >>>>>> BOOST_CHECK_CLOSE. The exit code 1 disappeared and everything
> seems
> >>>>>> okay.
> >>>>>>
> >>>>>> One more question. Other than the log output at compile time,
> is there
> >>>>>> any way to query the Moses binary to see which libraries are
> >>>>>> statically
> >>>>>> vs dynamically linked?
> >>>>>>
> >>>>>> For example, a while back, someone on the list gave this
> command to
> >>>>>> test
> >>>>>> if the binary was compiled --with-srilm, and there are others for
> >>>>>> IRSTLM, RANDLM and KENLM.
> >>>>>>
> >>>>>> nm -C "/path/to/moses" | grep "vtable for
> Moses::LanguageModelSRI"`
> >>>>>>
> >>>>>> I tried the obvious two grep searches "static" and "dynamic"
> (below),
> >>>>>> but they don't seem to relate to the libraries. Does anyone
> know a way
> >>>>>> to find/test if libraries are dynamically or statically linked?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Tom
> >>>>>>
> >>>>>>
> >>>>>> tahoar@library1:~/domy-2.5$ nm -C `which moses` | grep "static"
> >>>>>> 00000000005ddbc0 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005de7c1 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005ded37 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005e0067 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005e2fc6 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005ecae3 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005ef140 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f475f t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f4cd9 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f5e53 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f6f8e t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 00000000005f7183 t
> __static_initialization_and_destruction_0(int, int)
> >>>>>> 0000000000883cc0 d static_bl_desc
> >>>>>> 0000000000883ca0 d static_d_desc
> >>>>>> 0000000000610680 r static_dtree
> >>>>>> 0000000000883c80 d static_l_desc
> >>>>>> 0000000000610200 r static_ltree
> >>>>>>
> >>>>>> tahoar@library1:~/domy-2.5$ nm -C `which moses` | grep "dynamic"
> >>>>>> 0000000000570c50 W
> >>>>>>
> boost::unordered_detail::hash_table_unique_keys<std::pair<float const,
> >>>>>> boost::dynamic_bitset<unsigned long, std::allocator<unsigned
> long>
> >>>>>>>> ,
> >>>>>> float, boost::hash<float>, std::equal_to<float>,
> >>>>>> std::allocator<std::pair<float const,
> boost::dynamic_bitset<unsigned
> >>>>>> long, std::allocator<unsigned long> > > >
> >::operator[](float const&)
> >>>>>> 000000000057baf0 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<std::pair<unsigned
> >>>>>>
> >>>>>>
> >>>>>> char, unsigned char> const, boost::dynamic_bitset<unsigned long,
> >>>>>> std::allocator<unsigned long> > > >
> >::~hash_table_data_unique_keys()
> >>>>>> 0000000000570af0 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long,
> std::allocator<unsigned
> >>>>>> long> > > > >::create_buckets()
> >>>>>> 0000000000570c00 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long,
> std::allocator<unsigned
> >>>>>> long> > > > >::node_constructor::~node_constructor()
> >>>>>> 0000000000570b70 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<float
> >>>>>>
> >>>>>>
> >>>>>> const, boost::dynamic_bitset<unsigned long,
> std::allocator<unsigned
> >>>>>> long> > > > >::~hash_table_data_unique_keys()
> >>>>>> 000000000057ba60 W
> >>>>>>
> >>>>>>
> >>>>>>
> boost::unordered_detail::hash_table_data_unique_keys<std::allocator<std::pair<unsigned
> >>>>>>
> >>>>>>
> >>>>>> int const, boost::dynamic_bitset<unsigned long,
> >>>>>> std::allocator<unsigned
> >>>>>> long> > > > >::~hash_table_data_unique_keys()
> >>>>>> U __dynamic_cast@@CXXABI_1.3
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Mon, 06 Aug 2012 08:20:00 -0400, Kenneth Heafield
> >>>>>> <moses@kheafield.com <mailto:moses@kheafield.com>> wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> You're correct. There doesn't seem to be a static version of
> this
> >>>>>>> library. I've added the --nosegfault option (which isn't as
> cool as
> >>>>>>> it sounds) to skip this library.
> >>>>>>>
> >>>>>>> Kenneth
> >>>>>>>
> >>>>>>> On 08/06/2012 02:08 AM, Tom Hoar wrote:
> >>>>>>>> I read the comment "In order to obtain a fully static Moses,
> >>>>>>>> every g++
> >>>>>>>> command line that includes "-static" must pass with exit
> code 0."
> >>>>>>>> with
> >>>>>>>> interest.
> >>>>>>>>
> >>>>>>>> When we compile moses, this log output line shows an exit
> code 1.
> >>>>>>>> All
> >>>>>>>> the others are 0.
> >>>>>>>>
> >>>>>>>> bash -c "g++ -static -lSegFault -x c++ -<<<'int main() {}' -o
> >>>>>>>> /dev/null
> >>>>>>>>> /dev/null 2>/dev/null"
> >>>>>>>> 1
> >>>>>>>>
> >>>>>>>> Any suggestions as to what's missing/how to correct so we
> have a
> >>>>>>>> fully-static Moses?
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Tom
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, 03 Aug 2012 18:13:47 -0400, Kenneth Heafield
> >>>>>>>> <moses@kheafield.com <mailto:moses@kheafield.com>> wrote:
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Moses attempts to link statically by default but falls back to
> >>>>>>>>> dynamic
> >>>>>>>>> links. You must have static versions of all the dependencies
> >>>>>>>>> installed
> >>>>>>>>> as well. Run
> >>>>>>>>>
> >>>>>>>>> bjam --debug-configuration
> >>>>>>>>>
> >>>>>>>>> and, near the top, it will show you some command lines
> followed by
> >>>>>>>>> their
> >>>>>>>>> exit code. In order to obtain a fully static Moses, every g++
> >>>>>>>>> command
> >>>>>>>>> line that includes "-static" must pass with exit code 0.
> >>>>>>>>>
> >>>>>>>>> Kenneth
> >>>>>>
> >>>>
> >>
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> When a place gets crowded enough to require ID's, social collapse is not
> far away. It is time to go elsewhere. The best thing about space travel
> is that it made it possible to go elsewhere.
> -- R.A. Heinlein, "Time Enough For Love"
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 82, Issue 37
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 82, Issue 37"
Post a Comment