Moses-support Digest, Vol 109, Issue 40

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. baseline-system has very low BLEU-Score (Raphael Hoeps)
2. Re: error: ?intrusive_ptr_release? was not declared in this
scope (Hieu Hoang)
3. Re: baseline-system has very low BLEU-Score (Rico Sennrich)


----------------------------------------------------------------------

Message: 1
Date: Wed, 18 Nov 2015 14:36:23 +0100
From: Raphael Hoeps <raphael.hoeps@gmx.net>
Subject: [Moses-support] baseline-system has very low BLEU-Score
To: moses-support@mit.edu
Message-ID: <564C7ED7.8050406@gmx.net>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi,
I'm a Computer Science student from Germany working on a SMT-project. I
tried to get into the moses-system a little bit and did the
baseline-tutorial found here:
http://www.statmt.org/moses/?n=Moses.Baseline. I sticked to this
tutorial but used the German/English corpora.

Unfortunately in the end I got a poor BLEU-score of only 3.76:
BLEU = 3.76, 24.1/6.2/2.2/0.9 (BP=0.906, ratio=0.910, hyp_len=68049,
ref_len=74753)
In the tutorial, a score of 23.5 was received.

I think did everything as shown in the tutorial, except for one thing:
in the tuning-part I cut down the two development corpora to 500 lines
(from 2000), because my Laptop is quite old and I wanted to speed up the
process a little bit. (It still took my Laptop 6 hours).
Do you think that this is the reason for my poor score?
Is it maybe because I used the German/English corpora, so the score
can't be compared to the English/French system in the tutorial?
Or did I just make a mistake when typing all the commands? Any ideas how
to find this mistake?

Thank you very much for your help,
Raphael



------------------------------

Message: 2
Date: Wed, 18 Nov 2015 13:37:37 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] error: ?intrusive_ptr_release? was not
declared in this scope
To: Nicholas Ruiz <nicruiz@fbk.eu>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbi3V-5WPGz0fz4fOkeBgqjzTX=UOjor+i8=G9br+jDDPw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Looks like the boost library should be compiled statically only.
./b2 link=static ....
The docs tell you how to do it
http://www.statmt.org/moses/?n=Development.GetStarted

Hieu Hoang
http://www.hoang.co.uk/hieu

On 18 November 2015 at 12:58, Nicholas Ruiz <nicruiz@fbk.eu> wrote:

> Thanks, Ulrich! My compilation finished successfully. However, when I try
> to run moses, I get one of these:
>
> $ bin/moses
> bin/moses: error while loading shared libraries:
> libboost_serialization.so.1.59.0: cannot open shared object file: No such
> file or directory
>
> Incidentally, I noticed a similar problem on my macbook:
>
> 5534-MacBook-Pro:mosesdecoder nruiz$ bin/moses
> dyld: Library not loaded: libboost_iostreams.dylib
> Referenced from: /Users/nruiz/mosesdecoder/bin/moses
> Reason: image not found
> Trace/BPT trap: 5
> 5534-MacBook-Pro:mosesdecoder nruiz$ bin/lmplz
> dyld: Library not loaded: libboost_thread.dylib
> Referenced from: /Users/nruiz/mosesdecoder/bin/lmplz
> Reason: image not found
> 5534-MacBook-Pro:mosesdecoder nruiz$ otool -L bin/moses
> bin/moses:
> *libboost_iostreams.dylib* (compatibility version 0.0.0, current version
> 0.0.0)
> libboost_filesystem.dylib (compatibility version 0.0.0, current version
> 0.0.0)
> /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
> /usr/lib/liblzma.5.dylib (compatibility version 6.0.0, current version
> 6.3.0)
>
>
> In both cases, I'm using more or less a default compile.sh script:
>
> 5534-MacBook-Pro:mosesdecoder nruiz$ cat compile.sh
> #!/bin/bash
> # this script assumes that all 3rd-party dependencies are installed under
> ./opt
> # you can install all 3rd-party dependencies by running make -f
> contrib/Makefiles/install-dependencies.gmake
>
> set -e -o pipefail
> ./bjam --with-irstlm=./opt *--with-boost=./opt* --with-cmph=./opt
> --with-xmlrpc-c=./opt --with-mm --with-probing-pt -j$(getconf
> _NPROCESSORS_ONLN) $@
>
> It seems like in both cases, `--with-boost=/.opt` is not ensuring that the
> executable files can find the boost libs.
>
> z?n? ???u
>
> On Wed, Nov 18, 2015 at 5:37 AM, Ulrich Germann <ulrich.germann@gmail.com>
> wrote:
>
>> Hi Nicholas,
>>
>> it looks like gcc-4.4.7 and boost::intrusive_ptr don't get along.
>> Switched to shared_ptr instead.
>>
>> Please pull the latest version and try again.
>>
>> - Uli
>>
>> On Tue, Nov 17, 2015 at 7:48 PM, Nicholas Ruiz <nicruiz@fbk.eu> wrote:
>>
>>> Hi all,
>>>
>>> I'm following the steps on this website to install the HEAD version of
>>> moses on a CentOS
>>> http://www.statmt.org/moses/?n=Development.GetStarted
>>>
>>> $ cat /etc/issue
>>>
>>> CentOS release 6.7 (Final)
>>>
>>> Kernel \r on an \m
>>>
>>> $ gcc -v
>>>
>>> Using built-in specs.
>>>
>>> Target: x86_64-redhat-linux
>>>
>>> Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
>>> --infodir=/usr/share/info --with-bugurl=
>>> http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared
>>> --enable-threads=posix --enable-checking=release --with-system-zlib
>>> --enable-__cxa_atexit --disable-libunwind-exceptions
>>> --enable-gnu-unique-object
>>> --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
>>> --enable-java-awt=gtk --disable-dssi
>>> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
>>> --enable-libgcj-multifile --enable-java-maintainer-mode
>>> --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib
>>> --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686
>>> --build=x86_64-redhat-linux
>>>
>>> Thread model: posix
>>>
>>> gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)
>>>
>>>
>>> When attempting to compile using compile.sh, I get the following error:
>>>
>>> "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall
>>> -pthread -I/n/w10-nruiz/mosesdecoder/opt/include
>>> -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_SYSTEM_DYN_LINK
>>> -DBOOST_THREAD_DYN_DLL -DHAVE_CMPH -DHAVE_PROBINGPT -DHAVE_XMLRPC_C
>>> -DKENLM_MAX_ORDER=6 -DLM_IRST -DMAX_NUM_FACTORS=4 -DNDEBUG -DPT_UG
>>> -DTRACE_ENABLE=1 -DWITH_THREADS -D_FILE_OFFSET_BITS=64 -D_LARGE_FILES
>>> -I"." -I"moses/TranslationModel/CompactPT/opt/include" -I"opt/include"
>>> -I"util/double-conversion" -c -o
>>> "moses/TranslationModel/UG/bin/gcc-4.4.7/release/link-static/threading-multi/count-ptable-features.o"
>>> "moses/TranslationModel/UG/count-ptable-features.cc"
>>>
>>>
>>> ...failed gcc.compile.c++
>>> moses/TranslationModel/UG/bin/gcc-4.4.7/release/link-static/threading-multi/count-ptable-features.o...
>>>
>>> gcc.compile.c++
>>> moses/TranslationModel/UG/bin/gcc-4.4.7/release/link-static/threading-multi/ptable-describe-features.o
>>>
>>> In file included from
>>> /n/w10-nruiz/mosesdecoder/opt/include/boost/intrusive_ptr.hpp:16,
>>>
>>> from moses/TranslationModel/UG/mmsapt.h:9,
>>>
>>> from
>>> moses/TranslationModel/UG/ptable-describe-features.cc:2:
>>>
>>> /n/w10-nruiz/mosesdecoder/opt/include/boost/smart_ptr/intrusive_ptr.hpp:
>>> In destructor ?boost::intrusive_ptr<T>::~intrusive_ptr() [with T =
>>> sapt::mmBitext<sapt::L2R_Token<sapt::SimpleWordId> >]?:
>>>
>>> moses/TranslationModel/UG/mmsapt.h:56: instantiated from here
>>>
>>> /n/w10-nruiz/mosesdecoder/opt/include/boost/smart_ptr/intrusive_ptr.hpp:97:
>>> error: ?intrusive_ptr_release? was not declared in this scope
>>>
>>> I installed boost to ./opt using the instructions listed on the above
>>> site. I'm able to compile just fine following the instructions on my
>>> MacBook Pro (obviously different architecture). Any ideas why this might be
>>> happening?
>>>
>>> Thanks,
>>>
>>> Nick Ruiz
>>> z?n? ???u
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Ulrich Germann
>> Senior Researcher
>> School of Informatics
>> University of Edinburgh
>>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151118/ee533d00/attachment-0001.html

------------------------------

Message: 3
Date: Wed, 18 Nov 2015 13:56:24 +0000
From: Rico Sennrich <rico.sennrich@gmx.ch>
Subject: Re: [Moses-support] baseline-system has very low BLEU-Score
To: moses-support@mit.edu
Message-ID: <564C8388.5010908@gmx.ch>
Content-Type: text/plain; charset=windows-1252; format=flowed

Hello Raphael,

I suggest that you check if you mixed up the languages somewhere, and
check if your translation output is actually English.

3.76 BLEU is possible to achieve without translation (because names and
some function words are the same between English and German), and it's
possible that you used the wrong reference file when measuring BLEU, or
that your SMT system is broken in some way and just copies the source
text to the output.

best wishes,
Rico


On 18.11.2015 13:36, Raphael Hoeps wrote:
> Hi,
> I'm a Computer Science student from Germany working on a SMT-project. I
> tried to get into the moses-system a little bit and did the
> baseline-tutorial found here:
> http://www.statmt.org/moses/?n=Moses.Baseline. I sticked to this
> tutorial but used the German/English corpora.
>
> Unfortunately in the end I got a poor BLEU-score of only 3.76:
> BLEU = 3.76, 24.1/6.2/2.2/0.9 (BP=0.906, ratio=0.910, hyp_len=68049,
> ref_len=74753)
> In the tutorial, a score of 23.5 was received.
>
> I think did everything as shown in the tutorial, except for one thing:
> in the tuning-part I cut down the two development corpora to 500 lines
> (from 2000), because my Laptop is quite old and I wanted to speed up the
> process a little bit. (It still took my Laptop 6 hours).
> Do you think that this is the reason for my poor score?
> Is it maybe because I used the German/English corpora, so the score
> can't be compared to the English/French system in the tutorial?
> Or did I just make a mistake when typing all the commands? Any ideas how
> to find this mistake?
>
> Thank you very much for your help,
> Raphael
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 109, Issue 40
**********************************************

Tweet