Home » Uncategories » Moses-support Digest, Vol 107, Issue 23

Moses-support Digest, Vol 107, Issue 23

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. EMS Factored Model generation table (Alex Martinez)
2. Re: Building Moses in Eclipse on Mac OS (Hieu Hoang)
3. Re: Building Moses in Eclipse on Mac OS (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 08 Sep 2015 16:49:16 +0000 (GMT)
From: Alex Martinez <cmxela@me.com>
Subject: [Moses-support] EMS Factored Model generation table
To: moses-support@mit.edu
Message-ID: <fb6f2c2b-7939-4afd-8167-691369494bbb@me.com>
Content-Type: text/plain; charset="utf-8"

I'm getting this error on a EMS training during tuning phase, not sure if this configuration has sense, just trying to play with a POS tagger and with factored models:

Executing: /opt/moses/bin/moses -v 0 ? -config /mnt/a62/devel/en_es/tuning/moses.filtered.ini.1 -weight-overwrite 'GenerationModel0= 0.038462 LM1= 0.064103 TranslationModel2= 0.025641 0.025641 0.025641 0.025641 TranslationModel1= 0.025641 0.025641 0.025641 0.025641 WordPenalty0= -0.128205 LM2= 0.064103 Distortion0= 0.038462 TranslationModel0= 0.025641 0.025641 0.025641 0.025641 LexicalReordering0= 0.038462 0.038462 0.038462 0.038462 0.038462 0.038462 LM0= 0.064103 GenerationModel1= 0.038462 0.000000 PhrasePenalty0= 0.025641' ?-n-best-list run1.best100.out 100 distinct ?-input-file /mnt/a62/devel/en_es/data/corpora.tuning.en > run1.out
Can't read /mnt/a62/devel/en_es/model/generation.1-0
Exit code: 1
The decoder died. CONFIG WAS -weight-overwrite 'GenerationModel0= 0.038462 LM1= 0.064103 TranslationModel2= 0.025641 0.025641 0.025641 0.025641 TranslationModel1= 0.025641 0.025641 0.025641 0.025641 WordPenalty0= -0.128205 LM2= 0.064103 Distortion0= 0.038462 TranslationModel0= 0.025641 0.025641 0.025641 0.025641 LexicalReordering0= 0.038462 0.038462 0.038462 0.038462 0.038462 0.038462 LM0= 0.064103 GenerationModel1= 0.038462 0.000000 PhrasePenalty0= 0.025641'?
cp: cannot stat ?/mnt/a62/devel/en_es/tuning/tmp.1/moses.ini?: No such file or directory


I'm trying to use a factored model from en to es with the following factors:

input-factors = pos lemma surface
output-factors = pos lemma surface
alignment-factors = "surface -> surface"
translation-factors = "surface -> surface, pos -> pos"
reordering-factors = "surface -> surface"
generation-factors = "surface -> pos"
decoding-steps = "t0,t1,g0"

The corpus has been previously cleaned and factored with the format:

pos0|lemma0|surface0?pos1|lemma1|surface1?pos2|lemma2|surface2 ...?posN|lemmaN|surfaceN

before factoring I've filtered the corpus to filter characters like | < > [] .... and also filtered the segments with length > 80




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150908/cdbaca0c/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 9 Sep 2015 07:02:15 +0200
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Building Moses in Eclipse on Mac OS
To: Yuqi Zhang <zhang.yuqiyu@gmail.com>, Moses-support@mit.edu
Message-ID: <55EFBD57.30300@gmail.com>
Content-Type: text/plain; charset="windows-1252"

ok. PhraseDictionaryCompact is popular enough, Eclipse now compile it
with this phrase table.

You must softlink cmph to the root of your moses source code directory:
cd ..../mosesdecoder
ln -s /Users/hieu/workspace/cmph-2.0 cmph
Re-import the eclipse project files into your workspace and recompile

On 08/09/2015 18:18, Yuqi Zhang wrote:
> Dear Moses Developers,
>
> I am building the debug version of moses on my Mac laptop (OS 10.10.4)
> with Eclipse CDT (Luna).
> I can successfully built and debug the standard version with
> PhraseDictionMemory.
> However, when I want to debug the PhraseDictionaryCompact by invoking
> the conditional compiling symbol "HAVE_CMPH", I got link errors (e.g.
> on moses-cmd/):
>
> ------------------------------------------------------------------------------------------------------------------------
>
> Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
>
> Target: x86_64-apple-darwin14.4.0
>
> Thread model: posix
>
> "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld"
> -demangle -dynamic -arch x86_64 -macosx_version_min 10.10.0 -o
> moses-cmd
> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/../../boost/lib64
> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/moses/Debug
> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/lm/Debug
> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/OnDiskPt/Debug
> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/util/Debug
> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/search/Debug
> -L/opt/local/lib ./Main.o -lmoses -lsearch -lOnDiskPt -llm -lutil
> -lboost_iostreams -lboost_serialization -lboost_system -lboost_thread
> -lboost_filesystem -lpthread -lz -lbz2 -ldl -lc++ -lSystem
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin/libclang_rt.osx.a
>
> Undefined symbols for architecture x86_64:
>
> "Moses::PhraseDictionaryCompact::PhraseDictionaryCompact(std::string
> const&)", referenced from:
>
> Moses::(anonymous
> namespace)::DefaultFeatureFactory<Moses::PhraseDictionaryCompact>::Create(std::string
> const&) in libmoses.a(Factory.o)
>
> "Moses::LexicalReorderingTableCompact::CheckAndLoad(std::string
> const&, std::vector<unsigned long, std::allocator<unsigned long> >
> const&, std::vector<unsigned long, std::allocator<unsigned long> >
> const&, std::vector<unsigned long, std::allocator<unsigned long> >
> const&)", referenced from:
>
> Moses::LexicalReorderingTable::LoadAvailable(std::string const&,
> std::vector<unsigned long, std::allocator<unsigned long> > const&,
> std::vector<unsigned long, std::allocator<unsigned long> > const&,
> std::vector<unsigned long, std::allocator<unsigned long> > const&) in
> libmoses.a(LexicalReorderingTable.o)
>
> ... ...
>
> ld: symbol(s) not found for architecture x86_64
>
> clang: error: linker command failed with exit code 1 (use -v to see
> invocation)
>
> ------------------------------------------------------------------------------------------------------------------------
>
>
> I have worked on this for several days, but cannot fix it.
>
> The full log files is attached.
>
> Any suggestions or hints are welcome!
>
> Thank you very much!
>
> Best regards,
>
> Yuqi Zhang
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
http://www.hoang.co.uk/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150909/078f740b/attachment-0001.html

------------------------------

Message: 3
Date: Wed, 9 Sep 2015 07:02:59 +0200
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Building Moses in Eclipse on Mac OS
To: Yuqi Zhang <zhang.yuqiyu@gmail.com>, Moses-support@mit.edu
Message-ID: <55EFBD83.2090703@gmail.com>
Content-Type: text/plain; charset="windows-1252"

ps. here's the checkin if you're interested
https://github.com/moses-smt/mosesdecoder/commit/d55325cbdb33762aacf7549ae72e43e73df0d160

On 09/09/2015 07:02, Hieu Hoang wrote:
> ok. PhraseDictionaryCompact is popular enough, Eclipse now compile it
> with this phrase table.
>
> You must softlink cmph to the root of your moses source code directory:
> cd ..../mosesdecoder
> ln -s /Users/hieu/workspace/cmph-2.0 cmph
> Re-import the eclipse project files into your workspace and recompile
>
> On 08/09/2015 18:18, Yuqi Zhang wrote:
>> Dear Moses Developers,
>>
>> I am building the debug version of moses on my Mac laptop (OS
>> 10.10.4) with Eclipse CDT (Luna).
>> I can successfully built and debug the standard version with
>> PhraseDictionMemory.
>> However, when I want to debug the PhraseDictionaryCompact by invoking
>> the conditional compiling symbol "HAVE_CMPH", I got link errors (e.g.
>> on moses-cmd/):
>>
>> ------------------------------------------------------------------------------------------------------------------------
>>
>> Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
>>
>> Target: x86_64-apple-darwin14.4.0
>>
>> Thread model: posix
>>
>> "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld"
>> -demangle -dynamic -arch x86_64 -macosx_version_min 10.10.0 -o
>> moses-cmd
>> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/../../boost/lib64
>> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/moses/Debug
>> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/lm/Debug
>> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/OnDiskPt/Debug
>> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/util/Debug
>> -L/Users/d064586/Dev/moses-RELEASE-3.0/mosesbuild/moses/contrib/other-builds/search/Debug
>> -L/opt/local/lib ./Main.o -lmoses -lsearch -lOnDiskPt -llm -lutil
>> -lboost_iostreams -lboost_serialization -lboost_system -lboost_thread
>> -lboost_filesystem -lpthread -lz -lbz2 -ldl -lc++ -lSystem
>> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.1.0/lib/darwin/libclang_rt.osx.a
>>
>> Undefined symbols for architecture x86_64:
>>
>> "Moses::PhraseDictionaryCompact::PhraseDictionaryCompact(std::string
>> const&)", referenced from:
>>
>> Moses::(anonymous
>> namespace)::DefaultFeatureFactory<Moses::PhraseDictionaryCompact>::Create(std::string
>> const&) in libmoses.a(Factory.o)
>>
>> "Moses::LexicalReorderingTableCompact::CheckAndLoad(std::string
>> const&, std::vector<unsigned long, std::allocator<unsigned long> >
>> const&, std::vector<unsigned long, std::allocator<unsigned long> >
>> const&, std::vector<unsigned long, std::allocator<unsigned long> >
>> const&)", referenced from:
>>
>> Moses::LexicalReorderingTable::LoadAvailable(std::string const&,
>> std::vector<unsigned long, std::allocator<unsigned long> > const&,
>> std::vector<unsigned long, std::allocator<unsigned long> > const&,
>> std::vector<unsigned long, std::allocator<unsigned long> > const&) in
>> libmoses.a(LexicalReorderingTable.o)
>>
>> ... ...
>>
>> ld: symbol(s) not found for architecture x86_64
>>
>> clang: error: linker command failed with exit code 1 (use -v to see
>> invocation)
>>
>> ------------------------------------------------------------------------------------------------------------------------
>>
>>
>> I have worked on this for several days, but cannot fix it.
>>
>> The full log files is attached.
>>
>> Any suggestions or hints are welcome!
>>
>> Thank you very much!
>>
>> Best regards,
>>
>> Yuqi Zhang
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> Hieu Hoang
> http://www.hoang.co.uk/hieu

--
Hieu Hoang
http://www.hoang.co.uk/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150909/4ad8bde5/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 107, Issue 23
**********************************************

0 Response to "Moses-support Digest, Vol 107, Issue 23"

Post a Comment

Subscribe to: Post Comments (Atom)