Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Using factor in the hierarchical model (Lisa Kim)
2. Re: Moses v 3.0 (Tom Hoar)
3. Re: Suggested change to train-model.perl (Tom Hoar)
----------------------------------------------------------------------
Message: 1
Date: Wed, 4 Feb 2015 15:54:34 +0900
From: Lisa Kim <ekkim214@gmail.com>
Subject: Re: [Moses-support] Using factor in the hierarchical model
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<CAOmDmvYrX3NDhD0H+bLhwqCTqGwKAbpGnUVYU585NrVQGF6gGw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Thank you.
It is successful to change the LM name without "=" symbol.
However, decoding still fails with following message.
Could you check the error message?
--------------------
Start loading text phrase table. Moses format : [0.038] seconds
terminate called recursively
terminate called after throwing an instance of 'util::Exception'
what(): Aborted (core dumped)
Exit code: 134
The decoder died. CONFIG WAS -weight-overwrite 'LM1= 0.125000
PhrasePenalty0= 0.050000 WordPenalty0= -0.250000 TranslationModel0=
0.050000 0.050000 0.050000 0.050000 TranslationModel1= 0.250000 LM0=
0.125000'
2015. 2. 3. ?? 8:12? "Hieu Hoang" <hieuhoang@gmail.com>?? ??:
> as ken mentioned, don't use the = sign in the name of your LM file. I
> think it's fixed in the latest version of moses. Which version do you have,
> from when ago?
>
> moses_chart should handle factors. Again, if you version is from long ago,
> I suggest your update
>
> On 02/02/15 10:06, ekkim214 wrote:
>
>
> I want to use some factor (class) in the hierarchical model.
> I am using the ems script and using KenLM for surface and factor both.
> The building of LM is successful with settings "-discount_fallback" but
> I got an error during tuning phase like this.
>
> Exception: moses/LM/Ken.cpp:399 in Moses::LanguageModel*
> Moses::ConstructKenLM(const string&) threw util::Exception because
> `args.size() != 2'.
> Incorrect format of KenLM property:
> path=/home3/ekkim/working/E2K_1501/hierarchical_class/lm/travel22=mkcls.binlm.5
> Exit code: 1
>
> Does not "moses_chart" decoder which is distributed in the package
> support factored training?
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/47ba5c45/attachment-0001.htm
------------------------------
Message: 2
Date: Wed, 04 Feb 2015 18:18:04 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Moses v 3.0
To: moses-support@mit.edu
Message-ID: <54D1FFEC.30202@precisiontranslationtools.com>
Content-Type: text/plain; charset="windows-1252"
Congratulations team.
Please note that the hyperlinks in the Mosesv3.0releasenotes.pdf file
point to invalid URL's for RELEASE-1.
On 02/03/2015 10:30 PM, Hieu Hoang wrote:
> Hi Everyone
>
> We are proud to release the next version of Moses. It's been another
> busy year with lots of contributions from far and wide. Thanks for
> everyone who contributed code and time, or even just answered a few
> questions or suggestions.
>
> This is a list of the main development over the past year:
>
> 1.
>
> Faster decoder when using sparse features by Matthias Huck.
>
> 2.
>
> Faster, more memory efficient synchronous context-free grammar
> (SCFG) decoding (Sennrich, 2014) by Rico Sennrich.
>
> 3.
>
> Merging moses and moses_chart by Hieu Hoang.
>
> 4.
>
> Synchronous Forest and Tree Substitution Grammar (STSG) models by
> Phil Williams.
>
> 5.
>
> Smaller phrase-table by pruning according translation model scores
> only by Philipp Koehn.
>
> 6.
>
> Simulated post editing tools by Ulrich Germann.
>
> 7.
>
> Hypergraph batch-mira by Barry Haddow.
>
> 8.
>
> Dynamic Suffix-Array phrase-table by Ulrich Germann.
>
> 9.
>
> OxLM, a log-bilinear language model, by (Baltescu et al, 2014).
>
>10.
>
> Bilingual language model (Devlin et al, 2014) by Nikolay
> Bogoychev, Paul Baltescu, Rico Sennrich, and others.
>
>11.
>
> VW Classifier-based feature function by Ales Tamchyna and Marcin
> Junczys-Dowmunt.
>
>12.
>
> Cache-based Translation Model and Language Model by Nicola
> Bertoldi (Bertoldi, 2014).
>
> 13. Daily speed testing framework by Nikolay Bogoychev.
>
>
>
> As in previous versions, this release was forked about a month ago,
> tested to make sure it compiles on a number of platforms, and run
> end-to-end on Linux (Ubuntu 12.04 64-bit). We then distribute
> 1. source code for Moses, IRSTLM, MGIZA
> 2. binaries for Moses, IRSTLM, MGIZA, for the following platform:
> Linux (32 and 64-bit), OSX Yosemite, Windows/Cygwin (32 and 64-bit)
> 3. ready-made tuned models from the end-to-end tests
> 4. Linux-based virtual machines for VirtualBox
>
> More details in the release notes
> http://www.statmt.org/moses/RELEASE-3.0/Mosesv3.0releasenotes.pdf
>
> As ever, if you have any queries or problems, email the mailing list.
> Thank you all for using Moses
> --
> Hieu Hoang
> Research Associate (until March 2015)
> ** searching for interesting commercial MT position **
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/c1f02d7c/attachment-0001.htm
------------------------------
Message: 3
Date: Wed, 04 Feb 2015 18:23:18 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Suggested change to train-model.perl
To: moses-support@mit.edu
Message-ID: <54D20126.1040503@precisiontranslationtools.com>
Content-Type: text/plain; charset="windows-1252"
The path to merge_alignment.py is specific to MGIZA++, but I hadn't
thought of updating the MGIZA project. Seems sensible.
BTW, I've been using MGIZA++ since your cmake updates. It is much
faster! Thanks
Tom
On 02/03/2015 06:43 PM, Hieu Hoang wrote:
> personally, I prefer to keep that part simple and neutral to what
> aligner is used. It currently supports giza++, mgiza, fast align,
> berkeley aligner (probably).
>
> it might be better to change the 'make install' in mgiza to put exe
> and scripts together
>
> On 1 February 2015 at 06:11, Tom Hoar
> <tahoar@precisiontranslationtools.com
> <mailto:tahoar@precisiontranslationtools.com>> wrote:
>
> I acknowledge changes to train-model.perl are done with caution.
> Here's
> one suggestion I think will benefit anyone using MGIZA++.
>
> In the current RC-3 branch, the code looks like this starting at
> line 253:
>
> # supporting binaries from other packages
> my $MKCLS = "$_EXTERNAL_BINDIR/mkcls";
> my $MGIZA_MERGE_ALIGN = "$_EXTERNAL_BINDIR/merge_alignment.py";
> my $GIZA;
> my $SNT2COOC;
>
>
> A "standard" build of MGIZA includes these steps and creates a
> directory
> tree with $MGIZAPP_PREFIX/bin, $MGIZAPP_PREFIX/lib and
> $MGIZAPP_PREFIX/scripts.
>
> cmake . -DCMAKE_INSTALL_PREFIX="$MGIZAPP_PREFIX"
> make
> make install
>
>
> Today, the user must copy (symlink) the merge_alignment.py to complete
> the setup:
>
> $MGIZAPP_PREFIX/scripts/merge_alignment.py -->
> $MGIZAPP_PREFIX/bin/merge_alignment.py
>
> Then, the user sets the --external-bin-dir on the train-model.perl
> command line:
>
> --external-bin-dir $MGIZAPP_PREFIX/bin
>
>
> I propose this modification to train-model.perl
>
> # supporting binaries from other packages
> my $MKCLS = "$_EXTERNAL_BINDIR/mkcls";
> my $MGIZA_MERGE_ALIGN = "$_EXTERNAL_BINDIR/merge_alignment.py";
> # added to fall back to MGIZA++ default install directory
> if (! -x $MGIZA_MERGE_ALIGN) {
> $MGIZA_MERGE_ALIGN="$_EXTERNAL_BINDIR/../scripts/merge_alignment.py";
> }
> my $GIZA;
> my $SNT2COOC;
>
>
> By adding this fall back, users who setup with a "standard" MGIZA++
> `make install` don't have to copy the merge_alignment.py file, without
> losing today's functionality.
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Hieu Hoang
> Research Associate (until March 2015)
> ** searching for interesting commercial MT position **
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/c29294c6/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 100, Issue 11
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 100, Issue 11"
Post a Comment