Moses-support Digest, Vol 85, Issue 21

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: distortion model (Kevin Gimpel)
2. Compile moses without gcc optimization (jian zhang)
3. Loading issues with kenlm (Kenneth Heafield)
4. Re: Compile moses without gcc optimization (Kenneth Heafield)
5. Re: scores.size() == indexes.second - indexes.first failed
(Arththika Paramanathan)


----------------------------------------------------------------------

Message: 1
Date: Wed, 6 Nov 2013 12:57:52 -0600
From: Kevin Gimpel <kgimpel@cs.cmu.edu>
Subject: Re: [Moses-support] distortion model
To: "Read, James C" <jcread@essex.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CACAqqMuUMadOX8VUALWKw-b1Rp-SY2eAt4Dcamo2mGC2V14_yw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

James,
The simple distortion model penalizes reordering (assuming its weight has
the correct sign), but the lexicalized reordering model gets more at what
you're suggesting, and pretty much always improves BLEU.
Kevin


On Wed, Nov 6, 2013 at 12:08 PM, Read, James C <jcread@essex.ac.uk> wrote:

> Hi,
>
> I was just reading through the phrase tutorial on the Moses website and
> ran into this:
>
>
> Each of these models contributes information over one aspect of the
> characteristics of a good translation:
>
> * The phrase translation table ensures that the English phrases and
> the German phrases are good translations of each other.
> * The language model ensures that the output is fluent English.
> * The distortion model allows for reordering of the input sentence,
> but at a cost: The more reordering, the more expensive is the translation.
> * The word penalty ensures that the translations do not get too long
> or too short.
>
> I'm not sure if I'm reading this right. The more reordering the more a
> translation is penalised. Surely that's an over generalisation and there
> exist a set of phrase pairs (particularly for language pairs of different
> word order) that we would want to penalise if they were not reordered but
> reward for fulfilling the statistically observed necessity for some measure
> of reordering.
>
> James
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131106/5d971f1e/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 6 Nov 2013 19:52:14 +0000
From: jian zhang <jianzhang09@gmail.com>
Subject: [Moses-support] Compile moses without gcc optimization
To: moses-support@mit.edu
Message-ID:
<CALA=z0CcJ8FEm2X3djPXfpKNB7_Fwz-pjjSG6SKP9DyoQhsVhw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi,

Just wondering how could I turn off the compiler optimization option (set
parameter -O0 ) when compiling moses, thanks.


Jian

--
Jian Zhang
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131106/3c727d2f/attachment-0001.htm

------------------------------

Message: 3
Date: Wed, 06 Nov 2013 19:37:24 -0800
From: Kenneth Heafield <moses@kheafield.com>
Subject: [Moses-support] Loading issues with kenlm
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <527B0AF4.4030005@kheafield.com>
Content-Type: text/plain; charset=ISO-8859-1

Hi,

Some of you might have noticed slow loading kenlm in master (this does
not impact RELEASE-1.0). This is due to a bug introduced by Hieu,
namely that lazyken is not initialized (and could be 1) while it should
probably be 0 (aggressive loading) by default. I have checked in a fix.

LanguageModel *ConstructKenLM(const std::string &line)
{
FactorType factorType;
string filePath;
bool lazy;

vector<string> toks = Tokenize(line);
for (size_t i = 1; i < toks.size(); ++i) {
vector<string> args = Tokenize(toks[i], "=");
CHECK(args.size() == 2);

if (args[0] == "factor") {
factorType = Scan<FactorType>(args[1]);
} else if (args[0] == "order") {
//nGramOrder = Scan<size_t>(args[1]);
} else if (args[0] == "path") {
filePath = args[1];
} else if (args[0] == "lazyken") {
lazy = Scan<bool>(args[1]);
} else if (args[0] == "name") {
// that's ok. do nothing, passes onto LM constructor
}
}

return ConstructKenLM(line, filePath, factorType, lazy);
}

Kenneth


------------------------------

Message: 4
Date: Wed, 06 Nov 2013 19:39:02 -0800
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Compile moses without gcc optimization
To: moses-support@mit.edu
Message-ID: <527B0B56.5050401@kheafield.com>
Content-Type: text/plain; charset=ISO-8859-1

bjam debug. Also make sure that CXXFLAGS and CFLAGS in your environment
do not turn on optimization.

On 11/06/13 11:52, jian zhang wrote:
> Hi,
>
> Just wondering how could I turn off the compiler optimization option
> (set parameter -O0 ) when compiling moses, thanks.
>
>
> Jian
>
> --
> Jian Zhang
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 5
Date: Thu, 7 Nov 2013 15:47:54 +0530
From: Arththika Paramanathan <arthiparamanathan@gmail.com>
Subject: Re: [Moses-support] scores.size() == indexes.second -
indexes.first failed
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAJSfqEwMMS3otpX_omrnLqVWRmJ0R_Xjm6mFPcGB_1Z410wuFA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi,
Thank you for your kind response. This is the command I typed.
~/Desktop/arthi/mosesdecoder/scripts/ems/experiment.perl -config config.toy
-exec

Thank you


On Wed, Nov 6, 2013 at 11:56 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:

> can you please send the config file you used, the moses.ini file, and any
> other file you think would be useful in debugging this. Also, what is the
> EXACT command you typed?
>
>
> On 6 November 2013 11:58, Arththika Paramanathan <
> arthiparamanathan@gmail.com> wrote:
>
>> Hi,
>> I tried Experiment Management System (EMS) & faced an issue given below.
>> And there are no errors in TUNING_apply-weights.26.STDERR file.
>>
>> /home/arththika/Desktop/arthi/mosesdecoder/bin
>> line=UnknownWordPenalty
>> WEIGHT UnknownWordPenalty0=
>> line=WordPenalty
>> WEIGHT WordPenalty0=
>> Check scores.size() == indexes.second - indexes.first failed in
>> ./moses/ScoreComponentCollection.h:235
>> Aborted (core dumped)
>>
>>
>> --
>> regards,
>> P.Arththika
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>


--
regards,
P.Arththika
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131107/d56403fc/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini
Type: application/octet-stream
Size: 1040 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20131107/d56403fc/attachment.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: weight.ini
Type: application/octet-stream
Size: 321 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20131107/d56403fc/attachment-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: config.toy
Type: application/octet-stream
Size: 14594 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20131107/d56403fc/attachment-0002.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini.26
Type: application/octet-stream
Size: 972 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20131107/d56403fc/attachment-0003.obj

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 85, Issue 21
*********************************************

Tweet