Moses-support Digest, Vol 103, Issue 53

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: How can I change LM binarization in EMS without
re-tuning? (Lane Schwartz)
2. Re: How can I change LM binarization in EMS without
re-tuning? (Matthias Huck)
3. Re: How can I change LM binarization in EMS without
re-tuning? (Philipp Koehn)
4. Re: How can I change LM binarization in EMS without
re-tuning? (Lane Schwartz)


----------------------------------------------------------------------

Message: 1
Date: Wed, 20 May 2015 14:18:38 -0500
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] How can I change LM binarization in EMS
without re-tuning?
To: Matthias Huck <mhuck@inf.ed.ac.uk>, Philipp Koehn <phi@jhu.edu>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZko2MFrb54ipVcLy1TZ-DaJv1CDjBqFr8UXow8CWmnEGQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

What's the difference between what Philipp is suggesting (weight-config =
...) and what Matthias is suggesting (config-with-reused-weights = ...) ?

On Wed, May 20, 2015 at 2:14 PM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:

> Hi Lane,
>
> Just do the LM binarization manually, edit the LM feature line in your
> tuned moses.ini to point to your new binary LM, and tell the EMS where
> to look for the tuned moses.ini:
>
>
> [TUNING]
> config-with-reused-weights = $working-dir/tuning/moses.tuned.ini.10
>
>
> It won't run tuning if you set config-with-reused-weights.
>
> Cheers,
> Matthias
>
> On Wed, 2015-05-20 at 14:01 -0500, Lane Schwartz wrote:
> > I've got a system that I trained using EMS. I'd like to change the
> > binarization of my LM (for example, the original used KenLM probing, and
> > now I want KenLM trie with quantization).
> >
> > If I simply change the lm-binarizer line in my config, EMS assumes that
> it
> > should re-run tuning. Is there a way that I can force it to not re-tune
> in
> > this case?
> >
> > Thanks,
> > Lane
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>


--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150520/f67f7f84/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 20 May 2015 20:19:27 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] How can I change LM binarization in EMS
without re-tuning?
To: Philipp Koehn <phi@jhu.edu>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <1432149567.30904.731.camel@portedgar>
Content-Type: text/plain; charset="UTF-8"

Oh, are there two ways of doing this?
I use config-with-reused-weights rather than weight-config.


On Wed, 2015-05-20 at 15:11 -0400, Philipp Koehn wrote:
> Hi,
>
> you can point to the previous configuration file with the old weights:
>
> [TUNING]
>
> ### instead of tuning with this setting, old weights may be recycled
> # specify here an old configuration file with matching weights
> #
> weight-config = $toy-data/weight.ini
>
> -phi
>
> On Wed, May 20, 2015 at 3:01 PM, Lane Schwartz <dowobeha@gmail.com> wrote:
>
> > I've got a system that I trained using EMS. I'd like to change the
> > binarization of my LM (for example, the original used KenLM probing, and
> > now I want KenLM trie with quantization).
> >
> > If I simply change the lm-binarizer line in my config, EMS assumes that it
> > should re-run tuning. Is there a way that I can force it to not re-tune in
> > this case?
> >
> > Thanks,
> > Lane
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 3
Date: Wed, 20 May 2015 20:38:24 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] How can I change LM binarization in EMS
without re-tuning?
To: Matthias Huck <mhuck@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDvTn825M4+TcRL9yc0JWjB9BAuPUK0v1GSMdYYp16AJA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

here is the difference:

weight-config points to a file and just uses its weights (ignoring all the
other values)
config-with-reused-weights has to have all the correct pointers to model
files etc.

So,
-> if you want to have a EMS config file that runs the binarization and
then uses
the old weights, use "weight-config" (and don't worry about which file it
points to).
-> if you have already all the model files, use "config-with-reused-weights"
(which has to point to the right files).

-phi

On Wed, May 20, 2015 at 3:19 PM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:

> Oh, are there two ways of doing this?
> I use config-with-reused-weights rather than weight-config.
>
>
> On Wed, 2015-05-20 at 15:11 -0400, Philipp Koehn wrote:
> > Hi,
> >
> > you can point to the previous configuration file with the old weights:
> >
> > [TUNING]
> >
> > ### instead of tuning with this setting, old weights may be recycled
> > # specify here an old configuration file with matching weights
> > #
> > weight-config = $toy-data/weight.ini
> >
> > -phi
> >
> > On Wed, May 20, 2015 at 3:01 PM, Lane Schwartz <dowobeha@gmail.com>
> wrote:
> >
> > > I've got a system that I trained using EMS. I'd like to change the
> > > binarization of my LM (for example, the original used KenLM probing,
> and
> > > now I want KenLM trie with quantization).
> > >
> > > If I simply change the lm-binarizer line in my config, EMS assumes
> that it
> > > should re-run tuning. Is there a way that I can force it to not
> re-tune in
> > > this case?
> > >
> > > Thanks,
> > > Lane
> > >
> > >
> > > _______________________________________________
> > > Moses-support mailing list
> > > Moses-support@mit.edu
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
> > >
> > >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150520/87ae23d2/attachment-0001.htm

------------------------------

Message: 4
Date: Wed, 20 May 2015 20:06:03 -0500
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] How can I change LM binarization in EMS
without re-tuning?
To: Philipp Koehn <phi@jhu.edu>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZmPAzgUD9MFyzTg4Ysbn8QB_rkijvhTxxfhP08M7W4n1Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks, Philipp. That worked.

Also, for what it's worth, on this data set, doing KenLM trie quantization
with -q 8 -b did not reduce the BLEU score at all.

On Wed, May 20, 2015 at 7:38 PM, Philipp Koehn <phi@jhu.edu> wrote:

> Hi,
>
> here is the difference:
>
> weight-config points to a file and just uses its weights (ignoring all the
> other values)
> config-with-reused-weights has to have all the correct pointers to model
> files etc.
>
> So,
> -> if you want to have a EMS config file that runs the binarization and
> then uses
> the old weights, use "weight-config" (and don't worry about which file it
> points to).
> -> if you have already all the model files, use
> "config-with-reused-weights"
> (which has to point to the right files).
>
> -phi
>
> On Wed, May 20, 2015 at 3:19 PM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:
>
>> Oh, are there two ways of doing this?
>> I use config-with-reused-weights rather than weight-config.
>>
>>
>> On Wed, 2015-05-20 at 15:11 -0400, Philipp Koehn wrote:
>> > Hi,
>> >
>> > you can point to the previous configuration file with the old weights:
>> >
>> > [TUNING]
>> >
>> > ### instead of tuning with this setting, old weights may be recycled
>> > # specify here an old configuration file with matching weights
>> > #
>> > weight-config = $toy-data/weight.ini
>> >
>> > -phi
>> >
>> > On Wed, May 20, 2015 at 3:01 PM, Lane Schwartz <dowobeha@gmail.com>
>> wrote:
>> >
>> > > I've got a system that I trained using EMS. I'd like to change the
>> > > binarization of my LM (for example, the original used KenLM probing,
>> and
>> > > now I want KenLM trie with quantization).
>> > >
>> > > If I simply change the lm-binarizer line in my config, EMS assumes
>> that it
>> > > should re-run tuning. Is there a way that I can force it to not
>> re-tune in
>> > > this case?
>> > >
>> > > Thanks,
>> > > Lane
>> > >
>> > >
>> > > _______________________________________________
>> > > Moses-support mailing list
>> > > Moses-support@mit.edu
>> > > http://mailman.mit.edu/mailman/listinfo/moses-support
>> > >
>> > >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150520/06dfa656/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 103, Issue 53
**********************************************

Tweet