Moses-support Digest, Vol 99, Issue 68

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Negative weights for translation model and reordering models
(HOANG Cong Duy Vu)
2. Re: Negative weights for translation model and reordering
models (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Fri, 30 Jan 2015 13:37:01 +0800
From: HOANG Cong Duy Vu <duyvuleo@gmail.com>
Subject: [Moses-support] Negative weights for translation model and
reordering models
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAPRaJX0j-bN+KKzgFdyiddeXxCK9nrVOr_dd-3e-T05vbAfHJA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I trained the conventional baseline system with translation model, lexical
reordering model (wbe-msd-bidirectional-fe-allff), language model.

I've encountered the following problems:

1) When I *add* another hierarchical reordering model
(hier-mslr-bidirectional-fe-allff, with 8 dense features), after tuning
with MIRA, some of the weight of lexical reordering model become negative
like:
...
LexicalReordering0= -0.00398112195593915 0.00177901253382393
-0.00561620995293243 0.0115717603473397 0.00416629648355966
0.0123725554271622
LexicalReordering1= 0.0603959132391883 0.0637905029984998
0.0453474898750001 0.0255712011871197 0.0333243029043151 0.0360465652275341
0.0102744718438895 0.037224797723084
...

2) When I *add* the bilingual NPLM model, after tuning with MIRA, one of
the weight of translation model becomes negative like:
..
NNJM0= 0.0425000449522696
TranslationModel0= 0.0528812807140059 0.0459795108631494 0.0322642254177683
-0.00103257479908177
...

I suspect there is a sign of *over-fitting* problem.
Do you used to encounter this?

May I seek for your advise?

Thank you very much!

--
Cheers,
Vu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150130/0bb70547/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 30 Jan 2015 10:01:05 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Negative weights for translation model
and reordering models
To: Cong Duy Vu Hoang <duyvuleo@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAFADDAsQPxQ_RacHBq-8qCRnAYe8oLDKh2_5qeU=1zrQkMatg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

this is not uncommon.

There is complex interaction between the features, so that a highly
weighted correlated feature may push down a feature's value into
negative territory.

-phi

On Fri, Jan 30, 2015 at 12:37 AM, HOANG Cong Duy Vu <duyvuleo@gmail.com> wrote:
> Hi,
>
> I trained the conventional baseline system with translation model, lexical
> reordering model (wbe-msd-bidirectional-fe-allff), language model.
>
> I've encountered the following problems:
>
> 1) When I add another hierarchical reordering model
> (hier-mslr-bidirectional-fe-allff, with 8 dense features), after tuning with
> MIRA, some of the weight of lexical reordering model become negative like:
> ...
> LexicalReordering0= -0.00398112195593915 0.00177901253382393
> -0.00561620995293243 0.0115717603473397 0.00416629648355966
> 0.0123725554271622
> LexicalReordering1= 0.0603959132391883 0.0637905029984998 0.0453474898750001
> 0.0255712011871197 0.0333243029043151 0.0360465652275341 0.0102744718438895
> 0.037224797723084
> ...
>
> 2) When I add the bilingual NPLM model, after tuning with MIRA, one of the
> weight of translation model becomes negative like:
> ..
> NNJM0= 0.0425000449522696
> TranslationModel0= 0.0528812807140059 0.0459795108631494 0.0322642254177683
> -0.00103257479908177
> ...
>
> I suspect there is a sign of over-fitting problem.
> Do you used to encounter this?
>
> May I seek for your advise?
>
> Thank you very much!
>
> --
> Cheers,
> Vu
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 99, Issue 68
*********************************************

Tweet