Moses-support Digest, Vol 103, Issue 38

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. processLexicalTableMin with more than 6 scores (Michael Denkowski)
2. Re: processLexicalTableMin with more than 6 scores
(Marcin Junczys-Dowmunt)
3. Re: processLexicalTableMin with more than 6 scores (Matthias Huck)
4. Re: processLexicalTableMin with more than 6 scores
(Marcin Junczys-Dowmunt)
5. Re: processLexicalTableMin with more than 6 scores
(Michael Denkowski)


----------------------------------------------------------------------

Message: 1
Date: Fri, 15 May 2015 14:35:11 -0400
From: Michael Denkowski <michael.j.denkowski@gmail.com>
Subject: [Moses-support] processLexicalTableMin with more than 6
scores
To: Moses Support <moses-support@mit.edu>
Message-ID:
<CA+-GegL2QsnhzkhoKq3FS7N0j7DxfLWeW_nJTLL-pwHUH3XU0A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

Has anyone successfully used a compact reordering model with extra score
components? I added some features to a reordering table and ran
processLexicalTableMin, which appeared to encode everything (at least by
output file size inspection), but moses still seemed to think it had only 6
scores. I didn't see a way to specify nscores like in
processPhraseTableMin and a brief attempt to change the hard coded score
numbers in LexicalReorderingTableCompact.cpp didn't work. Has anyone else
looked at this?

Best,
Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150515/375ccb7e/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 15 May 2015 20:46:13 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] processLexicalTableMin with more than 6
scores
To: moses-support@mit.edu
Message-ID: <55563EF5.9090609@amu.edu.pl>
Content-Type: text/plain; charset="windows-1252"

Hi,
When I look at my code, I would say it's not hardcoded at all. It
inspects the first score set and uses that number later on. I guess you
would need to provide a reasonable interpretation for that additional
scores in the feature function itself. It probable gets that scores, but
does not use them. Retrieval should just work unless I am missing something.

W dniu 15.05.2015 o 20:35, Michael Denkowski pisze:
> Hi all,
>
> Has anyone successfully used a compact reordering model with extra
> score components? I added some features to a reordering table and ran
> processLexicalTableMin, which appeared to encode everything (at least
> by output file size inspection), but moses still seemed to think it
> had only 6 scores. I didn't see a way to specify nscores like in
> processPhraseTableMin and a brief attempt to change the hard coded
> score numbers in LexicalReorderingTableCompact.cpp didn't work. Has
> anyone else looked at this?
>
> Best,
> Michael
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150515/b0e6975e/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 15 May 2015 19:57:56 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] processLexicalTableMin with more than 6
scores
To: Michael Denkowski <michael.j.denkowski@gmail.com>, Marcin
Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support@mit.edu
Message-ID: <1431716276.30904.632.camel@portedgar>
Content-Type: text/plain; charset="UTF-8"

Hi,

Hmm, I thought the default number of lexical reordering scores was 8?
At least for hier-mslr-bidirectional-fe it's 8.

[feature]
LexicalReordering name=LexicalReordering0 num-features=8 type=hier-mslr-bidirectional-fe-allff input-factor=0 output-factor=0 path=/model-dir/reordering-table.3.hier-mslr-bidirectional-fe

Cheers,
Matthias


On Fri, 2015-05-15 at 20:46 +0200, Marcin Junczys-Dowmunt wrote:
> Hi,
> When I look at my code, I would say it's not hardcoded at all. It
> inspects the first score set and uses that number later on. I guess you
> would need to provide a reasonable interpretation for that additional
> scores in the feature function itself. It probable gets that scores, but
> does not use them. Retrieval should just work unless I am missing something.
>
> W dniu 15.05.2015 o 20:35, Michael Denkowski pisze:
> > Hi all,
> >
> > Has anyone successfully used a compact reordering model with extra
> > score components? I added some features to a reordering table and ran
> > processLexicalTableMin, which appeared to encode everything (at least
> > by output file size inspection), but moses still seemed to think it
> > had only 6 scores. I didn't see a way to specify nscores like in
> > processPhraseTableMin and a brief attempt to change the hard coded
> > score numbers in LexicalReorderingTableCompact.cpp didn't work. Has
> > anyone else looked at this?
> >
> > Best,
> > Michael
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 4
Date: Fri, 15 May 2015 21:03:12 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] processLexicalTableMin with more than 6
scores
To: Matthias Huck <mhuck@inf.ed.ac.uk>, Michael Denkowski
<michael.j.denkowski@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <555642F0.5040403@amu.edu.pl>
Content-Type: text/plain; charset=utf-8; format=flowed

Every type of reordering model has a different number of scores. That's
why you have to provide a meaningful interpretation for them in the
feature function. The compact reordering model is just a stupid
container, it will provide the feature function with all the scores that
were present in the text file, but it does not just add them to the
score component vector.
Best,
Marcin

W dniu 15.05.2015 o 20:57, Matthias Huck pisze:
> Hi,
>
> Hmm, I thought the default number of lexical reordering scores was 8?
> At least for hier-mslr-bidirectional-fe it's 8.
>
> [feature]
> LexicalReordering name=LexicalReordering0 num-features=8 type=hier-mslr-bidirectional-fe-allff input-factor=0 output-factor=0 path=/model-dir/reordering-table.3.hier-mslr-bidirectional-fe
>
> Cheers,
> Matthias
>
>
> On Fri, 2015-05-15 at 20:46 +0200, Marcin Junczys-Dowmunt wrote:
>> Hi,
>> When I look at my code, I would say it's not hardcoded at all. It
>> inspects the first score set and uses that number later on. I guess you
>> would need to provide a reasonable interpretation for that additional
>> scores in the feature function itself. It probable gets that scores, but
>> does not use them. Retrieval should just work unless I am missing something.
>>
>> W dniu 15.05.2015 o 20:35, Michael Denkowski pisze:
>>> Hi all,
>>>
>>> Has anyone successfully used a compact reordering model with extra
>>> score components? I added some features to a reordering table and ran
>>> processLexicalTableMin, which appeared to encode everything (at least
>>> by output file size inspection), but moses still seemed to think it
>>> had only 6 scores. I didn't see a way to specify nscores like in
>>> processPhraseTableMin and a brief attempt to change the hard coded
>>> score numbers in LexicalReorderingTableCompact.cpp didn't work. Has
>>> anyone else looked at this?
>>>
>>> Best,
>>> Michael
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>



------------------------------

Message: 5
Date: Fri, 15 May 2015 15:11:19 -0400
From: Michael Denkowski <michael.j.denkowski@gmail.com>
Subject: Re: [Moses-support] processLexicalTableMin with more than 6
scores
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: Moses Support <moses-support@mit.edu>
Message-ID:
<CA+-GegJ6-yY=Kg_GYoFa6JPwr_8cJgyxD1zpnLvoRX1ApUgGJA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thanks! Yes, telling moses what to do with the scores is my next task. I
was checking to make sure they were loaded at all in the first place.
Based on your responses it sounds like they are but the specified model
type controls the number of scores, not the model file or config option for
number of features. For now I'll work under the assumption that all of the
scores are being loaded.

Best,
Michael

On Fri, May 15, 2015 at 3:03 PM, Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
wrote:

> Every type of reordering model has a different number of scores. That's
> why you have to provide a meaningful interpretation for them in the feature
> function. The compact reordering model is just a stupid container, it will
> provide the feature function with all the scores that were present in the
> text file, but it does not just add them to the score component vector.
> Best,
> Marcin
>
> W dniu 15.05.2015 o 20:57, Matthias Huck pisze:
>
> Hi,
>>
>> Hmm, I thought the default number of lexical reordering scores was 8?
>> At least for hier-mslr-bidirectional-fe it's 8.
>>
>> [feature]
>> LexicalReordering name=LexicalReordering0 num-features=8
>> type=hier-mslr-bidirectional-fe-allff input-factor=0 output-factor=0
>> path=/model-dir/reordering-table.3.hier-mslr-bidirectional-fe
>>
>> Cheers,
>> Matthias
>>
>>
>> On Fri, 2015-05-15 at 20:46 +0200, Marcin Junczys-Dowmunt wrote:
>>
>>> Hi,
>>> When I look at my code, I would say it's not hardcoded at all. It
>>> inspects the first score set and uses that number later on. I guess you
>>> would need to provide a reasonable interpretation for that additional
>>> scores in the feature function itself. It probable gets that scores, but
>>> does not use them. Retrieval should just work unless I am missing
>>> something.
>>>
>>> W dniu 15.05.2015 o 20:35, Michael Denkowski pisze:
>>>
>>>> Hi all,
>>>>
>>>> Has anyone successfully used a compact reordering model with extra
>>>> score components? I added some features to a reordering table and ran
>>>> processLexicalTableMin, which appeared to encode everything (at least
>>>> by output file size inspection), but moses still seemed to think it
>>>> had only 6 scores. I didn't see a way to specify nscores like in
>>>> processPhraseTableMin and a brief attempt to change the hard coded
>>>> score numbers in LexicalReorderingTableCompact.cpp didn't work. Has
>>>> anyone else looked at this?
>>>>
>>>> Best,
>>>> Michael
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150515/d0767f4d/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 103, Issue 38
**********************************************

Tweet