Moses-support Digest, Vol 88, Issue 17

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: sparse features, optimization, custom metrics (Barry Haddow)
2. Re: sparse features, optimization, custom metrics
(Marcin Junczys-Dowmunt)
3. Re: exception during tuning step (Hieu Hoang)
4. Re: Error on compiling Moses: in "ProbingAll"
difference{100%} exceeds 0.01% (Hieu Hoang)
5. Re: sparse features, optimization, custom metrics (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Sun, 09 Feb 2014 19:53:58 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] sparse features, optimization, custom
metrics
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>, moses-support
<moses-support@mit.edu>
Message-ID: <52F7DCD6.7020903@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi Marcin

There was a project at MTM2012 for this, but I have not seen any outputs
from it
http://www.statmt.org/mtm12/index.php%3Fn=Projects.NewDevelopmentFuncionalityForTheAsiyaSuiteParameterOptimizationWithMert
I am not aware of anyone working on new metrics for pro and kbmira.

In principle I don't think it would be hard to implement. The current
implementations of pro and kbmira make use of the sufficient statistics
in the same way that mert does. The main difference is that they require
evaluations of single sentences, as opposed to mert which can optimise a
corpus metric. kbmira uses Chiang's technique (from his 2008 mira paper)
to approximate corpus bleu, but pro just optimises sentence bleu.
However it could (and perhaps should) also employ Chiang's technique.
Both pro and kbmira use methods from BleuScorer to score the sentences
-- smoothedSentenceBleu() and sentenceLevelBackgroundBleu() respectively.

cheers - Barry

On 09/02/14 09:19, Marcin Junczys-Dowmunt wrote:
> Hi list,
> It seems that currently for both, pro and kbmira, optimization of BLEU
> is hardwired into the code. I managed to add my custom metric to mert,
> but would like to experiment with it and sparse features, too.
>
> I see custom metrics is on a TODO list in the mert folder, is someone
> working on custom metrics for sparse features?
> Are pro and/or kbmira in principle compatible with this "sufficient
> statistics per sentence" approach as it is done for mert? Any pointers
> how I could best attack this?
>
> Best,
> Marcin
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



------------------------------

Message: 2
Date: Sun, 09 Feb 2014 21:03:00 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] sparse features, optimization, custom
metrics
To: moses-support <moses-support@mit.edu>
Message-ID: <52F7DEF4.8030001@amu.edu.pl>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi Barry,
OK, thanks for the confirmation, so there is sense to try it. I will see
whether I can manage to add my metric (which by itself is not
particularly useful to the community) and maybe I will manage to
convince pro or kbmira by the way to use the general Scorer classes from
mert.

Which one do you prefer for sparse features? How do they cope with
optimizer instability compared to mert?
Best,
Marcin

W dniu 09.02.2014 20:53, Barry Haddow pisze:
> Hi Marcin
>
> There was a project at MTM2012 for this, but I have not seen any
> outputs from it
> http://www.statmt.org/mtm12/index.php%3Fn=Projects.NewDevelopmentFuncionalityForTheAsiyaSuiteParameterOptimizationWithMert
>
> I am not aware of anyone working on new metrics for pro and kbmira.
>
> In principle I don't think it would be hard to implement. The current
> implementations of pro and kbmira make use of the sufficient
> statistics in the same way that mert does. The main difference is that
> they require evaluations of single sentences, as opposed to mert which
> can optimise a corpus metric. kbmira uses Chiang's technique (from his
> 2008 mira paper) to approximate corpus bleu, but pro just optimises
> sentence bleu. However it could (and perhaps should) also employ
> Chiang's technique. Both pro and kbmira use methods from BleuScorer to
> score the sentences -- smoothedSentenceBleu() and
> sentenceLevelBackgroundBleu() respectively.
>
> cheers - Barry
>
> On 09/02/14 09:19, Marcin Junczys-Dowmunt wrote:
>> Hi list,
>> It seems that currently for both, pro and kbmira, optimization of BLEU
>> is hardwired into the code. I managed to add my custom metric to mert,
>> but would like to experiment with it and sparse features, too.
>>
>> I see custom metrics is on a TODO list in the mert folder, is someone
>> working on custom metrics for sparse features?
>> Are pro and/or kbmira in principle compatible with this "sufficient
>> statistics per sentence" approach as it is done for mert? Any pointers
>> how I could best attack this?
>>
>> Best,
>> Marcin
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>



------------------------------

Message: 3
Date: Mon, 10 Feb 2014 10:36:22 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] exception during tuning step
To: amir haghighi <amir.haghighi.64@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbiYMvj5rbK0T1o2K2jax2C8JW9BY2fwQMH_vZw670vQJA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

You are trying to use IRSTLM for decoding but the decoder isn't compiled
with IRSTLM

Either
1. recompile Moses with irstlm:
./bjam --with-irstlm=[path/to/irstlm] -a
2. Uses kenlm



On 9 February 2014 13:11, amir haghighi <amir.haghighi.64@gmail.com> wrote:

> Hello all
>
>
>
> when I run moses EMS, in the tuning step, it gives this exception:
>
> Exception: moses/FF/Factory.cpp:235 in void
> Moses::FeatureRegistry::Construct(const string&, const string&) threw
> UnknownFeatureException because `i == registry_.end()'.
> Feature name IRSTLM is not registered.
> Exit code: 1
> Failed to run moses with the config
> /opt/working/ems/tuning/moses.filtered.ini.1 at
> /opt/tools/mosesdecoder/scripts/training/mert-moses.pl line 1271.
> cp: cannot stat '/opt/working/ems/tuning/tmp.1/moses.ini': No such file or
> directory
>
>
> I will be thankful if you could help me to solve this problem.
>
> Regards
>
> amir
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140210/1b61be6f/attachment-0001.htm

------------------------------

Message: 4
Date: Mon, 10 Feb 2014 10:40:10 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Error on compiling Moses: in "ProbingAll"
difference{100%} exceeds 0.01%
To: Masa Taka <grantaka36@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbg--_dVAL0gvjLugkXedE7w12hbqMU_Gs+Q9yNSbUVm4g@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

please see Lane's answer to a similar question:
http://article.gmane.org/gmane.comp.nlp.moses.user/10310


On 9 February 2014 15:34, Masa Taka <grantaka36@gmail.com> wrote:

> Hi,
>
> I have been tackling with one compiling failure as follows, would you
> please tell me how to work around?
>
> $HOME = /home/scott/
>
> $ cd /home/scott/tools
> $ cd mosesdecoder
> $ ./bjam -j2 --with-irstlm=/home/scott/tools/irstlm
> --with-giza=/home/scott/tools/giza-pp
> --with-boost=/home/scott/tools/boost_1_55_0 --debug-configuration -d2
> > build.log
>
> ====== BEGIN OUTPUT ======
> Running 1 test case...
> moses/LM/BackwardTest.cpp(162): error in "ProbingAll":
> difference{100%} between static_cast<double>(TransformLMScore( 0.0
> )){0} and static_cast<double>(ngramScore){1.3724476843890443e-07}
> exceeds 0.01%
>
> *** 1 failure detected in test suite "BackwardTest"
>
> EXIT STATUS: 201
> ====== END OUTPUT ======
>
> Regards,
> masa_36
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140210/b3083022/attachment-0001.htm

------------------------------

Message: 5
Date: Mon, 10 Feb 2014 11:07:09 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] sparse features, optimization, custom
metrics
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>, moses-support
<moses-support@mit.edu>
Message-ID: <52F8B2DD.50205@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi Marcin

> Which one do you prefer for sparse features? How do they cope with
> optimizer instability compared to mert?
We have been using kbmira. It seems a bit more stable than mert, and pro
can have problems with the sentence length -- see some recent papers by
Preslav Nakov et al on this problem,

cheers - Barry

On 09/02/14 20:03, Marcin Junczys-Dowmunt wrote:
> Hi Barry,
> OK, thanks for the confirmation, so there is sense to try it. I will see
> whether I can manage to add my metric (which by itself is not
> particularly useful to the community) and maybe I will manage to
> convince pro or kbmira by the way to use the general Scorer classes from
> mert.
>
> Which one do you prefer for sparse features? How do they cope with
> optimizer instability compared to mert?
> Best,
> Marcin
>
> W dniu 09.02.2014 20:53, Barry Haddow pisze:
>> Hi Marcin
>>
>> There was a project at MTM2012 for this, but I have not seen any
>> outputs from it
>> http://www.statmt.org/mtm12/index.php%3Fn=Projects.NewDevelopmentFuncionalityForTheAsiyaSuiteParameterOptimizationWithMert
>>
>> I am not aware of anyone working on new metrics for pro and kbmira.
>>
>> In principle I don't think it would be hard to implement. The current
>> implementations of pro and kbmira make use of the sufficient
>> statistics in the same way that mert does. The main difference is that
>> they require evaluations of single sentences, as opposed to mert which
>> can optimise a corpus metric. kbmira uses Chiang's technique (from his
>> 2008 mira paper) to approximate corpus bleu, but pro just optimises
>> sentence bleu. However it could (and perhaps should) also employ
>> Chiang's technique. Both pro and kbmira use methods from BleuScorer to
>> score the sentences -- smoothedSentenceBleu() and
>> sentenceLevelBackgroundBleu() respectively.
>>
>> cheers - Barry
>>
>> On 09/02/14 09:19, Marcin Junczys-Dowmunt wrote:
>>> Hi list,
>>> It seems that currently for both, pro and kbmira, optimization of BLEU
>>> is hardwired into the code. I managed to add my custom metric to mert,
>>> but would like to experiment with it and sparse features, too.
>>>
>>> I see custom metrics is on a TODO list in the mert folder, is someone
>>> working on custom metrics for sparse features?
>>> Are pro and/or kbmira in principle compatible with this "sufficient
>>> statistics per sentence" approach as it is done for mert? Any pointers
>>> how I could best attack this?
>>>
>>> Best,
>>> Marcin
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 88, Issue 17
*********************************************

Tweet