Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Hiero ems should die on decode error, but doesn't (Lane Schwartz)
2. Inconsistency in experiment.meta (Lane Schwartz)
3. Re: Inconsistency in experiment.meta (Hieu Hoang)
4. Re: Hiero ems should die on decode error, but doesn't
(Philipp Koehn)
5. Re: Inconsistency in experiment.meta (Philipp Koehn)
6. Re: Inconsistency in experiment.meta (Lane Schwartz)
----------------------------------------------------------------------
Message: 1
Date: Thu, 10 Oct 2013 13:13:14 -0400
From: Lane Schwartz <dowobeha@gmail.com>
Subject: [Moses-support] Hiero ems should die on decode error, but
doesn't
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZmQL-=6EuKdMf+f9Q2POrT4z=zpcNZY-80v6=sdH_d2bw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
I've been toying around again with ems. I know this should be easy,
but for some reason I'm really having trouble getting the toy config
changed to successfully run with a hiero model.
I started out with scripts/ems/example/config.toy, and attempted to
apply the changes described here:
http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc7. The
resulting setup runs, but dies at the final REPORTING stage.
I looked through the output for various steps, and it appears that the
EVALUATION:decode step is dying with error message "Feature function
TranslationModel1 specified 1 dense scores or weights. Actually has
0." As a result, the decoder failed to translate the requested
sentences.
I'm not an expert on ems, but I would have expected EMS to die at the
EVALUATION:decode if that's where the first critical error occurs.
That aside, I'd appreciate any thoughts on why ems might be dying. I'd
like to try running some hiero experiments with ems. FWIW, running the
config.toy phrase-based model works fine.
Thanks,
Lane
------------------------------
Message: 2
Date: Thu, 10 Oct 2013 13:21:49 -0400
From: Lane Schwartz <dowobeha@gmail.com>
Subject: [Moses-support] Inconsistency in experiment.meta
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZnKO8CyNQjmet_WhPmVaKnjm99OKA0HfY6upjw-1EhcCg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
One of the people in my lab has been attempting to add a custom
component to EMS. In trying to do this, we have uncovered what appears
to be an inconsistency in experiment.meta.
Lines 750 (under split-input in [TUNING]) and 758 (under
split-input-devtest in [TUNING]) defines the template as:
template: $input-splitter -model IN1.$input-extension $input-extension
< IN > OUT
However, these lines are in contrast to the templates on line 320
(under split-indomain-source in [MML]) and line 949 (under split-input
in [EVALUATION]) defined as:
template: $input-splitter -model IN1.$input-extension < IN > OUT
We can't explain the above inconsistency. We are assuming that the
extra "$input-extension" right before "< IN" in lines 750 and 758
should be removed. Is that correct, or is there some other reason for
the extra $input-extension parameter that we're not seeing?
Thanks,
Lane
------------------------------
Message: 3
Date: Thu, 10 Oct 2013 21:23:16 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Inconsistency in experiment.meta
To: Lane Schwartz <dowobeha@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbh-TdoHdiBtZmewO5fjSSDJBVSEH=pYCQ0U2iUg5sYf6w@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Try running in a new working directory, your experiment was 4. It may have
picked up some bad moses.ini file from a previous experiment.
The training pipeline seems ok to me. I ran with the toy data all the way
through to reporting the BLEU score without problems.
I've added the my actual config and run command to the example directory
https://github.com/moses-smt/mosesdecoder/commit/6e82e11adf58576bbe37ed1203e6b74fd3cd42b9
(I used the same data for tuning and testing, rather than split the
training data. Not very correct but that shouldn't make a big difference).
On 10 October 2013 18:21, Lane Schwartz <dowobeha@gmail.com> wrote:
> One of the people in my lab has been attempting to add a custom
> component to EMS. In trying to do this, we have uncovered what appears
> to be an inconsistency in experiment.meta.
>
> Lines 750 (under split-input in [TUNING]) and 758 (under
> split-input-devtest in [TUNING]) defines the template as:
>
> template: $input-splitter -model IN1.$input-extension $input-extension
> < IN > OUT
>
>
> However, these lines are in contrast to the templates on line 320
> (under split-indomain-source in [MML]) and line 949 (under split-input
> in [EVALUATION]) defined as:
>
> template: $input-splitter -model IN1.$input-extension < IN > OUT
>
>
> We can't explain the above inconsistency. We are assuming that the
> extra "$input-extension" right before "< IN" in lines 750 and 758
> should be removed. Is that correct, or is there some other reason for
> the extra $input-extension parameter that we're not seeing?
>
> Thanks,
> Lane
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131010/3dc2a4b9/attachment-0001.htm
------------------------------
Message: 4
Date: Fri, 11 Oct 2013 01:54:50 +0100
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Hiero ems should die on decode error, but
doesn't
To: Lane Schwartz <dowobeha@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDBwunO+jmo+a2cnk6SkTrtw5e6BUxEH6jprh6Zk+Weiig@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
In lines 1351ff of experiment.perl, a number of patterns are define,
which detect
crashes of steps:
foreach my $pattern (@{$ERROR{&defined_step_id($i)}},
'error','killed','core dumped','can\'t read',
'no such file or directory','unknown option',
'died at','exit code','permission denied',
'segmentation fault','abort',
'no space left on device',
'can\'t locate', 'unrecognized option') {
If you want to define additional detector patterns for a specific step, you can
add these in experiment.meta:
See for instance in LM:factorize
factorize
in: tokenized-corpus
out: factorized-corpus
rerun-on-change: TRAINING:output-factors
default-name: lm/factored
pass-unless: factors
parallelizable: yes
error: can't open
error: incompatible number of words in factor
To your question about this particular crash of Moses: you seem to have defined
TranslationModel with num-features=1, but the phrase pairs in the table actually
do not have any feature values.
-phi
On Thu, Oct 10, 2013 at 6:13 PM, Lane Schwartz <dowobeha@gmail.com> wrote:
> I've been toying around again with ems. I know this should be easy,
> but for some reason I'm really having trouble getting the toy config
> changed to successfully run with a hiero model.
>
> I started out with scripts/ems/example/config.toy, and attempted to
> apply the changes described here:
> http://www.statmt.org/moses/?n=FactoredTraining.EMS#ntoc7. The
> resulting setup runs, but dies at the final REPORTING stage.
>
> I looked through the output for various steps, and it appears that the
> EVALUATION:decode step is dying with error message "Feature function
> TranslationModel1 specified 1 dense scores or weights. Actually has
> 0." As a result, the decoder failed to translate the requested
> sentences.
>
> I'm not an expert on ems, but I would have expected EMS to die at the
> EVALUATION:decode if that's where the first critical error occurs.
>
> That aside, I'd appreciate any thoughts on why ems might be dying. I'd
> like to try running some hiero experiments with ems. FWIW, running the
> config.toy phrase-based model works fine.
>
> Thanks,
> Lane
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
------------------------------
Message: 5
Date: Fri, 11 Oct 2013 02:02:19 +0100
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Inconsistency in experiment.meta
To: Lane Schwartz <dowobeha@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDB1zkU=rUxjUrAgT9-cW0wf-kOtSvEocEzKa6p0Kj1AAg@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
the additional $input-extension in lines 750 and 758 in fact do not do
anything, so they should be removed.
I updated this.
-phi
On Thu, Oct 10, 2013 at 6:21 PM, Lane Schwartz <dowobeha@gmail.com> wrote:
> One of the people in my lab has been attempting to add a custom
> component to EMS. In trying to do this, we have uncovered what appears
> to be an inconsistency in experiment.meta.
>
> Lines 750 (under split-input in [TUNING]) and 758 (under
> split-input-devtest in [TUNING]) defines the template as:
>
> template: $input-splitter -model IN1.$input-extension $input-extension
> < IN > OUT
>
>
> However, these lines are in contrast to the templates on line 320
> (under split-indomain-source in [MML]) and line 949 (under split-input
> in [EVALUATION]) defined as:
>
> template: $input-splitter -model IN1.$input-extension < IN > OUT
>
>
> We can't explain the above inconsistency. We are assuming that the
> extra "$input-extension" right before "< IN" in lines 750 and 758
> should be removed. Is that correct, or is there some other reason for
> the extra $input-extension parameter that we're not seeing?
>
> Thanks,
> Lane
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
------------------------------
Message: 6
Date: Thu, 10 Oct 2013 21:20:02 -0400
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] Inconsistency in experiment.meta
To: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZn24-XGk8YRitPaU2Ad+ucG4=8RcMQU-Ktau08kZX=igw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Thanks!
On Thu, Oct 10, 2013 at 9:02 PM, Philipp Koehn <pkoehn@inf.ed.ac.uk> wrote:
> Hi,
>
> the additional $input-extension in lines 750 and 758 in fact do not do
> anything, so they should be removed.
>
> I updated this.
>
> -phi
>
> On Thu, Oct 10, 2013 at 6:21 PM, Lane Schwartz <dowobeha@gmail.com> wrote:
>> One of the people in my lab has been attempting to add a custom
>> component to EMS. In trying to do this, we have uncovered what appears
>> to be an inconsistency in experiment.meta.
>>
>> Lines 750 (under split-input in [TUNING]) and 758 (under
>> split-input-devtest in [TUNING]) defines the template as:
>>
>> template: $input-splitter -model IN1.$input-extension $input-extension
>> < IN > OUT
>>
>>
>> However, these lines are in contrast to the templates on line 320
>> (under split-indomain-source in [MML]) and line 949 (under split-input
>> in [EVALUATION]) defined as:
>>
>> template: $input-splitter -model IN1.$input-extension < IN > OUT
>>
>>
>> We can't explain the above inconsistency. We are assuming that the
>> extra "$input-extension" right before "< IN" in lines 750 and 758
>> should be removed. Is that correct, or is there some other reason for
>> the extra $input-extension parameter that we're not seeing?
>>
>> Thanks,
>> Lane
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 84, Issue 15
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 84, Issue 15"
Post a Comment