Moses-support Digest, Vol 82, Issue 52

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Check scores.size() == indexes.second - indexes.first
failed (Hieu Hoang)
2. Problem in using MML (Hassan Sajjad)


----------------------------------------------------------------------

Message: 1
Date: Fri, 30 Aug 2013 18:56:53 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Check scores.size() == indexes.second -
indexes.first failed
To: moses-support@mit.edu
Message-ID: <5220DCE5.2000705@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

can you send me your moses.ini file and the tuning stderr file

ta

On 30/08/2013 18:50, Eleftherios Avramidis wrote:
> Hi all,
>
> the latest development version of the decoder (checked out yesterday)
> fails to run with 3 kenlm language models during tuning. I am not sure
> that this is the issue.
>
> Translating line 0 i
> Translating line 0 in thread id 140521597654784
> reading bin ttable
> size of OFF_T 8
> binary phrasefile loaded, default OFF_T: -1
> binary file loaded, default OFF_T: -1
> Check scores.size() == indexes.second - indexes.first failed in ./moses/ScoreComponentCollection.h:235
> Aborted (core dumped)
> Exit code: 134
> The decoder died. CONFIG WAS -weight-overwrite 'LM0= 0.089286 LM1= 0.089286 WordPenalty0= -0.178571 PhrasePenalty0= 0.035714 LM2= 0.089286 TranslationModel0= 0.035714 0.035714 0.035714 0.035714 Distortion0= 0.053571 LexicalReordering0= 0.053571 0.053571 0.053571 0.053571 0.053571 0.053571'
> cp: cannot stat `/share/taraxu/systems/bmmt/20130812_MAN/tuning/tmp.6/moses.ini': No such file or directory
>
> best
> Lefteris
>
>
>



------------------------------

Message: 2
Date: Sat, 31 Aug 2013 15:36:16 +0300
From: Hassan Sajjad <sajjad@ims.uni-stuttgart.de>
Subject: [Moses-support] Problem in using MML
To: Moses-support@mit.edu
Message-ID:
<CAOiX71Z6_-WRG+LxcXOUZU3_y7y9RkSxRDMbGph-pAUhSpfJ6A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I am trying to use MML but it's crashing at the
TRAINING_mml-filter-before-wa step. I could not resolve the problem. The
error and conf entries are copied here.

The corpus-mml-score.3 contains lines equal to my in-domain data and have
score 99999 on all lines. Is this correct?

Thank you,

Regards,
Hassan

------------------------------------------------------------------------------
/work/moses-2013-07-10/scripts/ems/support/mml-filter.py
/training/corpus-mml.3.ini
2013-08-31 12:29:57,126 Loading configuration from
/training/corpus-mml.3.ini
2013-08-31 12:29:57,128 Configuration:
general:strategy = Score
general:source_language = ar
general:target_language = en
general:input_stem = /training/corpus.1
general:output_stem = /training/corpus-mml.3
general:domain_file = /model/domains.3
general:domain_file_out = /training/corpus-mml.3
score:score_file = /training/corpus-mml-score.3
score:proportion = 0.9

2013-08-31 12:29:57,170 Retaining at least 0 entries and ignoring 149244
Traceback (most recent call last):
File "/work/moses-2013-07-10/scripts/ems/support/mml-filter.py", line
156, in <module>
main()
File "/work/moses-2013-07-10/scripts/ems/support/mml-filter.py", line
111, in main
strategy = strategy_class(config)
File "/work/moses-2013-07-10/scripts/ems/support/mml-filter.py", line 72,
in __init__
[float(line[:-1]) for line in open(self.score_file)],
reverse=True)[ignore_count + count]
IndexError: list index out of range
~
------------------------------------------------------------------------

Here are the entries in the conf file:

[MML]

### specifications for language models to be trained
#

lm-training = $srilm-dir/ngram-count
lm-settings = "-interpolate -kndiscount -unk"
lm-binarizer = $moses-src-dir/bin/build_binary
lm-query = $moses-src-dir/bin/query
order = 5
type = 8

raw-indomain-source = $training/train.$pair-extension.$input-extension
raw-indomain-target = $training/train.$pair-extension.$output-extension

outdomain-stem = /adapt/un.$pair-extension.utf8.ng.clean
settings = "--line-count 100000"


In TRAINING

### filtering some corpora with modified Moore-Lewis
# specify corpora to be filtered and ratio to be kept, either before or
after word alignment
mml-filter-corpora = /adapt/un.$pair-extension.utf8.ng.clean
mml-before-wa = "-proportion 0.9"
#mml-after-wa = "-proportion 0.9"

### domain adaptation settings
# options: sparse, any of: indicator, subset, ratio
domain-features = "subset"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20130831/2ae0fc56/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 82, Issue 52
*********************************************

Tweet