Moses-support Digest, Vol 107, Issue 51

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Existing documentation to use own language models (Arturo Argueta)
2. experiment.perl, processPhraseTableMin and threads option
(Tomasz Gawryl)
3. is there a way to remove a bad entry in the phrase table ?
(Vincent Nguyen)


----------------------------------------------------------------------

Message: 1
Date: Mon, 21 Sep 2015 14:24:15 -0400
From: Arturo Argueta <arturoargueta@gmail.com>
Subject: [Moses-support] Existing documentation to use own language
models
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAN1wo9gBZEXgiCPGwJiWe5a=mtGjJcLb8QbYDgRe+8RooHX7Hg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

I was wondering if there's any documentation out there on how to
use/integrate a self-made LM on moses.
Any pointers will be appreciated.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150921/20720ac3/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 22 Sep 2015 12:16:48 +0200
From: "Tomasz Gawryl" <tomasz.gawryl@skrivanek.pl>
Subject: [Moses-support] experiment.perl, processPhraseTableMin and
threads option
To: <moses-support@mit.edu>
Message-ID: <033901d0f51f$cad40bf0$607c23d0$@gawryl@skrivanek.pl>
Content-Type: text/plain; charset="us-ascii"

Hi,

I'm trying to add this line to config:

ttable-binarizer = "$moses-bin-dir/processPhraseTableMin -threads all"

This option is supported by processPhraseTableMin
command(https://github.com/moses-smt/mosesdecoder/blob/master/misc/processPh
raseTableMin.cpp line 24).

/home/moses/src/mosesdecoder/scripts/training/binarize-model.perl
/home/moses/working/experiments/model/moses.ini.1 /home/mose
s/working/experiments/model/moses.bin.ini.3 -Binarizer
/home/moses/src/mosesdecoder/bin/processPhraseTableMin -threads all

But it produces error "Unknown option: threads" in file
TRAINING_binarize-config.3.STDERR (and stops training).

I removed this option but it seems that such case it uses only one thread:

moses 1470 113 10.8 4255200 3448392 pts/13 Sl 11:05 66:03
/home/moses/src/mosesdecoder/bin/processPhraseTableMin -in
/home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0-
0.1.1.gz.sorted -out
/home/moses/working/experiments/model/moses.bin.ini.5.tables/phrase-table.0-
0.1.1 -nscores 4 -threads 1

I know that my server is able to run around 16 threads (and indeed does
during former steps).

What can I do to improve this step to use more threads?

Regards,
TG





------------------------------

Message: 3
Date: Tue, 22 Sep 2015 16:05:28 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: [Moses-support] is there a way to remove a bad entry in the
phrase table ?
To: moses-support <moses-support@mit.edu>
Message-ID: <56016028.6000400@neuf.fr>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi,

I was wondering if after an analysis of the BLEU-Annotation file we
realize that there must be a bad entry in the phrase table,
we could remove it manually or in some other ways ?

Gracias.
V.


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 107, Issue 51
**********************************************

Tweet