Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Threads and Cores when using Moses (liling tan)
2. Several Issues with Baseline and EMS (Vincent Nguyen)
3. Generating Segment Level BLEU, NIST and METEOR scores (liling tan)
4. Re: Generating Segment Level BLEU, NIST and METEOR scores
(Matthias Huck)
----------------------------------------------------------------------
Message: 1
Date: Wed, 2 Sep 2015 18:40:09 +0200
From: liling tan <alvations@gmail.com>
Subject: [Moses-support] Threads and Cores when using Moses
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAKzPaJLfjQcw0U3XjJym+upsRq7ACpBg8jyopVWJX9+1D_uxVA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Moses devs/users,
Is there a way to control how many threads and how many cores Moses use?
`moses/scripts/training/train-model.perl` has a --cores option while
`moses/scripts//training/mert-moses.pl` has a --threads option.
Does the mert-moses.pl --thread option refer to the no. of threads that
MERT will take? Is there a way to control how many cores the no. of threads
will be running on?
And in train-model.perl the Mgiza option allows --mgiza-cpus option, does
that refer to the no. of threads assigned or the cores mgiza will take?
In some experiments the optimal no. of threads assigned to an algorithm
might depend on the algorithm's complexity. Is there any experimental
evidence for the optimal no. of threads to assign when running MERT? Or
when running MGIZA?
Pardon my lack of knowledge in cores and threads. Thank you in advance for
the tips on constraining Moses on the server.
Best Regards,
Liling
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150902/3fb7e2aa/attachment-0001.html
------------------------------
Message: 2
Date: Wed, 2 Sep 2015 18:39:00 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: [Moses-support] Several Issues with Baseline and EMS
To: moses-support@mit.edu
Message-ID: <55E72624.5090401@neuf.fr>
Content-Type: text/plain; charset="windows-1252"
if you're new to linux you will fight for ever.
I would probably go to Slate instead for sure.
Le 02/09/2015 17:34, Anita Pal a ?crit :
> For the time being, I'm trying to finish building the baseline system.
> I've just been following the commands as listed on the Moses website.
>
> It's still not running. I tried out countless variants -- it's still
> telling me to Set IRSTLM environment variable with path to irstlm.
>
> liam@liam-Aspire-E1-571G:~$ export=$HOME/irstlm;
> ~/irstlm/bin/build-lm.sh \
> > -i news-commentary-v8.fr-en.sb.en \
> > -t ./tmp -p -s improved-kneser-ney -o news-commentary-v8.fr-en.lm.en
> ~/irstlm/bin/compile-lm \
> > --text=yes \
> > news-commentary-v8.fr-en.lm.en.gz \
> > news-commentary-v8.fr-en.arpa.en
> Set IRSTLM environment variable with path to irstlm
> liam@liam-Aspire-E1-571G:~$
>
> --> this was the last command I entered.
>
> The full (recommended) command via the website is:
>
> mkdir ~/lm
> cd ~/lm
> ~/irstlm/bin/add-start-end.sh \
> < ~/corpus/news-commentary-v8.fr-en.true.en \
> > news-commentary-v8.fr-en.sb.en
> export IRSTLM=$HOME/irstlm; ~/irstlm/bin/build-lm.sh \
> -i news-commentary-v8.fr-en.sb.en \
> -t ./tmp -p -s improved-kneser-ney -o news-commentary-v8.fr-en.lm.en
> ~/irstlm/bin/compile-lm \
> --text=yes \
> news-commentary-v8.fr-en.lm.en.gz \
> news-commentary-v8.fr-en.arpa.en
>
> Has anyone got a sample command of how they got this to run?
>
>
>
> On Wed, Sep 2, 2015 at 11:59 AM, Jeroen Vermeulen
> <jtv@precisiontranslationtools.com
> <mailto:jtv@precisiontranslationtools.com>> wrote:
>
> Yes, the semicolon ends one command and starts another.
>
> Don't set HOME. The shell does that for you. If you want a
> different setting than your actual home directory, give it a
> different name or you might confuse programs you run from that shell.
>
>
> Jeroen
>
> On September 2, 2015 1:26:43 AM GMT+07:00, Anita Pal
> <fujitahibaraki@gmail.com <mailto:fujitahibaraki@gmail.com>> wrote:
>
> Hey!
>
> I'm really, really new to Linux and have no idea how to set
> /change the config file when it comes to running the
> experiments as described here
> (http://www.statmt.org/moses/?n=moses.baseline)
>
> home-dir = /home/liam/
>
> working-dir =/home/liam/working/experiments
> moses-src-dir = /home/liam/mosesdecoder
> moses-script-dir = home/liam/working/experiments (this is
> where config is located?)
> moses-bin-dir = /home/liam/mosesdecoder/bin
> external-bin-dir =/home/liam/mosesdecoder/tools
> data-dir =/home/liam/corpus
> train-dir =/home/liam/corpus/training
> dev-dir = /home/liam/corpus/dev
> irstlm-dir =/home/liam/irstlm/bin
>
> Is this correct? I really have no idea ):
>
> Because I just keep getting errors no matter what I do. I have
> the same problem when it comes to training the language model
> via IRSTLM. For example:
>
> export IRSTLM=$HOME/irstlm; ~/irstlm/bin/build-lm.sh \ -i
> news-commentary-v8.fr-en.sb.en \ -t ./tmp -p -s
> improved-kneser-ney -o news-commentary-v8.fr-en.lm.en
> ~/irstlm/bin/compile-lm \ --text=yes \
> news-commentary-v8.fr-en.lm.en.gz \
> news-commentary-v8.fr-en.arpa.en
>
> I assume this is a separate command. I keep getting errors
> though I do set HOME=/home/liam/irstlm.
>
> What am I doing wrong?
>
>
>
> ------------------------------------------------------------------------
>
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150902/b152e7f0/attachment-0001.html
------------------------------
Message: 3
Date: Wed, 2 Sep 2015 21:47:51 +0200
From: liling tan <alvations@gmail.com>
Subject: [Moses-support] Generating Segment Level BLEU, NIST and
METEOR scores
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAKzPaJ+Yk0UQstBU2M8ZpJYW5TLakcuK7FmK_g32-uBTyBmkKA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Dear Moses devs/users,
Is there any script in moses or other MT libraries that can generate the
segment level BLEU, NIST and METEOR scores for each sentence in the test
set?
Best Regards,
Liling
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150902/328b35e8/attachment-0001.html
------------------------------
Message: 4
Date: Wed, 02 Sep 2015 23:05:44 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] Generating Segment Level BLEU, NIST and
METEOR scores
To: liling tan <alvations@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <1441231544.1857.25.camel@inf.ed.ac.uk>
Content-Type: text/plain; charset="UTF-8"
Hi Liling,
This tool calculates sentence-level BLEU scores (smoothed via
incrementing the n-gram counts by 1):
bin/sentence-bleu
Make sure that you provide the hypothesis and reference files in an
appropriately processed way. The tool doesn't apply any tokenization or
remove any markup internally.
You find the source code in
mert/sentence-bleu.cpp
if you need it.
There's also a special version for use with n-best lists:
bin/sentence-bleu-nbest
Cheers,
Matthias
On Wed, 2015-09-02 at 21:47 +0200, liling tan wrote:
> Dear Moses devs/users,
>
>
> Is there any script in moses or other MT libraries that can generate
> the segment level BLEU, NIST and METEOR scores for each sentence in
> the test set?
>
>
> Best Regards,
> Liling
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 107, Issue 10
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 107, Issue 10"
Post a Comment