Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: BLEU score difference about 0.13 for one dataset is
normal? (Davood Mohammadifar)
2. segmentation fault when using -sctype option (Arefeh Kazemi)
3. Re: how to run moses through shell script in linux (Apurva Joshi)
4. problem while tuning (Khyati Gupta)
----------------------------------------------------------------------
Message: 1
Date: Wed, 14 Oct 2015 05:50:04 +0000
From: Davood Mohammadifar <davood_mf@hotmail.com>
Subject: Re: [Moses-support] BLEU score difference about 0.13 for one
dataset is normal?
To: Tom Hoar <tahoar@precisiontranslationtools.com>, Moses Support
<moses-support@mit.edu>, "michael.j.denkowski@gmail.com"
<michael.j.denkowski@gmail.com>
Message-ID: <SNT150-W927CDCE97F8984C28CFA558C3F0@phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"
Thanks Michael for the paper and thanks Tom.
Based on the paper, one solution is replication of MERT and testing at least three times.
My ideas have subtle effects on BLUE. Do you recommend me run MERT and testing three times or more? should i increase the number of sentences for tuning?
my dataset for Persian to English includes:
Training: about 240000 sentences
Tune: 1000 sentences
Test: 1000 sentences
From: tahoar@precisiontranslationtools.com
Date: Sun, 11 Oct 2015 12:53:37 +0700
To: moses-support@mit.edu
Subject: Re: [Moses-support] BLEU score difference about 0.13 for one dataset is normal?
Yes. Each tuning with the same test set will give you small variations in the final BLEU. Yours looks like they're in a normal range.
Date: Sun, 11 Oct 2015 04:23:56 +0000
From: Davood Mohammadifar <davood_mf@hotmail.com>
Subject: [Moses-support] BLEU score difference about 0.13 for one
dataset is normal?
To: Moses Support <moses-support@mit.edu>
Hello every one
I noticed different BLEU scores for same dataset. Also the difference is not so much and is about 0.13.
I trained my dataset and tuned development set for Persian-English translation. after testing, the score was 21.95. For second time i did the same process and obtained 21.82. (my tools were mgiza, mert, ...)
is this difference normal?
My system:
CPU: Core i7-4790K
RAM: 16GB
OS: ubuntu 12.04
Thanks
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151014/977db010/attachment-0001.html
------------------------------
Message: 2
Date: Wed, 14 Oct 2015 10:37:09 +0330
From: Arefeh Kazemi <kazemina@dcu.ie>
Subject: [Moses-support] segmentation fault when using -sctype option
To: moses-support@mit.edu
Message-ID:
<CAB2b+QVJnLKk8nqURmuBg_-BV_r=h9cuftTD79EVsKY9Je1ESw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
when I use -sctype option in tuning command, after the first iteration of
tuning , It gives "segmentation fault" error.
the exact command that I use:
nohup nice $SCRIPTS_ROOTDIR/training/mert-moses.pl --mertargs="--sctype
KENDALL,BLEU --scconfig weights:0.6+0.4" m.en m.fa \
$MOSESCHART --decoder-flags="-threads all" $WorkingDir/moses.ini \
--mertdir ~/mosesdecoder/bin/ \
--no-filter-phrase-table > mert.out
the error :
Exit code: 139
ERROR: Failed to run '../mert-work/extractor.sh'
error in extract.err file:
extractor.sh: line 3: 22321 Segmentation fault
can anyone help me to solve this?
Thanks
Arefeh
--
*Email Disclaimer"This e-mail and any files transmitted with it are confidential and are intended solely for use by the addressee. Any unauthorised dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received this e-mail in error, please notify the sender and delete the message. Any views or opinions presented in this e-mail may solely be the views of the author and cannot be relied upon as being those of Dublin City University. E-mail communications such as this cannot be guaranteed to be virus-free, timely, secure or error-free and Dublin City University does not accept liability for any such matters or their consequences. Please consider the environment before printing this e-mail."S?anadh R?omhphoist"T? an r?omhphost seo agus aon chomhad a sheoltar leis faoi r?n agus is lena ?s?id ag an seola? agus sin amh?in ?. T? cosc ioml?n ar scaipeadh, dh?ileadh n? ch?ipe?il neamh?daraithe ar an teachtaireacht seo agus ar ao!
n cheangalt?n at? ag dul leis. M? t? an r?omhphost seo faighte agat tr? dhearmad cuir sin in i?l le do thoil don seolt?ir agus scrios an teachtaireacht. D?fh?adfadh s? gurb iad tuairim? an ?dair agus sin amh?in at? in aon tuairim? no dearctha? at? curtha i l?thair sa r?omhphost seo agus n?or ch?ir glacadh leo mar thuairim? n? dhearctha? Ollscoil Chathair Bhaile ?tha Cliath. N? ghlactar leis go bhfuil cumars?id r?omhphoist den s?rt seo saor ? v?reas, in am, sl?n, n? saor ? earr?id agus n? ghlacann Ollscoil Chathair Bhaile ?tha Cliath le dliteanas in aon ch?s den s?rt sin n? as aon iarmhairt a d?eascr?dh astu. Cuimhnigh ar an timpeallacht le do thoil sula gcuireann t? an r?omhphost seo i gcl?."*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151014/8f6f0938/attachment-0001.html
------------------------------
Message: 3
Date: Wed, 14 Oct 2015 14:36:27 +0530
From: Apurva Joshi <apurvajoshi1992@gmail.com>
Subject: Re: [Moses-support] how to run moses through shell script in
linux
To: Philipp Koehn <phi@jhu.edu>, moses-support@mit.edu
Message-ID:
<CAMfCXbX+gSwXvamJjTpECVL0d47+0f+ntN4d_HQTWJzDspP1og@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
hi philipp
i already knw above command...i executed this command through shell
script...but if i use file....then moses will converts only those words
which are in file....but in real time when i want to input any no of
english words...then in that case file is not good option....in my previous
posts...i have mentioned that when i run moses.ini....it loads its big
table....after loading , my linux prompt goes out..(it enters into
moses...)....it asks for input from user...we give it...nd moses gives us
translation...again it waits for next ip....this process conitinues till i
press control + c to quit it and go back to my linux prompt...
in file case...using above command, it will give translation of only words
in file...then it will quit...and i come back to my linux prompt...bcoz it
doesnt waits for next ip conitnously....nw suppose i created another txt
file for new more words....then again...it will loads...and translate only
words in file....so guyzzz
finally.....what i have decided is....to use multi threading concept.....
i will write following logic in script file
generate new terminal ...load moses.ini...===> ..after this new terminal
will be generated and moses will load there and then linux prompt will
gone...as i said above this moses will waits for ip word(eg "hello")....
now i will give this "hello" from parent terminal to child terminal ....so
scenario is m giving ip words from parent ...and child is accepting it in
real time nd will gv conversion.....
in short...i want to avoid loading of moses for each file....
plzz guyzz help....
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151014/aa6415d5/attachment-0001.html
------------------------------
Message: 4
Date: Wed, 14 Oct 2015 10:08:40 +0000 (UTC)
From: Khyati Gupta <gkhyati@rocketmail.com>
Subject: [Moses-support] problem while tuning
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<1705332871.105479.1444817320774.JavaMail.yahoo@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"
There is error in running tuning process.
Exact command used is-
~/SMT/mosesdecoder/scripts/training/mert-moses.pl ~/SMT/Corpus/tuning.hi ~/SMT/Corpus/tuning.en ~/SMT/mosesdecoder/bin/moses ~/SMT/working/train/model/moses.ini --mertdir? ~/SMT/mosesdecoder/bin --rootdir ~/SMT/mosesdecoder/scripts
Error is-
Using SCRIPTS_ROOTDIR: /root/SMT/mosesdecoder/scripts
Assuming the tables are already filtered, reusing filtered/moses.ini
Asking moses for feature names and values from filtered/moses.ini
Executing: /root/SMT/mosesdecoder/bin/moses? -config filtered/moses.ini -show-weights
exec: /root/SMT/mosesdecoder/bin/moses? -config filtered/moses.ini -show-weights
Executing: /root/SMT/mosesdecoder/bin/moses? -config filtered/moses.ini -show-weights > ./features.list 2> /dev/null
Exit code: 1
ERROR: Failed to run '/root/SMT/mosesdecoder/bin/moses? -config filtered/moses.ini -show-weights'. at /root/SMT/mosesdecoder/scripts/training/mert-moses.pl line 1744.
[root@urdu-wali training]# /root/SMT/mosesdecoder/bin/moses -f string-to-tree/moses.ini < string-to-tree/in > out.stt
bash: string-to-tree/in: No such file or directory
Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151014/894814f2/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 108, Issue 49
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 108, Issue 49"
Post a Comment