Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Incremental Giza(Inc-giza) crashes with core dump when
doing Incremental training using MOSES EMS (Hegde, Sujay)
2. Re: Come work at the birthplace of the Moses decoder for a
year or more to bring MT to the next level! (Barry Haddow)
----------------------------------------------------------------------
Message: 1
Date: Tue, 26 May 2015 12:22:37 +0000
From: "Hegde, Sujay" <Sujay.Hegde@xerox.com>
Subject: Re: [Moses-support] Incremental Giza(Inc-giza) crashes with
core dump when doing Incremental training using MOSES EMS
To: Hieu Hoang <hieuhoang@gmail.com>, "moses-support@mit.edu"
<moses-support@mit.edu>
Cc: "Venkatapathy, Sriram \(Calendar\)"
<sriram.venkatapathy@xrce.xerox.com>, "MudaliarMudaliar, Preeti J"
<preeti.mudaliarmudaliar@xerox.com>
Message-ID:
<586EA7C483504E48870F5BF54319B6EC3A4815@USA7109MB006.na.xerox.net>
Content-Type: text/plain; charset="us-ascii"
HI Hieu-Hong,
Thank you for your reply.
We had used the baseline model created by MGiza , and then used inc-giza for incremental training.
So probably the file formats were incompatible.Hence inc-giza crashed.
But we are still struggling to do Incremental training using MGiza.
There is a separate post for the same.
Please let us know on the same.
Thanks and Regards,
Sujay
From: Hieu Hoang [mailto:hieuhoang@gmail.com]
Sent: Monday, May 25, 2015 10:23 AM
To: Hegde, Sujay; moses-support@mit.edu
Cc: Venkatapathy, Sriram (Calendar); MudaliarMudaliar, Preeti J
Subject: Re: [Moses-support] Incremental Giza(Inc-giza) crashes with core dump when doing Incremental training using MOSES EMS
seems like no-one knows. If you managed to figure it out, please tell us
On 20/05/2015 14:32, Hegde, Sujay wrote:
Hi Admin,
Inc-giza taken from : https://code.google.com/p/inc-giza-pp/
And manually compiled.
The command invoked by Moses EMS(actually train-model.perl ) is :
/mnt/hd1/git/mosesdecoder/training-tools/GIZA++ -CoocurrenceFile /mnt/hd1/training/working-dir-en-es/training/giza.5/en-es.cooc -c /mnt/hd1/training/working-dir-en-es/training/prepared.5/en-es-int-train.snt -hmmdumpfrequency 5 -hmmiterations 5 -m1 5 -m2 0 -m3 0 -m4 0 -m5 0 -model1dumpfrequency 0 -model2dumpfrequency 0 -model345dumpfrequency 0 -model4smoothfactor 0.4 -nodumps 0 -nsmooth 4 -o /mnt/hd1/training/working-dir-en-es/training/giza.5/en-es -oldAlPrbs /mnt/hd1/training/working-dir-en-es/training/giza.1/en-es.hhmm.5 -oldTrPrbs /mnt/hd1/training/working-dir-en-es/training/giza.1/en-es.thmm.5 -onlyaldumps 1 -p0 0.999 -s /mnt/hd1/training/working-dir-en-es/training/prepared.5/es.vcb -step_k 1 -t /mnt/hd1/training/working-dir-en-es/training/prepared.5/en.vcb
The baseline model used is 1.
It crashes with below output on console.It is a HEAP CORRPUTION.
-----------
Model1: Iteration 1
number of French (target) words = 1255
initial unifrom prob = 0.000796813
Model1: (1) TRAIN CROSS-ENTROPY 16.2367 PERPLEXITY 77219
Model1: (1) VITERBI TRAIN CROSS-ENTROPY 20.8406 PERPLEXITY 1.87773e+06
Model 1 Iteration: 1 took: 0 seconds
-----------
Model1: Iteration 2
number of French (target) words = 1255
initial unifrom prob = 0.000796813
Model1: (2) TRAIN CROSS-ENTROPY 5.75346 PERPLEXITY 53.9466
Model1: (2) VITERBI TRAIN CROSS-ENTROPY 9.11781 PERPLEXITY 555.562
Model 1 Iteration: 2 took: 0 seconds
-----------
Model1: Iteration 3
number of French (target) words = 1255
initial unifrom prob = 0.000796813
Model1: (3) TRAIN CROSS-ENTROPY 5.61082 PERPLEXITY 48.868
Model1: (3) VITERBI TRAIN CROSS-ENTROPY 8.86927 PERPLEXITY 467.646
Model 1 Iteration: 3 took: 0 seconds
-----------
Model1: Iteration 4
number of French (target) words = 1255
initial unifrom prob = 0.000796813
Model1: (4) TRAIN CROSS-ENTROPY 5.51417 PERPLEXITY 45.7014
Model1: (4) VITERBI TRAIN CROSS-ENTROPY 8.64774 PERPLEXITY 401.078
Model 1 Iteration: 4 took: 0 seconds
-----------
Model1: Iteration 5
number of French (target) words = 1255
initial unifrom prob = 0.000796813
Model1: (5) TRAIN CROSS-ENTROPY 5.44065 PERPLEXITY 43.4309
Model1: (5) VITERBI TRAIN CROSS-ENTROPY 8.44812 PERPLEXITY 349.25
Model 1 Iteration: 5 took: 0 seconds
Entire Model1 Training took: 0 seconds
NOTE: I am doing iterations with the HMM model!
Loading HMM alignments from file.
*** glibc detected *** /mnt/hd1/git/mosesdecoder/training-tools/GIZA++: malloc(): memory corruption: 0x00000000011c4470 ***
======= Backtrace: =========
[0x524d22]
[0x526f4d]
[0x527b7b]
[0x4f0fcd]
[0x4d2829]
[0x4d329a]
[0x4d340b]
[0x495c39]
[0x48bdf2]
[0x42822f]
[0x4294ba]
[0x505e66]
[0x4001e9]
======= Memory map: ========
00400000-005d9000 r-xp 00000000 08:10 6030184 /mnt/hd1/git/mosesdecoder/training-tools/GIZA++
007d8000-007db000 rw-p 001d8000 08:10 6030184 /mnt/hd1/git/mosesdecoder/training-tools/GIZA++
007db000-007f2000 rw-p 00000000 00:00 0
00fc4000-01438000 rw-p 00000000 00:00 0 [heap]
7f3054000000-7f3054028000 rw-p 00000000 00:00 0
7f3054028000-7f3058000000 ---p 00000000 00:00 0
7f3059db4000-7f3059db5000 rw-p 00000000 00:00 0
7fff689b0000-7fff689c5000 rw-p 00000000 00:00 0 [stack]
7fff689ff000-7fff68a00000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
Aborted (core dumped)
Also core analysis using GDB yields:
Core was generated by `/mnt/hd1/git/mosesdecoder/training-tools/GIZA++ -CoocurrenceFile /mnt/hd1/train'.
Program terminated with signal 6, Aborted.
#0 0x0000000000547ee5 in raise ()
(gdb) where
#0 0x0000000000547ee5 in raise ()
#1 0x000000000050e635 in abort ()
#2 0x000000000051f365 in __libc_message ()
#3 0x0000000000524d22 in malloc_printerr ()
#4 0x0000000000526f4d in _int_malloc ()
#5 0x0000000000527b7b in malloc ()
#6 0x00000000004f0fcd in operator new(unsigned long) ()
#7 0x00000000004d2829 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) ()
#8 0x00000000004d329a in char* std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) ()
#9 0x00000000004d340b in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) ()
#10 0x0000000000495c39 in HMMTables<int, WordClasses>::readJumps(std::basic_istream<char, std::char_traits<char> >&) ()
#11 0x000000000048bdf2 in hmm::load_table(char const*) ()
#12 0x000000000042822f in StartTraining(int&) ()
#13 0x00000000004294ba in main ()
Is it a problem with inc-Giza or could it be a problem with the libs being used.
Regards,
Sujay
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150526/1fb4ff4d/attachment-0001.htm
------------------------------
Message: 2
Date: Tue, 26 May 2015 14:23:49 +0100
From: Barry Haddow <bhaddow@inf.ed.ac.uk>
Subject: Re: [Moses-support] Come work at the birthplace of the Moses
decoder for a year or more to bring MT to the next level!
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <556473E5.3040809@inf.ed.ac.uk>
Content-Type: text/plain; charset="windows-1252"
Hi All
The deadline for this has been extended to June 2nd. We are looking for
a research associate to join the Edinburgh SMT group, initially for 12
months, but could be extended if current funding applications are
successful.
The advert mentions specific projects, but we can be quite flexible. We
have several projects going on at the moment
(http://www.statmt.org/ued/?n=Public.Projects) and are looking for
someone interested in MT with strong software development skills.
Feel free to drop myself or Uli a line if you want to know more,
cheers - Barry
On 13/05/15 01:33, Ulrich Germann wrote:
> We have an open position for a research associate in machine
> translation for 12 months initially, with the possibility of extension
> (depending on funding).
>
> https://www.vacancies.ed.ac.uk/pls/corehrrecruit/erq_jobspec_version_4.jobspec?p_id=033241
>
> - Uli
>
>
> --
> Ulrich Germann
> Senior Researcher
> School of Informatics
> University of Edinburgh
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150526/3d317045/attachment.htm
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150526/3d317045/attachment.bat
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 103, Issue 62
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 103, Issue 62"
Post a Comment