Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Training error ("Terminate called after throwing an
instance of 'std::bad_alloc' ") (Hieu Hoang)
2. Binarising phrase table (Tin Pavelic)
3. Re: Binarising phrase table (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Sun, 28 Sep 2014 19:17:04 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Training error ("Terminate called after
throwing an instance of 'std::bad_alloc' ")
To: i.ramadan@saudisoft.com
Cc: 'moses-support' <moses-support@mit.edu>
Message-ID: <542850A0.3080705@gmail.com>
Content-Type: text/plain; charset="utf-8"
there are ways of making training run ok even with large amount of data.
However, we need to know exactly where the problem is. What the the
EXACT command you ran, including all arguments? Can you send me the
entire command line output that contains the error, not just the single
line with the error
On 28/09/14 15:07, Ihab Ramadan wrote:
>
> I was training the translation model the language model is done
> successfully, the command was train-model.perl
>
> It generates the corpus folder and start to make the other files in
> the giza.src-trg files using GIZA program, after a while it generates
> this error
>
> I agree with you that could be a run out of memory exception because
> the data I use this time is much bigger than the previous one but what
> you mean by ?something odd with the data?
>
> *From:*hieuhoang@gmail.com [mailto:hieuhoang@gmail.com] *On Behalf Of
> *Hieu Hoang
> *Sent:* Sunday, September 28, 2014 3:38 PM
> *To:* i.ramadan@saudisoft.com
> *Cc:* moses-support
> *Subject:* Re: [Moses-support] Training error ("Terminate called after
> throwing an instance of 'std::bad_alloc' ")
>
> what is the exact command you ran? Were you training the translation
> model or the language model? and can you provide more details on the
> output.
>
> it seems to have run out of memory. That shouldn't happen unless
> there's something odd with the data
>
> On 28 September 2014 11:02, Ihab Ramadan <i.ramadan@saudisoft.com
> <mailto:i.ramadan@saudisoft.com>> wrote:
>
> Dears,
>
> I got this error during the training process of some big parallel
> files ( 600 MB for each) ?Terminate called after throwing an instance
> of 'std::bad_alloc'?
>
> Note : I ran the clean script before training
>
> Any suggestions?
>
> Best Regards
>
> /Ihab Ramadan/| Senior Developer|Saudisoft <http://www.saudisoft.com/>
> - Egypt| *Tel * +2 02 330 320 37 Ext- 0| Mob+201007570826
> <tel:%2B201007570826> | Fax+20233032036 <tel:%2B20233032036> | *Follow
> us on *linked
> <http://www.linkedin.com/company/77017?trk=vsrp_companies_res_name&trkInfo=VSRPsearchId%3A1489659901402995947155%2CVSRPtargetId%3A77017%2CVSRPcmpt%3Aprimary>* |
> **ZA102637861*
> <https://www.facebook.com/pages/Saudisoft-Co-Ltd/289968997768973?ref_type=bookmark>* |
> **ZA102637858* <https://twitter.com/Saudisoft>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140928/d3f08289/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 1314 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140928/d3f08289/attachment-0003.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 1317 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140928/d3f08289/attachment-0004.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 1351 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140928/d3f08289/attachment-0005.gif
------------------------------
Message: 2
Date: Mon, 29 Sep 2014 02:22:16 +0200
From: Tin Pavelic <tin.pavelic@gmail.com>
Subject: [Moses-support] Binarising phrase table
To: moses-support@mit.edu
Message-ID: <5428A638.5040503@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Sorry to bother,
The first problem happened when i tried to tune the translation. I
followed all the steps, tokenized, and truecasted the data and ran the
mort-moses.pl. The result was phrase table with size of 0. The original
training data is about 1.2 Mil lines, and the tuning data was around
600k line. Is size the problem?
After i waited (quite some time) for the tuning process to finish and i
noticed it failed i went straight on to translation with only the
training part. so using moses.ini file from the "train/model" directory.
The loading as mentioned on the site was quite slow so i decided to
binarize it.
the output error is:
terminate called after throwing an instance of 'util::Exception'
what(): moses/PDTAimp.cpp:169 in void Moses::PDTAimp::Create(const
std::vector<long unsigned int>&, const std::vector<long unsigned int>&,
const string&, const std::vector<float>&) threw util::Exception'.
bin ttable does not exist
Aborted
Please help and thank you all in advance,
--
__________________________
Tin Pavelic
__________________________
------------------------------
Message: 3
Date: Mon, 29 Sep 2014 13:31:03 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Binarising phrase table
To: Tin Pavelic <tin.pavelic@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbh4WyRUQmykGJouq55uH46_xq1zuMCdFmieiWaiocE6HQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
if the phrase table size is 0, it's definitely a problem. All subsequent
steps will be wrong or they will crash, like you saw.
Is the empty pt the filtered version, or the original pt you created in the
training step?
On 29 September 2014 01:22, Tin Pavelic <tin.pavelic@gmail.com> wrote:
> Sorry to bother,
>
> The first problem happened when i tried to tune the translation. I
> followed all the steps, tokenized, and truecasted the data and ran the
> mort-moses.pl. The result was phrase table with size of 0. The original
> training data is about 1.2 Mil lines, and the tuning data was around
> 600k line. Is size the problem?
>
> After i waited (quite some time) for the tuning process to finish and i
> noticed it failed i went straight on to translation with only the
> training part. so using moses.ini file from the "train/model" directory.
> The loading as mentioned on the site was quite slow so i decided to
> binarize it.
>
> the output error is:
>
> terminate called after throwing an instance of 'util::Exception'
> what(): moses/PDTAimp.cpp:169 in void Moses::PDTAimp::Create(const
> std::vector<long unsigned int>&, const std::vector<long unsigned int>&,
> const string&, const std::vector<float>&) threw util::Exception'.
> bin ttable does not exist
> Aborted
>
>
> Please help and thank you all in advance,
> --
> __________________________
>
> Tin Pavelic
> __________________________
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140929/b814a869/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 95, Issue 46
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 95, Issue 46"
Post a Comment