Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Segmentation Fault (Hieu Hoang)
2. Re: Segmentation Fault (Jasneet Sabharwal)
3. Re: Segmentation Fault (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Fri, 12 Feb 2016 23:39:50 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Segmentation Fault
To: Jasneet Sabharwal <jasneet.sabharwal@sfu.ca>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <56BE6D46.6010706@gmail.com>
Content-Type: text/plain; charset="windows-1252"
you can run the decoder
./moses -v 3
however, you should put debugging messages in your feature functions to
find out where the problem is. It looks like its in the Load() method so
add lots of debugging message in there and all functions it calls
On 12/02/16 23:34, Jasneet Sabharwal wrote:
> Thanks Hieu for your reply.
>
> Is it possible to do a verbose output of what?s happening, so that I
> can identify when it?s going out of memory? I?m only running it for
> 1928 sentences. I have almost 170gb of free memory and additional
> 400gb memory in buffer.
>
> Thanks,
> Jasneet
>
>> On Feb 12, 2016, at 2:36 PM, Hieu Hoang <hieuhoang@gmail.com
>> <mailto:hieuhoang@gmail.com>> wrote:
>>
>> looks like it's run out of memory.
>>
>> On 11/02/16 23:23, Jasneet Sabharwal wrote:
>>> Hi,
>>>
>>> I was adding a new feature function in Moses
>>> (https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp).
>>> It works fine when I test it for 1-2 sentences, but when I?m trying
>>> to tune my parameters, I?m getting segmentation faults or sometimes
>>> it is bad_alloc. Following was one of the commands that was executed
>>> during the tuning process which caused the Segmentation Fault or
>>> bad_alloc:
>>>
>>> moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite
>>> 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered=
>>> 0.075758 WordPenalty0= -0.151515 PhrasePenalty0= 0.030303
>>> CoarseBiLMClustered= 0.075758 TranslationModel0= 0.030303 0.030303
>>> 0.030303 0.030303 Distortion0= 0.045455 CoarseLM1600= 0.075758
>>> LexicalReordering0= 0.045455 0.045455 0.045455 0.045455 0.045455
>>> 0.045455' -n-best-list run1.best100.out 100 distinct -input-file
>>> tune.word.lc.cn <http://tune.word.lc.cn/>
>>>
>>> The log is enclosed in this email.
>>>
>>> Any pointers would be very useful.
>>>
>>> Thanks,
>>> Jasneet
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>> --
>> Hieu Hoang
>> http://www.hoang.co.uk/hieu
>
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160212/723b8437/attachment-0001.html
------------------------------
Message: 2
Date: Fri, 12 Feb 2016 15:56:40 -0800
From: Jasneet Sabharwal <jasneet.sabharwal@sfu.ca>
Subject: Re: [Moses-support] Segmentation Fault
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <140E58A9-A250-4028-B91F-819355DF0ADD@sfu.ca>
Content-Type: text/plain; charset="windows-1252"
Thanks, will give that a try.
Also, is it possible to get the value of feature name inside the feature function. I?m specifically talking about ?name? parameter in moses.ini. I?m running multiple copies of my feature function with different parameter as follows:
CoarseBiLM name=CoarseBiLM tgtWordId...
CoarseBiLM name=CoarseLM100 tgtWordId?
CoarseBiLM name=CoarseLM1600 tgtWordId...
CoarseBiLM name=CoarseBiLMWithoutClustering tgtWordId?
Thanks,
Jasneet
> On Feb 12, 2016, at 3:39 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
> you can run the decoder
> ./moses -v 3
> however, you should put debugging messages in your feature functions to find out where the problem is. It looks like its in the Load() method so add lots of debugging message in there and all functions it calls
>
> On 12/02/16 23:34, Jasneet Sabharwal wrote:
>> Thanks Hieu for your reply.
>>
>> Is it possible to do a verbose output of what?s happening, so that I can identify when it?s going out of memory? I?m only running it for 1928 sentences. I have almost 170gb of free memory and additional 400gb memory in buffer.
>>
>> Thanks,
>> Jasneet
>>
>>> On Feb 12, 2016, at 2:36 PM, Hieu Hoang < <mailto:hieuhoang@gmail.com>hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>> wrote:
>>>
>>> looks like it's run out of memory.
>>>
>>> On 11/02/16 23:23, Jasneet Sabharwal wrote:
>>>> Hi,
>>>>
>>>> I was adding a new feature function in Moses (https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp>). It works fine when I test it for 1-2 sentences, but when I?m trying to tune my parameters, I?m getting segmentation faults or sometimes it is bad_alloc. Following was one of the commands that was executed during the tuning process which caused the Segmentation Fault or bad_alloc:
>>>>
>>>> moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered= 0.075758 WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 CoarseBiLMClustered= 0.075758 TranslationModel0= 0.030303 0.030303 0.030303 0.030303 Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0= 0.045455 0.045455 0.045455 0.045455 0.045455 0.045455' -n-best-list run1.best100.out 100 distinct -input-file tune.word.lc.cn <http://tune.word.lc.cn/>
>>>>
>>>> The log is enclosed in this email.
>>>>
>>>> Any pointers would be very useful.
>>>>
>>>> Thanks,
>>>> Jasneet
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>
>>> --
>>> Hieu Hoang
>>> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu>
>
> --
> Hieu Hoang
> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160212/38e8f818/attachment-0001.html
------------------------------
Message: 3
Date: Fri, 12 Feb 2016 23:58:40 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Segmentation Fault
To: Jasneet Sabharwal <jasneet.sabharwal@sfu.ca>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <56BE71B0.1080904@gmail.com>
Content-Type: text/plain; charset="windows-1252"
I think it's
FeatureFunction::GetScoreProducerDescription()
On 12/02/16 23:56, Jasneet Sabharwal wrote:
> Thanks, will give that a try.
>
> Also, is it possible to get the value of feature name inside the
> feature function. I?m specifically talking about ?name? parameter in
> moses.ini. I?m running multiple copies of my feature function with
> different parameter as follows:
>
> CoarseBiLM name=CoarseBiLM tgtWordId...
> CoarseBiLM name=CoarseLM100 tgtWordId?
> CoarseBiLM name=CoarseLM1600 tgtWordId...
> CoarseBiLM name=CoarseBiLMWithoutClustering tgtWordId?
>
> Thanks,
> Jasneet
>> On Feb 12, 2016, at 3:39 PM, Hieu Hoang <hieuhoang@gmail.com
>> <mailto:hieuhoang@gmail.com>> wrote:
>>
>> you can run the decoder
>> ./moses -v 3
>> however, you should put debugging messages in your feature functions
>> to find out where the problem is. It looks like its in the Load()
>> method so add lots of debugging message in there and all functions it
>> calls
>>
>> On 12/02/16 23:34, Jasneet Sabharwal wrote:
>>> Thanks Hieu for your reply.
>>>
>>> Is it possible to do a verbose output of what?s happening, so that I
>>> can identify when it?s going out of memory? I?m only running it for
>>> 1928 sentences. I have almost 170gb of free memory and additional
>>> 400gb memory in buffer.
>>>
>>> Thanks,
>>> Jasneet
>>>
>>>> On Feb 12, 2016, at 2:36 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>>
>>>> looks like it's run out of memory.
>>>>
>>>> On 11/02/16 23:23, Jasneet Sabharwal wrote:
>>>>> Hi,
>>>>>
>>>>> I was adding a new feature function in Moses
>>>>> (https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp).
>>>>> It works fine when I test it for 1-2 sentences, but when I?m
>>>>> trying to tune my parameters, I?m getting segmentation faults or
>>>>> sometimes it is bad_alloc. Following was one of the commands that
>>>>> was executed during the tuning process which caused the
>>>>> Segmentation Fault or bad_alloc:
>>>>>
>>>>> moses -threads 40 -v 0 -config filtered/moses.ini
>>>>> -weight-overwrite 'CoarseLM100= 0.075758 LM0= 0.075758
>>>>> CoarseBiLMNotClustered= 0.075758 WordPenalty0= -0.151515
>>>>> PhrasePenalty0= 0.030303 CoarseBiLMClustered= 0.075758
>>>>> TranslationModel0= 0.030303 0.030303 0.030303 0.030303
>>>>> Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0=
>>>>> 0.045455 0.045455 0.045455 0.045455 0.045455 0.045455'
>>>>> -n-best-list run1.best100.out 100 distinct -input-file
>>>>> tune.word.lc.cn <http://tune.word.lc.cn/>
>>>>>
>>>>> The log is enclosed in this email.
>>>>>
>>>>> Any pointers would be very useful.
>>>>>
>>>>> Thanks,
>>>>> Jasneet
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>> --
>>>> Hieu Hoang
>>>> http://www.hoang.co.uk/hieu
>>>
>>
>> --
>> Hieu Hoang
>> http://www.hoang.co.uk/hieu
>
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160212/67ee9c67/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 112, Issue 26
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 112, Issue 26"
Post a Comment