Moses-support Digest, Vol 112, Issue 36

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Segmentation Fault (Jasneet Sabharwal)
2. Job: Open position for a Research Fellowship in the Language
Technology Lab in Berlin (Eleftherios Avramidis)


----------------------------------------------------------------------

Message: 1
Date: Thu, 18 Feb 2016 11:04:29 -0800
From: Jasneet Sabharwal <jasneet.sabharwal@sfu.ca>
Subject: Re: [Moses-support] Segmentation Fault
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <9A6C8B67-4161-4852-9ED4-59F3922ED7E4@sfu.ca>
Content-Type: text/plain; charset="utf-8"

Hi,

I was able to solve the segmentation fault issue. It was because of OOVs. I?m currently trying to tune the parameters using mert, but it is running extremely slow. For example, from the logs:

Translating: ?? ? ? ?? ? ? ? ? ? ??????? ? ? ? ? ? ? ?? ? , ? ? ?? ?? ?? ??? ? ? ? ?? ? ? ? ??????? ? ?? ?? ?
Line 43: Initialize search took 0.007 seconds total
Line 43: Collecting options took 0.191 seconds at moses/Manager.cpp:117
Line 38: Search took 1092.075 seconds
Line 38: Decision rule took 0.000 seconds total
Line 38: Additional reporting took 0.041 seconds total
Line 38: Translation took 1092.132 seconds total

I tried to time the functions in my feature function <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp> using clock_t but all of them show up as 0.000. I?m not sure why tuning is taking too much time. My moses.ini is attached in this email.

Any suggestions would be helpful.

Regards,
Jasneet


> On Feb 12, 2016, at 3:58 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
> I think it's
> FeatureFunction::GetScoreProducerDescription()
>
> On 12/02/16 23:56, Jasneet Sabharwal wrote:
>> Thanks, will give that a try.
>>
>> Also, is it possible to get the value of feature name inside the feature function. I?m specifically talking about ?name? parameter in moses.ini. I?m running multiple copies of my feature function with different parameter as follows:
>> CoarseBiLM name=CoarseBiLM tgtWordId...
>> CoarseBiLM name=CoarseLM100 tgtWordId?
>> CoarseBiLM name=CoarseLM1600 tgtWordId...
>> CoarseBiLM name=CoarseBiLMWithoutClustering tgtWordId?
>>
>> Thanks,
>> Jasneet
>>> On Feb 12, 2016, at 3:39 PM, Hieu Hoang < <mailto:hieuhoang@gmail.com>hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>> wrote:
>>>
>>> you can run the decoder
>>> ./moses -v 3
>>> however, you should put debugging messages in your feature functions to find out where the problem is. It looks like its in the Load() method so add lots of debugging message in there and all functions it calls
>>>
>>> On 12/02/16 23:34, Jasneet Sabharwal wrote:
>>>> Thanks Hieu for your reply.
>>>>
>>>> Is it possible to do a verbose output of what?s happening, so that I can identify when it?s going out of memory? I?m only running it for 1928 sentences. I have almost 170gb of free memory and additional 400gb memory in buffer.
>>>>
>>>> Thanks,
>>>> Jasneet
>>>>
>>>>> On Feb 12, 2016, at 2:36 PM, Hieu Hoang <hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>> wrote:
>>>>>
>>>>> looks like it's run out of memory.
>>>>>
>>>>> On 11/02/16 23:23, Jasneet Sabharwal wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I was adding a new feature function in Moses ( <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp>https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp <https://github.com/KonceptGeek/mosesdecoder/blob/master/moses/FF/CoarseBiLM.cpp>). It works fine when I test it for 1-2 sentences, but when I?m trying to tune my parameters, I?m getting segmentation faults or sometimes it is bad_alloc. Following was one of the commands that was executed during the tuning process which caused the Segmentation Fault or bad_alloc:
>>>>>>
>>>>>> moses -threads 40 -v 0 -config filtered/moses.ini -weight-overwrite 'CoarseLM100= 0.075758 LM0= 0.075758 CoarseBiLMNotClustered= 0.075758 WordPenalty0= -0.151515 PhrasePenalty0= 0.030303 CoarseBiLMClustered= 0.075758 TranslationModel0= 0.030303 0.030303 0.030303 0.030303 Distortion0= 0.045455 CoarseLM1600= 0.075758 LexicalReordering0= 0.045455 0.045455 0.045455 0.045455 0.045455 0.045455' -n-best-list run1.best100.out 100 distinct -input-file tune.word.lc.cn <http://tune.word.lc.cn/>
>>>>>>
>>>>>> The log is enclosed in this email.
>>>>>>
>>>>>> Any pointers would be very useful.
>>>>>>
>>>>>> Thanks,
>>>>>> Jasneet
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Moses-support mailing list
>>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>>
>>>>> --
>>>>> Hieu Hoang
>>>>> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu>
>>>
>>> --
>>> Hieu Hoang
>>> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu>
>
> --
> Hieu Hoang
> http://www.hoang.co.uk/hieu <http://www.hoang.co.uk/hieu>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160218/968db25d/attachment-0002.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini
Type: application/octet-stream
Size: 3506 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160218/968db25d/attachment-0001.obj
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160218/968db25d/attachment-0003.html

------------------------------

Message: 2
Date: Fri, 19 Feb 2016 16:46:11 +0100
From: Eleftherios Avramidis <eleftherios.avramidis@dfki.de>
Subject: [Moses-support] Job: Open position for a Research Fellowship
in the Language Technology Lab in Berlin
To: Moses-support <moses-support@mit.edu>
Message-ID: <56C738C3.5040806@dfki.de>
Content-Type: text/plain; charset="utf-8"

The Language Technology Lab of the German Research Center for Artificial
Intelligence (DFKI LT-Lab) invites applications in Berlin, Germany for a
*Research Fellowship*
in language technology, primarily in the area of machine translation.
The focus will be on the analytical improvement of MT techniques and on
MT evaluation. We are looking for people who enjoy working in an
innovative and enthusiastic research team, love challenges and are
passionate about research and development. We offer an excellent working
environment with a variety of research projects. The successful
applicant will work in a multi-disciplinary and international team in
one of the world?s foremost research centers in AI and NLP. DFKI Berlin
is located in the center of the dynamic, livable and affordable capital
of Germany.

The candidates should have a master degree or PhD in computational
linguistics, computer science or related disciplines.

*Expertise and experience in more than one of the following areas is
expected:*

* statistical MT
* rule-based MT
* MT evaluation ?
* CAT techniques

*Experience in these areas is a plus:*

* machine learning
* natural language parsing
* NL syntax and semantics
* language data (corpora, dictionaries, thesauri)
* mathematical background (probability & statistics)

*Expected are programming skills:*

* Experience with programming in Java and/or C++
* Experience with scripting languages Python/Pearl
* Experience in web and/or database technology
* A good grasp of algorithms and software architectures

*Expected general skills:*

* Good communication and teamwork skills
* Excellent problem-solving skills
* High degree of motivation
* Able to manage own workload and meet deadlines
* Good organizational skills
* Good standard of written and spoken English

*Pay

*Successful candidates will be offered a competitive salary based on
their qualifications and experience.

*Duration

*The position is bound to work in projects and can last up to six
months. Extension after this period may be possible.

*How to Apply

*Please send your electronic application (preferably in PDF format) to
Prof. Hans Uszkoreit ( uszkoreit@dfki.de ) and Dr. Aljoscha Burchardt (
burchardt@dfki.de ) referring to job opening No. 6-16/HU. A meaningful
application should include a cover letter, a recent CV, and a brief
summary of professional interests, and contact information for three
references.

Preference is given to applications received by February 29, 2016.

--
MSc. Inf. Eleftherios Avramidis
DFKI GmbH, Alt-Moabit 91c, 10559 Berlin
Tel. +49-30 238 95-1806

Fax. +49-30 238 95-1810

-------------------------------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Strasse 122, D-67663 Kaiserslautern

Geschaeftsfuehrung:
Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff

Vorsitzender des Aufsichtsrats:
Prof. Dr. h.c. Hans A. Aukes

Amtsgericht Kaiserslautern, HRB 2313
-------------------------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160219/2c75a3f5/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: LT_HU_ResearchFellow.pdf
Type: application/pdf
Size: 44118 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160219/2c75a3f5/attachment.pdf

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 112, Issue 36
**********************************************

Tweet