Moses-support Digest, Vol 112, Issue 1

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Segmentation fault on hierarchical model with moses in
server mode (Martin Baumg?rtner)


----------------------------------------------------------------------

Message: 1
Date: Mon, 1 Feb 2016 13:47:46 +0100
From: Martin Baumg?rtner <martin.baumgaertner@star-group.net>
Subject: Re: [Moses-support] Segmentation fault on hierarchical model
with moses in server mode
To: Matthias Huck <mhuck@inf.ed.ac.uk>, Hieu Hoang
<hieuhoang@gmail.com>, Ulrich Germann <ugermann@inf.ed.ac.uk>, Barry
Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <56AF53F2.2040809@star-group.net>
Content-Type: text/plain; charset="utf-8"

With the current fix the crash is gone, but there is still an unexpected
difference between server mode and command line moses: the first
produces an empty string, now, whereas the latter produces "this is a
house". I tried another engine (english->chinese) and got the same
behaviour - empty string vs. correct translation.

Cheers,
Martin

Am 29.01.2016 um 22:37 schrieb Matthias Huck:
> On Fri, 2016-01-29 at 21:26 +0000, Hieu Hoang wrote:
>> The decoder should handle no translation without falling over. But
>> yes, the model is too toy
> Normally the decoder would always produce some translation. (The translation could be an empty sentence, of course.) If it's misconfigured, it should tell you about it. But maybe not with a segmentation fault. :-)
>
>
>> On 29 Jan 2016 9:15 pm, "Matthias Huck" <mhuck@inf.ed.ac.uk> wrote:
>>> Hi,
>>>
>>> It seems to me that this toy string-to-tree setup is either
>>> outdated,
>>> or it always had issues. It should be replaced.
>>>
>>> Under real-world conditions, the decoder should always be able to
>>> produce some hypothesis. We would therefore usually extract a whole
>>> set
>>> of glue rules. And we would typically also add an [unknown-lhs]
>>> section
>>> to the moses.ini that would tell the decoder which left-hand side
>>> non
>>> -terminal labels to use for out-of-vocabulary words. To my
>>> knowledge,
>>> these two techniques are crucial for being able to parse any input
>>> sentence provided to the chart decoder in syntax-based translation.
>>>
>>> So, in my opinion, the problem is most likely neither the server
>>> implementation nor the syntax-based decoder, but a problematic
>>> setup.
>>> I would consider it okay for the server to crash (or at least print
>>> a
>>> warning) under such circumstances. You don't want it to silently
>>> not
>>> translate complete sentences.
>>>
>>> (I must admit that I didn't look into it in too much detail, but it
>>> sho
>>> uld be easy to confirm.)
>>>
>>> Cheers,
>>> Matthias
>>>
>>>
>>> On Fri, 2016-01-29 at 20:28 +0000, Barry Haddow wrote:
>>>> Hi All
>>>>
>>>> I think I see what happened now.
>>>>
>>>> When you give the input "dies ist ein haus" to the sample model,
>>> the
>>>> "dies" is unknown, and there is no translation. The server did
>>> not check
>>>> for this condition, and got a seg fault. I have added a check, so
>>> if you
>>>> pull and try again it should not crash.
>>>>
>>>> In the log pasted by Martin, he passed "das ist ein haus" to
>>>> command-line Moses, which works, and gives a translation.
>>>>
>>>> I think ideally the sample models should handle unknown words,
>>> and give
>>>> a translation. Maybe adding a glue rule would be sufficient?
>>>>
>>>> cheers - Barry
>>>>
>>>> On 29/01/16 11:13, Barry Haddow wrote:
>>>>> Hi
>>>>>
>>>>> When I run command-line Moses, I get the output below - i.e. no
>>> best
>>>>> translation. The server crashes for me since it does not check
>>> for the
>>>>> null pointer, but the command-line version does.
>>>>>
>>>>> I think there should be a translation for this example.
>>>>>
>>>>> cheers - Barry
>>>>>
>>>>> [gna]bhaddow: echo 'dies ist ein haus' | ~/moses.new/bin/moses
>>> -f
>>>>> string-to-tree/moses.ini
>>>>> Defined parameters (per moses.ini or switch):
>>>>> config: string-to-tree/moses.ini
>>>>> cube-pruning-pop-limit: 1000
>>>>> feature: KENLM name=LM factor=0 order=3 num
>>> -features=1
>>>>> path=lm/europarl.srilm.gz WordPenalty UnknownWordPenalty
>>>>> PhraseDictionaryMemory input-factor=0 output-factor=0
>>>>> path=string-to-tree/rule-table num-features=1 table-limit=20
>>>>> input-factors: 0
>>>>> inputtype: 3
>>>>> mapping: 0 T 0
>>>>> max-chart-span: 20 1000
>>>>> non-terminals: X S
>>>>> search-algorithm: 3
>>>>> translation-details: translation-details.log
>>>>> weight: WordPenalty0= 0 LM= 0.5
>>> PhraseDictionaryMemory0= 0.5
>>>>> line=KENLM name=LM factor=0 order=3 num-features=1
>>> path=lm/europarl.srilm.gz
>>>>> Loading the LM will be faster if you build a binary file.
>>>>> Reading lm/europarl.srilm.gz
>>>>> ----5---10---15---20---25---30---35---40---45---50---55---60--
>>> -65---70---75---80---85---90---95--100
>>>>> **The ARPA file is missing . Substituting log10 probability
>>> -100.000.
>>> *******************************************************************
>>> *******************************
>>>>> FeatureFunction: LM start: 0 end: 0
>>>>> line=WordPenalty
>>>>> FeatureFunction: WordPenalty0 start: 1 end: 1
>>>>> line=UnknownWordPenalty
>>>>> FeatureFunction: UnknownWordPenalty0 start: 2 end: 2
>>>>> line=PhraseDictionaryMemory input-factor=0 output-factor=0
>>>>> path=string-to-tree/rule-table num-features=1 table-limit=20
>>>>> FeatureFunction: PhraseDictionaryMemory0 start: 3 end: 3
>>>>> Loading LM
>>>>> Loading WordPenalty0
>>>>> Loading UnknownWordPenalty0
>>>>> Loading PhraseDictionaryMemory0
>>>>> Start loading text phrase table. Moses format : [3.038] seconds
>>>>> Reading string-to-tree/rule-table
>>>>> ----5---10---15---20---25---30---35---40---45---50---55---60--
>>> -65---70---75---80---85---90---95--100
>>> *******************************************************************
>>> *********************************
>>>>> max-chart-span: 20
>>>>> Created input-output object : [3.041] seconds
>>>>> Line 0: Initialize search took 0.000 seconds total
>>>>> Translating: dies ist ein haus ||| [0,0]=X (1) [0,1]=X (1)
>>>>> [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1)
>>> [1,2]=X (1)
>>>>> [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1)
>>> [2,4]=X (1)
>>>>> [2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1)
>>> [4,5]=X (1)
>>>>> [5,5]=X (1)
>>>>>
>>>>> 0 1 2 3 4 5
>>>>> 0 1 2 2 1 0
>>>>> 0 0 0 2 0
>>>>> 0 0 4 0
>>>>> 0 0 0
>>>>> 0 0
>>>>> 0
>>>>> Line 0: Additional reporting took 0.000 seconds total
>>>>> Line 0: Translation took 0.002 seconds total
>>>>> Translation took 0.000 seconds
>>>>> Name:moses VmPeak:74024 kB VmRSS:11084 kB RSSMax:36832 kB
>>>>> user:2.972 sys:0.048 CPU:3.020 real:3.058
>>>>>
>>>>>
>>>>> On 29/01/16 00:40, Hieu Hoang wrote:
>>>>>> If it works ok on the command line but crashes when using the
>>> server,
>>>>>> then that suggest a server issue.
>>>>>>
>>>>>> I don't know much about the server code, to be honest.
>>>>>>
>>>>
>>> --
>>> The University of Edinburgh is a charitable body, registered in
>>> Scotland, with registration number SC005336.
>>>
>>>

--

*STAR Group* <http://www.star-group.net>
<http://www.star-group.net/>

*Martin Baumg?rtner*

STAR Language Technology & Solutions GmbH
Umberto-Nobile-Stra?e 19 | 71063 Sindelfingen | Germany
Tel. +49 70 31-4 10 92-0 martin.baumgaertner@star-group.net
<mailto:martin.baumgaertner@star-group.net>
Fax +49 70 31-4 10 92-70 www.star-group.net <http://www.star-group.net/>
Gesch?ftsf?hrer: Oliver Rau, Bernd Barth
Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160201/6f763bf0/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jggigjbj.gif
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160201/6f763bf0/attachment.gif

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 112, Issue 1
*********************************************

Tweet