Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Segmentation fault on hierarchical model with moses in
server mode (Barry Haddow)
----------------------------------------------------------------------
Message: 1
Date: Mon, 1 Feb 2016 16:29:09 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Segmentation fault on hierarchical model
with moses in server mode
To: Martin Baumg?rtner <martin.baumgaertner@star-group.net>, Matthias
Huck <mhuck@inf.ed.ac.uk>, Hieu Hoang <hieuhoang@gmail.com>, Ulrich
Germann <ugermann@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <56AF87D5.2080707@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"
Hi Martin
OK, I see what is happening now. Moses chart expects a TreeInput as its
source, but was getting a Sentence. I have pushed a fix, so could you
try again?
cheers - Barry
On 01/02/16 15:32, Martin Baumg?rtner wrote:
> Hi Barry,
>
> sorry, my mistake ...
>
> Here comes the experiment with the *same* string:
>
> 1. COMMAND-LINE:
>
> % echo 'das ist ein haus' | moses -f string-to-tree/moses.ini
>
> Defined parameters (per moses.ini or switch):
> config: string-to-tree/moses.ini
> cube-pruning-pop-limit: 1000
> feature: KENLM name=LM factor=0 order=3 num-features=1
> path=lm/europarl.srilm.gz WordPenalty UnknownWordPenalty
> PhraseDictionaryMemory input-factor=0 output-factor=0
> path=string-to-tree/rule-table num-features=1 table-limit=20
> input-factors: 0
> inputtype: 3
> mapping: 0 T 0
> max-chart-span: 20 1000
> non-terminals: X S
> search-algorithm: 3
> translation-details: translation-details.log
> weight: WordPenalty0= 0 LM= 0.5 PhraseDictionaryMemory0= 0.5
> line=KENLM name=LM factor=0 order=3 num-features=1
> path=lm/europarl.srilm.gz
> Loading the LM will be faster if you build a binary file.
> Reading lm/europarl.srilm.gz
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> **The ARPA file is missing <unk>. Substituting log10 probability
> -100.000.
> **************************************************************************************************
> FeatureFunction: LM start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 2 end: 2
> line=PhraseDictionaryMemory input-factor=0 output-factor=0
> path=string-to-tree/rule-table num-features=1 table-limit=20
> FeatureFunction: PhraseDictionaryMemory0 start: 3 end: 3
> Loading LM
> Loading WordPenalty0
> Loading UnknownWordPenalty0
> Loading PhraseDictionaryMemory0
> Start loading text phrase table. Moses format : [0.773] seconds
> Reading string-to-tree/rule-table
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> ****************************************************************************************************
> max-chart-span: 20
> Created input-output object : [0.773] seconds
> Line 0: Initialize search took 0.000 seconds total
> Translating: <s> das ist ein haus </s> ||| [0,0]=X (1) [0,1]=X
> (1) [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1)
> [1,2]=X (1) [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1)
> [2,3]=X (1) [2,4]=X (1) [2,5]=X (1) [3,3]=X (1) [3,4]=X (1)
> [3,5]=X (1) [4,4]=X (1) [4,5]=X (1) [5,5]=X (1)
>
> 0 1 2 3 4 5
> 0 3 2 2 1 0
> 0 0 0 2 0
> 0 0 4 0
> 0 8 0
> 0 0
> 1
> BEST TRANSLATION: 31 TOP -> <s> S </s> :1-1 : term=: nonterm=1-1
> : c=-3.206 core=(-6.413,-2.000,0.000,0.000) [0..5] 15
> [total=-12.332] core=(-22.361,-6.000,0.000,-2.303)
> this is a house
> Line 0: Additional reporting took 0.000 seconds total
> Line 0: Translation took 0.001 seconds total
> Translation took 0.010 seconds
> Name:moses VmPeak:79168 kB VmRSS:7432 kB RSSMax:35968 kB
> user:0.758 sys:0.021 CPU:0.780 real:0.776
>
>
> 2.) SERVER:
>
> % /home/maba/bin/moses --server --config string-to-tree/moses.ini
>
> Defined parameters (per moses.ini or switch):
> config: string-to-tree/moses.ini
> cube-pruning-pop-limit: 1000
> feature: KENLM name=LM factor=0 order=3 num-features=1
> path=lm/europarl.srilm.gz WordPenalty UnknownWordPenalty
> PhraseDictionaryMemory input-factor=0 output-factor=0
> path=string-to-tree/rule-table num-features=1 table-limit=20
> input-factors: 0
> inputtype: 3
> mapping: 0 T 0
> max-chart-span: 20 1000
> non-terminals: X S
> search-algorithm: 3
> server:
> translation-details: translation-details.log
> weight: WordPenalty0= 0 LM= 0.5 PhraseDictionaryMemory0= 0.5
> line=KENLM name=LM factor=0 order=3 num-features=1
> path=lm/europarl.srilm.gz
> Loading the LM will be faster if you build a binary file.
> Reading lm/europarl.srilm.gz
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> **The ARPA file is missing <unk>. Substituting log10 probability
> -100.000.
> **************************************************************************************************
> FeatureFunction: LM start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 2 end: 2
> line=PhraseDictionaryMemory input-factor=0 output-factor=0
> path=string-to-tree/rule-table num-features=1 table-limit=20
> FeatureFunction: PhraseDictionaryMemory0 start: 3 end: 3
> Loading LM
> Loading WordPenalty0
> Loading UnknownWordPenalty0
> Loading PhraseDictionaryMemory0
> Start loading text phrase table. Moses format : [0.747] seconds
> Reading string-to-tree/rule-table
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> ****************************************************************************************************
> max-chart-span: 20
> RUN SERVER at pid 0
> [moses/server/Server.cpp:49] Listening on port 8080
> [moses/server/TranslationRequest.cpp:315] Input: das ist ein haus
> Translating: <s> das ist ein haus </s>
> 0 1 2 3 4 5
> 0 3 2 2 1 0
> 0 0 0 0 0
> 0 0 0 0
> 0 0 0
> 0 0
> 0
> Translation took 0.000 seconds
>
>
> => XML-RPC result after calling (see moses server output just above) ...
>
> % curl --data @rpc.xml '192.168.178.14:8080/RPC2
> <http://192.168.178.14:8080/RPC2>'
>
> <?xml version="1.0" encoding="UTF-8"?>
> <methodResponse>
> <params>
> <param><value><struct>
> <member><name>text</name>
> <value><string></string></value></member>
> </struct></value></param>
> </params>
> </methodResponse>
>
> ... with rpc.xml ...
>
> <?xml version="1.0" encoding="UTF-8"?>
> <methodCall>
> <methodName>translate</methodName>
> <params><param><value>
> <struct>
> <member>
> <name>text</name>
> <value>
> <string>das ist ein haus</string>
> </value>
> </member>
> </struct></value></param>
> </params>
> </methodCall>
>
>
> Cheers,
> Martin
>
>
> Am 01.02.2016 um 14:16 schrieb Barry Haddow:
>> Hi Martin
>>
>> In your original mail, you passed different inputs to the server and
>> command-line version. The following are extracted from your debug:
>>
>> command-line:
>> Translating: <s> das ist ein haus </s>
>>
>> server:
>> Translating: <s> dies ist ein haus . </s>
>>
>> (Although you specify the command-line in the first case as "echo
>> 'dies ist ein haus' | moses -f string-to-tree/moses.ini", which
>> contradicts the debug. I assume you spliced together different runs.)
>>
>> Could you show an example where you pass the *same* string to the two
>> different Moses versions and get different outputs?
>>
>> cheers - Barry
>>
>> On 01/02/16 12:47, Martin Baumg?rtner wrote:
>>> With the current fix the crash is gone, but there is still an
>>> unexpected difference between server mode and command line moses:
>>> the first produces an empty string, now, whereas the latter produces
>>> "this is a house". I tried another engine (english->chinese) and got
>>> the same behaviour - empty string vs. correct translation.
>>>
>>> Cheers,
>>> Martin
>>>
>>> Am 29.01.2016 um 22:37 schrieb Matthias Huck:
>>>> On Fri, 2016-01-29 at 21:26 +0000, Hieu Hoang wrote:
>>>>> The decoder should handle no translation without falling over. But
>>>>> yes, the model is too toy
>>>> Normally the decoder would always produce some translation. (The translation could be an empty sentence, of course.) If it's misconfigured, it should tell you about it. But maybe not with a segmentation fault. :-)
>>>>
>>>>
>>>>> On 29 Jan 2016 9:15 pm, "Matthias Huck"<mhuck@inf.ed.ac.uk> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> It seems to me that this toy string-to-tree setup is either
>>>>>> outdated,
>>>>>> or it always had issues. It should be replaced.
>>>>>>
>>>>>> Under real-world conditions, the decoder should always be able to
>>>>>> produce some hypothesis. We would therefore usually extract a whole
>>>>>> set
>>>>>> of glue rules. And we would typically also add an [unknown-lhs]
>>>>>> section
>>>>>> to the moses.ini that would tell the decoder which left-hand side
>>>>>> non
>>>>>> -terminal labels to use for out-of-vocabulary words. To my
>>>>>> knowledge,
>>>>>> these two techniques are crucial for being able to parse any input
>>>>>> sentence provided to the chart decoder in syntax-based translation.
>>>>>>
>>>>>> So, in my opinion, the problem is most likely neither the server
>>>>>> implementation nor the syntax-based decoder, but a problematic
>>>>>> setup.
>>>>>> I would consider it okay for the server to crash (or at least print
>>>>>> a
>>>>>> warning) under such circumstances. You don't want it to silently
>>>>>> not
>>>>>> translate complete sentences.
>>>>>>
>>>>>> (I must admit that I didn't look into it in too much detail, but it
>>>>>> sho
>>>>>> uld be easy to confirm.)
>>>>>>
>>>>>> Cheers,
>>>>>> Matthias
>>>>>>
>>>>>>
>>>>>> On Fri, 2016-01-29 at 20:28 +0000, Barry Haddow wrote:
>>>>>>> Hi All
>>>>>>>
>>>>>>> I think I see what happened now.
>>>>>>>
>>>>>>> When you give the input "dies ist ein haus" to the sample model,
>>>>>> the
>>>>>>> "dies" is unknown, and there is no translation. The server did
>>>>>> not check
>>>>>>> for this condition, and got a seg fault. I have added a check, so
>>>>>> if you
>>>>>>> pull and try again it should not crash.
>>>>>>>
>>>>>>> In the log pasted by Martin, he passed "das ist ein haus" to
>>>>>>> command-line Moses, which works, and gives a translation.
>>>>>>>
>>>>>>> I think ideally the sample models should handle unknown words,
>>>>>> and give
>>>>>>> a translation. Maybe adding a glue rule would be sufficient?
>>>>>>>
>>>>>>> cheers - Barry
>>>>>>>
>>>>>>> On 29/01/16 11:13, Barry Haddow wrote:
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> When I run command-line Moses, I get the output below - i.e. no
>>>>>> best
>>>>>>>> translation. The server crashes for me since it does not check
>>>>>> for the
>>>>>>>> null pointer, but the command-line version does.
>>>>>>>>
>>>>>>>> I think there should be a translation for this example.
>>>>>>>>
>>>>>>>> cheers - Barry
>>>>>>>>
>>>>>>>> [gna]bhaddow: echo 'dies ist ein haus' | ~/moses.new/bin/moses
>>>>>> -f
>>>>>>>> string-to-tree/moses.ini
>>>>>>>> Defined parameters (per moses.ini or switch):
>>>>>>>> config: string-to-tree/moses.ini
>>>>>>>> cube-pruning-pop-limit: 1000
>>>>>>>> feature: KENLM name=LM factor=0 order=3 num
>>>>>> -features=1
>>>>>>>> path=lm/europarl.srilm.gz WordPenalty UnknownWordPenalty
>>>>>>>> PhraseDictionaryMemory input-factor=0 output-factor=0
>>>>>>>> path=string-to-tree/rule-table num-features=1 table-limit=20
>>>>>>>> input-factors: 0
>>>>>>>> inputtype: 3
>>>>>>>> mapping: 0 T 0
>>>>>>>> max-chart-span: 20 1000
>>>>>>>> non-terminals: X S
>>>>>>>> search-algorithm: 3
>>>>>>>> translation-details: translation-details.log
>>>>>>>> weight: WordPenalty0= 0 LM= 0.5
>>>>>> PhraseDictionaryMemory0= 0.5
>>>>>>>> line=KENLM name=LM factor=0 order=3 num-features=1
>>>>>> path=lm/europarl.srilm.gz
>>>>>>>> Loading the LM will be faster if you build a binary file.
>>>>>>>> Reading lm/europarl.srilm.gz
>>>>>>>> ----5---10---15---20---25---30---35---40---45---50---55---60--
>>>>>> -65---70---75---80---85---90---95--100
>>>>>>>> **The ARPA file is missing . Substituting log10 probability
>>>>>> -100.000.
>>>>>> *******************************************************************
>>>>>> *******************************
>>>>>>>> FeatureFunction: LM start: 0 end: 0
>>>>>>>> line=WordPenalty
>>>>>>>> FeatureFunction: WordPenalty0 start: 1 end: 1
>>>>>>>> line=UnknownWordPenalty
>>>>>>>> FeatureFunction: UnknownWordPenalty0 start: 2 end: 2
>>>>>>>> line=PhraseDictionaryMemory input-factor=0 output-factor=0
>>>>>>>> path=string-to-tree/rule-table num-features=1 table-limit=20
>>>>>>>> FeatureFunction: PhraseDictionaryMemory0 start: 3 end: 3
>>>>>>>> Loading LM
>>>>>>>> Loading WordPenalty0
>>>>>>>> Loading UnknownWordPenalty0
>>>>>>>> Loading PhraseDictionaryMemory0
>>>>>>>> Start loading text phrase table. Moses format : [3.038] seconds
>>>>>>>> Reading string-to-tree/rule-table
>>>>>>>> ----5---10---15---20---25---30---35---40---45---50---55---60--
>>>>>> -65---70---75---80---85---90---95--100
>>>>>> *******************************************************************
>>>>>> *********************************
>>>>>>>> max-chart-span: 20
>>>>>>>> Created input-output object : [3.041] seconds
>>>>>>>> Line 0: Initialize search took 0.000 seconds total
>>>>>>>> Translating: dies ist ein haus ||| [0,0]=X (1) [0,1]=X (1)
>>>>>>>> [0,2]=X (1) [0,3]=X (1) [0,4]=X (1) [0,5]=X (1) [1,1]=X (1)
>>>>>> [1,2]=X (1)
>>>>>>>> [1,3]=X (1) [1,4]=X (1) [1,5]=X (1) [2,2]=X (1) [2,3]=X (1)
>>>>>> [2,4]=X (1)
>>>>>>>> [2,5]=X (1) [3,3]=X (1) [3,4]=X (1) [3,5]=X (1) [4,4]=X (1)
>>>>>> [4,5]=X (1)
>>>>>>>> [5,5]=X (1)
>>>>>>>>
>>>>>>>> 0 1 2 3 4 5
>>>>>>>> 0 1 2 2 1 0
>>>>>>>> 0 0 0 2 0
>>>>>>>> 0 0 4 0
>>>>>>>> 0 0 0
>>>>>>>> 0 0
>>>>>>>> 0
>>>>>>>> Line 0: Additional reporting took 0.000 seconds total
>>>>>>>> Line 0: Translation took 0.002 seconds total
>>>>>>>> Translation took 0.000 seconds
>>>>>>>> Name:moses VmPeak:74024 kB VmRSS:11084 kB RSSMax:36832 kB
>>>>>>>> user:2.972 sys:0.048 CPU:3.020 real:3.058
>>>>>>>>
>>>>>>>>
>>>>>>>> On 29/01/16 00:40, Hieu Hoang wrote:
>>>>>>>>> If it works ok on the command line but crashes when using the
>>>>>> server,
>>>>>>>>> then that suggest a server issue.
>>>>>>>>>
>>>>>>>>> I don't know much about the server code, to be honest.
>>>>>>>>>
>>>>>> --
>>>>>> The University of Edinburgh is a charitable body, registered in
>>>>>> Scotland, with registration number SC005336.
>>>>>>
>>>>>>
>>>
>>> --
>>>
>>> *STAR Group* <http://www.star-group.net>
>>> <http://www.star-group.net/>
>>>
>>> *Martin Baumg?rtner*
>>>
>>> STAR Language Technology & Solutions GmbH
>>> Umberto-Nobile-Stra?e 19 | 71063 Sindelfingen | Germany
>>> Tel. +49 70 31-4 10 92-0
>>> <mailto:martin.baumgaertner@star-group.net>martin.baumgaertner@star-group.net
>>>
>>> Fax +49 70 31-4 10 92-70 www.star-group.net
>>> <http://www.star-group.net/>
>>> Gesch?ftsf?hrer: Oliver Rau, Bernd Barth
>>> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>>>
>>
>>
>>
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>
> --
>
> *STAR Group* <http://www.star-group.net>
> <http://www.star-group.net/>
>
> *Martin Baumg?rtner*
>
> STAR Language Technology & Solutions GmbH
> Umberto-Nobile-Stra?e 19 | 71063 Sindelfingen | Germany
> Tel. +49 70 31-4 10 92-0
> <mailto:martin.baumgaertner@star-group.net>martin.baumgaertner@star-group.net
>
> Fax +49 70 31-4 10 92-70 www.star-group.net <http://www.star-group.net/>
> Gesch?ftsf?hrer: Oliver Rau, Bernd Barth
> Handelsregister Stuttgart HRB 245654 | St.-Nr. 56098/11677
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160201/d65f98d6/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160201/d65f98d6/attachment.gif
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 8030 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160201/d65f98d6/attachment-0001.gif
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160201/d65f98d6/attachment.pl
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 112, Issue 5
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 112, Issue 5"
Post a Comment