Moses-support Digest, Vol 90, Issue 31

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: output options (Tom Hoar)
2. Re: Lattice decoding issues (Hieu Hoang)
3. Re: Lattice decoding issues (Ratish Puduppully)
4. Fwd: no best translation with factored model (Jonathan Geiger)


----------------------------------------------------------------------

Message: 1
Date: Fri, 11 Apr 2014 21:00:11 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] output options
To: undisclosed-recipients:;
Cc: Moses-Support <moses-support@mit.edu>
Message-ID: <5347F56B.30907@precisiontranslationtools.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Ah, thanks.

Looks like this is a newer option. I'm still on Revision 1. I get a
message "ERROR:illegal switch: -tt". Looks like I'll have to save to
file and parse output unless there's another option.

Tom


On 04/11/2014 08:55 PM, Philipp Koehn wrote:
> Hi,
>
> "-tt" gives you word alignments in addition to phrase alignments.
>
> -phi
>
> On Fri, Apr 11, 2014 at 9:41 AM, Tom Hoar
> <tahoar@precisiontranslationtools.com> wrote:
>> The -t option reports phrase alignments in the stdout output. The
>> -alignment-output-file saves the word alignments into given file. Is
>> there an option to report the word alignments to the stdout output?
>> Nothing stands out in the Usage output.
>>
>> Thanks.
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support



------------------------------

Message: 2
Date: Fri, 11 Apr 2014 15:08:45 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Lattice decoding issues
To: Ratish Puduppully <ratish123@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbhwrTk6srLZ1BFAdZx776OvzYyyOZS0KWVYtTCatuPpkw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

ah, apologies.

>From the examples in the regression tests, you have to put the whole
lattice on 1 line, eg

https://github.com/moses-smt/moses-regression-tests/blob/master/tests/phrase.lattice-surface/to-translate.txt


On 11 April 2014 14:07, Ratish Puduppully <ratish123@gmail.com> wrote:

> Hi Hieu,
> I fave attached moses.ini and input.plf file in the original email
>
> Thanks,
> Ratish Puduppully
>
>
> On Fri, Apr 11, 2014 at 3:05 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> Can you please send me a copy of the ini file you are using, and a small
>> file which contains a lattice
>>
>> I'll try and take a look
>>
>> Sent while bumping into things
>>
>> On 9 Apr 2014, at 05:20 pm, Ratish Puduppully <ratish123@gmail.com>
>> wrote:
>>
>> Hi,
>> I am trying to decode with lattice.
>> I followed the steps in
>> http://www.statmt.org/moses/?n=Moses.WordLattices
>>
>> http://article.gmane.org/gmane.comp.nlp.moses.user/9493/match=decoding+lattice
>> The command i execute is:
>> $ ~/Project/SMT/mosesdecoder/bin/moses -f
>> ~/Project/SMT/working-smsen/binarised-model-untuned/moses.ini -inputtype 2
>> < ~/Project/SMT/tools/input.plf > ~/Project/SMT/tools/latticeoutput
>>
>> The last few lines in output I see on console is:
>> Loading WordPenalty0
>> Loading PhrasePenalty0
>> Loading LexicalReordering0
>> binary file loaded, default OFF_T: -1
>> Loading Distortion0
>> Loading LM0
>> Loading InputFeature0
>> Loading TranslationModel0
>> IO from STDOUT/STDIN
>> Created input-output object : [0.092] seconds
>> Name:moses VmPeak:53092 kB VmRSS:5168 kB RSSMax:34896 kB user:0.080
>> sys:0.016 CPU:0.096 real:0.097
>> confusion net statistics:
>> created: 1
>> destroyed: 1
>> succ. read: 0
>> columns: 0
>> words: 0
>> avg. word/column: -nan
>> avg. cols/sent: -nan
>>
>> The latticeoutput file has no output.
>> I have attached plf file and moses.ini
>>
>> I am not sure where I am going wrong.
>> Please help.
>>
>> Thanks,
>> Ratish Puduppully
>>
>> <input.plf>
>>
>> <moses.ini>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>


--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140411/2d2909c2/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 11 Apr 2014 19:56:14 +0530
From: Ratish Puduppully <ratish123@gmail.com>
Subject: Re: [Moses-support] Lattice decoding issues
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEb1jG7-YNG4NQQwQVTNoND-=9u3PbOyc3oXJWFREqxJMjHDjQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Thanks a lot Hieu.
It worked.

Regards,
Ratish Puduppully


On Fri, Apr 11, 2014 at 7:38 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:

> ah, apologies.
>
> From the examples in the regression tests, you have to put the whole
> lattice on 1 line, eg
>
> https://github.com/moses-smt/moses-regression-tests/blob/master/tests/phrase.lattice-surface/to-translate.txt
>
>
> On 11 April 2014 14:07, Ratish Puduppully <ratish123@gmail.com> wrote:
>
>> Hi Hieu,
>> I fave attached moses.ini and input.plf file in the original email
>>
>> Thanks,
>> Ratish Puduppully
>>
>>
>> On Fri, Apr 11, 2014 at 3:05 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> Can you please send me a copy of the ini file you are using, and a small
>>> file which contains a lattice
>>>
>>> I'll try and take a look
>>>
>>> Sent while bumping into things
>>>
>>> On 9 Apr 2014, at 05:20 pm, Ratish Puduppully <ratish123@gmail.com>
>>> wrote:
>>>
>>> Hi,
>>> I am trying to decode with lattice.
>>> I followed the steps in
>>> http://www.statmt.org/moses/?n=Moses.WordLattices
>>>
>>> http://article.gmane.org/gmane.comp.nlp.moses.user/9493/match=decoding+lattice
>>> The command i execute is:
>>> $ ~/Project/SMT/mosesdecoder/bin/moses -f
>>> ~/Project/SMT/working-smsen/binarised-model-untuned/moses.ini -inputtype 2
>>> < ~/Project/SMT/tools/input.plf > ~/Project/SMT/tools/latticeoutput
>>>
>>> The last few lines in output I see on console is:
>>> Loading WordPenalty0
>>> Loading PhrasePenalty0
>>> Loading LexicalReordering0
>>> binary file loaded, default OFF_T: -1
>>> Loading Distortion0
>>> Loading LM0
>>> Loading InputFeature0
>>> Loading TranslationModel0
>>> IO from STDOUT/STDIN
>>> Created input-output object : [0.092] seconds
>>> Name:moses VmPeak:53092 kB VmRSS:5168 kB RSSMax:34896 kB user:0.080
>>> sys:0.016 CPU:0.096 real:0.097
>>> confusion net statistics:
>>> created: 1
>>> destroyed: 1
>>> succ. read: 0
>>> columns: 0
>>> words: 0
>>> avg. word/column: -nan
>>> avg. cols/sent: -nan
>>>
>>> The latticeoutput file has no output.
>>> I have attached plf file and moses.ini
>>>
>>> I am not sure where I am going wrong.
>>> Please help.
>>>
>>> Thanks,
>>> Ratish Puduppully
>>>
>>> <input.plf>
>>>
>>> <moses.ini>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140411/6f143dab/attachment-0001.htm

------------------------------

Message: 4
Date: Fri, 11 Apr 2014 16:44:10 +0200
From: Jonathan Geiger <j.geiger@lingenio.de>
Subject: [Moses-support] Fwd: no best translation with factored model
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAM0M1Ypkp3Ub5NKzypF78An6X1VHDaYFSy1qFet_R54JBXmk-Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,
I am trying to build a factored translation system, but I don't get a
translation from it ("NO BEST TRANSLATION"). This happens for two
language pairs; the regular "word-for-word" models work, however.
Attached you'll find one ini file and an output (with verbosity level 3).
Maybe you know what's wrong (and who knows, it might just be the data
being useless)?!

Kind regards and thank you very much,
Jonathan Geiger
-------------- next part --------------
Defined parameters (per moses.ini or switch):
config: moses.pruned.ini
cube-pruning-pop-limit: 2000
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/phrase-table.2-2.pruned input-factor=2 output-factor=2 PhraseDictionaryMemory name=TranslationModel1 table-limit=0 num-features=4 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/phrase-table.0,1-0,1.pruned input-factor=0,1 output-factor=0,1 Generation name=GenerationModel0 num-features=2 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/generation.2-1.gz input-factor=2 output-factor=1 Generation name=GenerationModel1 num-features=2 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/generation.1,2-0.gz input-factor=1,2 output-factor=0 Distortion KENLM name=LM0 factor=0 path=/mnt/data/hyghtra/jona_devel/corpora/LM/lm_de/wiki_third.de.binary order=5 KENLM name=LM1 factor=1 path=/mnt/data/hyghtra/jona_deve!
l/corpora/LM/lm_de/wiki.pos.de.binary order=5
input-factors: 0 1 2
mapping: 0 T 0 0 G 0 0 T 1 0 G 1
search-algorithm: 1
stack: 2000
verbose: 4
weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 TranslationModel1= 0.2 0.2 0.2 0.2 GenerationModel0= 0.3 0 GenerationModel1= 0.3 0 Distortion0= 0.3 LM0= 0.5 LM1= 0.5

input type is: text input
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/phrase-table.2-2.pruned input-factor=2 output-factor=2
FeatureFunction: TranslationModel0 start: 3 end: 6
DecodeFeature:
line=PhraseDictionaryMemory name=TranslationModel1 table-limit=0 num-features=4 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/phrase-table.0,1-0,1.pruned input-factor=0,1 output-factor=0,1
FeatureFunction: TranslationModel1 start: 7 end: 10
DecodeFeature:
line=Generation name=GenerationModel0 num-features=2 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/generation.2-1.gz input-factor=2 output-factor=1
FeatureFunction: GenerationModel0 start: 11 end: 12
DecodeFeature:
line=Generation name=GenerationModel1 num-features=2 path=/mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/generation.1,2-0.gz input-factor=1,2 output-factor=0
FeatureFunction: GenerationModel1 start: 13 end: 14
DecodeFeature:
line=Distortion
FeatureFunction: Distortion0 start: 15 end: 15
line=KENLM name=LM0 factor=0 path=/mnt/data/hyghtra/jona_devel/corpora/LM/lm_de/wiki_third.de.binary order=5
FeatureFunction: LM0 start: 16 end: 16
line=KENLM name=LM1 factor=1 path=/mnt/data/hyghtra/jona_devel/corpora/LM/lm_de/wiki.pos.de.binary order=5
FeatureFunction: LM1 start: 17 end: 17
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading GenerationModel0
Loading GenerationModel1
Loading Distortion0
Loading LM0
Loading LM1
Loading TranslationModel0
Start loading text SCFG phrase table. Moses format : [12.689] seconds
Reading /mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/phrase-table.2-2.pruned
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Loading TranslationModel1
Start loading text SCFG phrase table. Moses format : [12.736] seconds
Reading /mnt/data/hyghtra/jona_devel/corpora/pt-de/euro/train_factored/model/phrase-table.0,1-0,1.pruned
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
DecodeStep():
outputFactors=FactorMask<2>
conflictFactors=FactorMask<>
newOutputFactors=FactorMask<2>
DecodeStep():
outputFactors=FactorMask<1,2>
conflictFactors=FactorMask<>
newOutputFactors=FactorMask<1>
DecodeStep():
outputFactors=FactorMask<0,1,2>
conflictFactors=FactorMask<1>
newOutputFactors=FactorMask<0>
DecodeStep():
outputFactors=FactorMask<0,1,2>
conflictFactors=FactorMask<0>
newOutputFactors=FactorMask<>
IO from STDOUT/STDIN
Created input-output object : [12.782] seconds
The global weight vector looks like this: core=(1.000,-1.000,0.200,0.200,0.200,0.200,0.200,0.200,0.200,0.200,0.200,0.300,0.000,0.300,0.000,0.300,0.500,0.500)
Translating line 0 in thread id 140357010097920
InitializeForInput( UnknownWordPenalty0 ) = 0.000
InitializeForInput( WordPenalty0 ) = 0.000
InitializeForInput( PhrasePenalty0 ) = 0.000
InitializeForInput( TranslationModel0 ) = 0.000
InitializeForInput( TranslationModel1 ) = 0.000
InitializeForInput( GenerationModel0 ) = 0.000
InitializeForInput( GenerationModel1 ) = 0.000
InitializeForInput( Distortion0 ) = 0.000
InitializeForInput( LM0 ) = 0.000
InitializeForInput( LM1 ) = 0.000
Line 0: Initialize search took 0.000 seconds total
Translation Option Collection

Total translation options: 1
Total translation options pruned: 0
translation options spanning from 0 to 0 is 1
translation options generated in total: 1
future cost from 0 to 0 is -101.128
Line 0: Collecting options took 0.000 seconds
added hyp to stack, best on stack, now size 1
Cube Pruning pop limit is 2000
Cube Pruning diversity is 0

creating hypothesis 1 from 0 ( <s> )
base score 0.000
covering 0-0: agindo|V|agir
translated as: agindo|V|agir|UNK
score -inf + future cost 0.000 = -inf
unweighted feature scores: core=(-100.000,-1.000,1.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-8.571,-inf)
discarded, constraint
new stack entry flag is 0
processing hypothesis from next stack
Line 0: Search took 0.000 seconds
total hypotheses considered = 2
number popped from cube = 1
number not built = 0
number discarded early = 0
number discarded = 1
number recombined = 0
number pruned = 0
time to collect opts 0.000 (29%)
create hyps 0.000 (0%)
estimate score 0.000 (0%)
calc lm 0.000 (0%)
other hyp score 0.000 (2%)
set up cubes 0.000 (0%)
manage cubes 0.000 (0%)
manage stacks 0.000 (0%)
other 0.000 (66%)
total source words = 1
words deleted = 0 ()
words inserted = 0 ()
NO BEST TRANSLATION
Line 0: Decision rule took 0.000 seconds total
Line 0: Additional reporting took 0.000 seconds total
Line 0: Translation took 0.000 seconds total
Sentence Decoding Time: : [28.526] seconds
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.pruned.ini
Type: application/octet-stream
Size: 1552 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20140411/f2d60eab/attachment.obj

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 90, Issue 31
*********************************************

Tweet