Moses-support Digest, Vol 92, Issue 13

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. n-best-list diversity (Marcin Junczys-Dowmunt)
2. Re: Regarding Moses Server (Philipp Koehn)
3. Re: Segmentation fault (Philipp Koehn)
4. Re: Segmentation fault (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Thu, 05 Jun 2014 17:23:39 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: [Moses-support] n-best-list diversity
To: moses-support <moses-support@mit.edu>
Message-ID: <53908B7B.9000905@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed

Hi list,
is there a way to make the n-best list in Moses a little bit more
diverse without increasing the maximum n-best list size? Let's say I am
willing to sacrifice decoding speed.

For instance

-n-best-list file 100 distinct

generates 100 and collapses them, so sometimes there may be only one
hypothesis. I can get more distinct hypotheses by setting the size to
e.g. 1000, but then some sentences might actually come up with 1000
distinct hypotheses which is overkill.

The options -cube-pruning-diversity and -stack-diversity sound
promising. I should add that this is a minolingual task with reordering
switched off. I understand this has some impact on these options?
Best,
Marcin


------------------------------

Message: 2
Date: Thu, 5 Jun 2014 11:35:22 -0400
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Regarding Moses Server
To: Pranjal Das <pranjal4456@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAAFADDDzMbT_t2hzTWXh_EHCBXXQ_ciVGVmhCbfH-ggX6Mtc4w@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

the error method spells out what you have to do:

- select the hostname (maybe "localhost" or the IP of the machine that
you can get by running ifconfig)
- select a port (could be anything: port numbers in the range
8000-9999 are popular)
- the moses.ini configuration file

With these three values you can start the daemon script.

-phi

On Thu, Jun 5, 2014 at 6:19 AM, Pranjal Das <pranjal4456@gmail.com> wrote:
> Hello, I am trying to install the moses server, but i am not able to
> activate the server. I am stuck at ./daemon.pl <hostname> <port>. I am
> getting this error:
>
> usage: daemon.pl <hostname> <port> <ini> at ./daemon.pl line 33.
> How should i proceed ?
>
> Regards,
>
> Pranjal Das
> Department of Information Technology,
> Gauhati University Institute of Science and Technology,
> Phone- +91-8399879454
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 3
Date: Thu, 5 Jun 2014 11:44:00 -0400
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Segmentation fault
To: charmaine ponay <csponay@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDBC7=F0n3CHuKP4wPX0Ld-EtBx7GBK9+m77FU_Kas8aKQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

there seems to be some oddity since Moses assumes to have to load a
SCFG phrase table.

In any case:

It is better not to run Moses with a plain "PhraseDictionaryMemory"
which is very
inefficient and uses a lot of RAM. Rather convert the phrase table first into
compact or binary format.

http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc10

http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc10

-phi

On Thu, Jun 5, 2014 at 12:55 AM, charmaine ponay <csponay@gmail.com> wrote:
> hi, what does it mean when i get this error after escaping special
> characters and training my corpus?
>
> $ ../mosesdecoder-master/bin/moses -f Charmaine/training/model/moses.ini <
> Charmaine/testing.fl > Charmaine/Testing/CharmaineOut
> Defined parameters (per moses.ini or switch):
> config: Charmaine/training/model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=4
> path=/masters/Charmaine/training/model/phrase-table.gz input-factor=0
> output-factor=0 Distortion SRILM name=LM0 factor=0
> path=/masters/Charmaine/preprocessed/training.clean.en.lm order=3
> input-factors: 0
> mapping: 0 T 0
> weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2 Distortion0= 0.3 LM0= 0.5
> ../mosesdecoder-master/bin
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> num-features=4 path=/masters/Charmaine/training/model/phrase-table.gz
> input-factor=0 output-factor=0
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=Distortion
> FeatureFunction: Distortion0 start: 7 end: 7
> line=SRILM name=LM0 factor=0
> path=/masters/Charmaine/preprocessed/training.clean.en.lm order=3
> FeatureFunction: LM0 start: 8 end: 8
> Start loading text SCFG phrase table. Moses format : [0.000] seconds
> Reading /masters/Charmaine/training/model/phrase-table.gz
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> Segmentation fault (core dumped)
>
> Regards,
>
> Charmaine Salvador - Ponay
> Instructor
> Information and Computer Studies Dept.
> University of Santo Tomas
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 4
Date: Thu, 5 Jun 2014 17:17:17 +0100
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Segmentation fault
To: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, charmaine ponay
<csponay@gmail.com>
Message-ID:
<CAEKMkbhO8CMDOe6+_MFET7-f=TTkjeSm0p0O9rECjvP5bV5GZA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

On 5 June 2014 16:44, Philipp Koehn <pkoehn@inf.ed.ac.uk> wrote:

> Hi,
>
> there seems to be some oddity since Moses assumes to have to load a
> SCFG phrase table.
>
The memory phrase table is for both phrase-based and SCFG models. I've just
changed the debugging output to avoid confusion


>
> In any case:
>
> It is better not to run Moses with a plain "PhraseDictionaryMemory"
> which is very
> inefficient and uses a lot of RAM. Rather convert the phrase table first
> into
> compact or binary format.
>
> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc10
>
> http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc10
>
> -phi
>
> On Thu, Jun 5, 2014 at 12:55 AM, charmaine ponay <csponay@gmail.com>
> wrote:
> > hi, what does it mean when i get this error after escaping special
> > characters and training my corpus?
> >
> > $ ../mosesdecoder-master/bin/moses -f Charmaine/training/model/moses.ini
> <
> > Charmaine/testing.fl > Charmaine/Testing/CharmaineOut
> > Defined parameters (per moses.ini or switch):
> > config: Charmaine/training/model/moses.ini
> > distortion-limit: 6
> > feature: UnknownWordPenalty WordPenalty PhrasePenalty
> > PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> num-features=4
> > path=/masters/Charmaine/training/model/phrase-table.gz input-factor=0
> > output-factor=0 Distortion SRILM name=LM0 factor=0
> > path=/masters/Charmaine/preprocessed/training.clean.en.lm order=3
> > input-factors: 0
> > mapping: 0 T 0
> > weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0=
> 0.2
> > TranslationModel0= 0.2 0.2 0.2 0.2 Distortion0= 0.3 LM0= 0.5
> > ../mosesdecoder-master/bin
> > line=UnknownWordPenalty
> > FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> > line=WordPenalty
> > FeatureFunction: WordPenalty0 start: 1 end: 1
> > line=PhrasePenalty
> > FeatureFunction: PhrasePenalty0 start: 2 end: 2
> > line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
> > num-features=4 path=/masters/Charmaine/training/model/phrase-table.gz
> > input-factor=0 output-factor=0
> > FeatureFunction: TranslationModel0 start: 3 end: 6
> > line=Distortion
> > FeatureFunction: Distortion0 start: 7 end: 7
> > line=SRILM name=LM0 factor=0
> > path=/masters/Charmaine/preprocessed/training.clean.en.lm order=3
> > FeatureFunction: LM0 start: 8 end: 8
> > Start loading text SCFG phrase table. Moses format : [0.000] seconds
> > Reading /masters/Charmaine/training/model/phrase-table.gz
> >
> ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
> > Segmentation fault (core dumped)
> >
> > Regards,
> >
> > Charmaine Salvador - Ponay
> > Instructor
> > Information and Computer Studies Dept.
> > University of Santo Tomas
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140605/35b8b39a/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 92, Issue 13
*********************************************

Tweet