Home » Uncategories » Moses-support Digest, Vol 112, Issue 32

Moses-support Digest, Vol 112, Issue 32

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: EMS appears to require SRILM for OSM, even when not
interpolating (Lane Schwartz)
2. Bug? Using Multiple Generation Models (Michaeel Kazi)
3. Re: EMS appears to require SRILM for OSM, even when not
interpolating (Nadir Durrani)


----------------------------------------------------------------------

Message: 1
Date: Tue, 16 Feb 2016 12:10:05 -0600
From: Lane Schwartz <dowobeha@gmail.com>
Subject: Re: [Moses-support] EMS appears to require SRILM for OSM,
even when not interpolating
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CABv3vZk6JZFAhA1DYVB7Gzf0+jUoZ+twBt0ZjAWhU+BHpJ6Mug@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Great. Will do. This was with the moses v3 release.


On Tue, Feb 16, 2016 at 10:28 AM, Barry Haddow <bhaddow@staffmail.ed.ac.uk>
wrote:

> Hi Lane
>
> SRILM is no longer required, since Nadir made some EMS updates last
> October. Try upgrading to a recent version,
>
> cheers - Barry
>
>
> On 16/02/16 15:04, Lane Schwartz wrote:
>
> Hi,
>
> This is mostly an FYI, but I thought I'd point it out. The OSM
> documentation (http://www.statmt.org/moses/?n=Advanced.Models#ntoc3)
> mentions that SRILM is required when training an interpolated OSM model.
> This makes sense, because KenLM currently doesn't support interpolation.
>
> However, the documentation doesn't state that SRILM is required for
> (non-interpolated) OSM training. This also makes sense, because regular OSM
> training can use KenLM.
>
> The problem comes when running EMS using OSM when srilm-dir is not defined
> in your config file. My experience is that doing so results in the error:
> "ERROR: you need to define GENERAL:srilm-dir".
>
> operation-sequence-model = "yes"
>> operation-sequence-model-order = 5
>> operation-sequence-model-settings = ""
>
>
> As far as I can tell, this is because the build-osm section of
> experiment.meta references the variable $srilm-dir:
>
> build-osm
>> in: corpus word-alignment
>> out: osm-model
>> ignore-unless: operation-sequence-model
>> rerun-on-change: operation-sequence-model training-options script
>> giza-settings operation-sequence-model-settings
>> template: $moses-script-dir/OSM/OSM-Train.perl --corpus-f
>> IN0.$input-extension --corpus-e IN0.$output-extension --alignment
>> IN1.$alignment-symmetrization-method --order
>> $operation-sequence-model-order --out-dir OUT --moses-src-dir
>> $moses-src-dir --srilm-dir $srilm-dir $operation-sequence-model-settings
>> default-name: model/OSM
>
>
> I'm not terribly familiar with this code, but it seems that a solution
> would be to specify --lmplz instead of --srilm-dir in the build-osm section
> of experiment.meta, since OSM-Train.perl accepts --lmplz as an alternative
> to --srilm-dir.
>
> Thanks,
> Lane
>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>


--
When a place gets crowded enough to require ID's, social collapse is not
far away. It is time to go elsewhere. The best thing about space travel
is that it made it possible to go elsewhere.
-- R.A. Heinlein, "Time Enough For Love"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160216/8e90379f/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 16 Feb 2016 20:35:46 +0000 (UTC)
From: Michaeel Kazi <michaeel.kazi@ll.mit.edu>
Subject: [Moses-support] Bug? Using Multiple Generation Models
To: moses-support@mit.edu
Message-ID: <loom.20160216T203215-720@post.gmane.org>
Content-Type: text/plain; charset=us-ascii

I'm trying to decode using two factors and two Generation Models. This used
to work (using a moses repository from approximately Feb of 2015), but no
longer does. It generates blank translations.

Debugging its workings, I've found that the first GM works fine, but the
second one, it seems to try to look up the word plus the first factor
(instead of just the input factor), and fails:

Generation Model (./models/lc.classes.gm): Word = announcement found of size 1
Generation Model (./models/lc.classes.gm): Word = 92 found of size 1
Generation Model (./models/lc.classes.gm): Word = an found of size 1
Generation Model (./models/lc.classes.gm): Word = see found of size 1
Generation Model (./models/lc.classes.gm): Word = field found of size 1
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word =
announcement|67 NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word = 92|55
NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word = an|18
NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word = see|58
NOT FOUND!
Generation Model (/localdata/mkazi/lmCache/nyt600.classes.gm): Word =
field|67 NOT FOUND!

I ran these same debug prints through the old code, and while the word also
had factors appended to it, it successfully found the match. So I think this
has to do with the underlying collection used by the GenerationDictionary:

Current version:
typedef boost::unordered_map<const Word* , OutputWordCollection,
UnorderedComparer<Word>, UnorderedComparer<Word> > Collection;

Old version:
typedef std::map<const Word* , OutputWordCollection, WordComparer>
Collection;

Perhaps the older one only compared on the first factor. (It is unclear if
it made use of the 'input-factor' to GenerationModel, or if it just assumed
factor 0...)

I will probably hack around this issue for now in my own local repository. I
am timid to make changes since I am not sure what is the correct 'moses' way
to fix this problem.

Thanks all!



------------------------------

Message: 3
Date: Wed, 17 Feb 2016 10:37:48 +0300
From: Nadir Durrani <nadir.durrani@nu.edu.pk>
Subject: Re: [Moses-support] EMS appears to require SRILM for OSM,
even when not interpolating
To: moses-support@mit.edu
Message-ID:
<CAFDj2Q0nc9nK=07_TBgYyeX7+Jt8FZyVK2u=_4ObW+dN=dq8ZQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Lane,


I hope upgrading to a newer version will have solved your problem. The
training script

OSM-Train.perl

takes both




*'srilm-dir=s' => \$SRILM_DIR,'lmplz=s' => \$LMPLZ,*and if the user doesn't
specify any, which should trigger with

*operation-sequence-model-settings = ""*

then

if (!defined($LMPLZ)) {
$LMPLZ = "$MOSES_SRC_DIR/bin/lmplz";
}

should be default. Let me know if you are still having problem. Send me
your config and the command generated for build-osm step.

Cheers,
Nadir

On Tue, Feb 16, 2016 at 7:28 PM, <moses-support-request@mit.edu> wrote:

> Send Moses-support mailing list submissions to
> moses-support@mit.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/moses-support
> or, via email, send a message with subject or body 'help' to
> moses-support-request@mit.edu
>
> You can reach the person managing the list at
> moses-support-owner@mit.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Moses-support digest..."
>
>
> Today's Topics:
>
> 1. EMS appears to require SRILM for OSM, even when not
> interpolating (Lane Schwartz)
> 2. Xmlrpc-c Error 408 timeout (Marwa Refaie)
> 3. Re: Xmlrpc-c Error 408 timeout (Marwa Refaie)
> 4. Re: EMS appears to require SRILM for OSM, even when not
> interpolating (Barry Haddow)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 16 Feb 2016 09:04:56 -0600
> From: Lane Schwartz <dowobeha@gmail.com>
> Subject: [Moses-support] EMS appears to require SRILM for OSM, even
> when not interpolating
> To: "moses-support@mit.edu" <moses-support@mit.edu>
> Message-ID:
> <CABv3vZmODk=
> CvT3CcvEtoGkrcQ9HwvuhCJpLmxeAtSSiAtO0AQ@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> This is mostly an FYI, but I thought I'd point it out. The OSM
> documentation (http://www.statmt.org/moses/?n=Advanced.Models#ntoc3)
> mentions that SRILM is required when training an interpolated OSM model.
> This makes sense, because KenLM currently doesn't support interpolation.
>
> However, the documentation doesn't state that SRILM is required for
> (non-interpolated) OSM training. This also makes sense, because regular OSM
> training can use KenLM.
>
> The problem comes when running EMS using OSM when srilm-dir is not defined
> in your config file. My experience is that doing so results in the error:
> "ERROR: you need to define GENERAL:srilm-dir".
>
> operation-sequence-model = "yes"
> > operation-sequence-model-order = 5
> > operation-sequence-model-settings = ""
>
>
> As far as I can tell, this is because the build-osm section of
> experiment.meta references the variable $srilm-dir:
>
> build-osm
> > in: corpus word-alignment
> > out: osm-model
> > ignore-unless: operation-sequence-model
> > rerun-on-change: operation-sequence-model training-options script
> > giza-settings operation-sequence-model-settings
> > template: $moses-script-dir/OSM/OSM-Train.perl --corpus-f
> > IN0.$input-extension --corpus-e IN0.$output-extension --alignment
> > IN1.$alignment-symmetrization-method --order
> > $operation-sequence-model-order --out-dir OUT --moses-src-dir
> > $moses-src-dir --srilm-dir $srilm-dir $operation-sequence-model-settings
> > default-name: model/OSM
>
>
> I'm not terribly familiar with this code, but it seems that a solution
> would be to specify --lmplz instead of --srilm-dir in the build-osm section
> of experiment.meta, since OSM-Train.perl accepts --lmplz as an alternative
> to --srilm-dir.
>
> Thanks,
> Lane
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20160216/fe3da885/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Tue, 16 Feb 2016 15:45:30 +0000
> From: Marwa Refaie <basmallah@hotmail.com>
> Subject: [Moses-support] Xmlrpc-c Error 408 timeout
> To: Moses <moses-support@mit.edu>
> Message-ID: <DUB112-W1AF6AE02404FBBF16EC58BAAD0@phx.gbl>
> Content-Type: text/plain; charset="windows-1256"
>
>
> After run mosesserver successfully read TM & LM, listening to port 8088
> .Then when try to connect from the web as client to start transaltion I got
> this error !!
> HTTP/1.1 408 Request TimeoutContent-type: text/htmlConnection:
> closeDate: Tue, 16 Feb 2016 15:14:41 UTCServer: Xmlrpc-c_Abyss/1.33.17
> <HTML><HEAD><TITLE>Error 408</TITLE></HEAD><BODY><H1>Error
> 408</H1><P>Problem getting the request header</P><p><HR><b><i><a href="
> http://xmlrpc-c.sourceforge.net">ABYSS Web Server for XML-RPC For
> C/C++</a></i></b> version 1.33.17<br></p></BODY></HTML>
> What to configure or update !!
> Marwa N. Refaie
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20160216/2c12464d/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Tue, 16 Feb 2016 16:11:00 +0000
> From: Marwa Refaie <basmallah@hotmail.com>
> Subject: Re: [Moses-support] Xmlrpc-c Error 408 timeout
> To: Moses <moses-support@mit.edu>
> Message-ID: <DUB112-W129FD9E90F7046B5F98061DBAAD0@phx.gbl>
> Content-Type: text/plain; charset="windows-1256"
>
>
> Okk done
> client.perl
>
>
> Marwa N. Refaie
>
>
>
> From: basmallah@hotmail.com
> To: moses-support@mit.edu
> Date: Tue, 16 Feb 2016 15:45:30 +0000
> Subject: [Moses-support] Xmlrpc-c Error 408 timeout
>
>
>
>
>
> After run mosesserver successfully read TM & LM, listening to port 8088
> .Then when try to connect from the web as client to start transaltion I got
> this error !!
> HTTP/1.1 408 Request TimeoutContent-type: text/htmlConnection:
> closeDate: Tue, 16 Feb 2016 15:14:41 UTCServer: Xmlrpc-c_Abyss/1.33.17
> <HTML><HEAD><TITLE>Error 408</TITLE></HEAD><BODY><H1>Error
> 408</H1><P>Problem getting the request header</P><p><HR><b><i><a href="
> http://xmlrpc-c.sourceforge.net">ABYSS Web Server for XML-RPC For
> C/C++</a></i></b> version 1.33.17<br></p></BODY></HTML>
> What to configure or update !!
> Marwa N. Refaie
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20160216/60995908/attachment-0001.html
>
> ------------------------------
>
> Message: 4
> Date: Tue, 16 Feb 2016 16:28:36 +0000
> From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
> Subject: Re: [Moses-support] EMS appears to require SRILM for OSM,
> even when not interpolating
> To: Lane Schwartz <dowobeha@gmail.com>, "moses-support@mit.edu"
> <moses-support@mit.edu>
> Message-ID: <56C34E34.8000306@staffmail.ed.ac.uk>
> Content-Type: text/plain; charset="windows-1252"
>
> Hi Lane
>
> SRILM is no longer required, since Nadir made some EMS updates last
> October. Try upgrading to a recent version,
>
> cheers - Barry
>
> On 16/02/16 15:04, Lane Schwartz wrote:
> > Hi,
> >
> > This is mostly an FYI, but I thought I'd point it out. The OSM
> > documentation (http://www.statmt.org/moses/?n=Advanced.Models#ntoc3)
> > mentions that SRILM is required when training an interpolated OSM
> > model. This makes sense, because KenLM currently doesn't support
> > interpolation.
> >
> > However, the documentation doesn't state that SRILM is required for
> > (non-interpolated) OSM training. This also makes sense, because
> > regular OSM training can use KenLM.
> >
> > The problem comes when running EMS using OSM when srilm-dir is not
> > defined in your config file. My experience is that doing so results in
> > the error: "ERROR: you need to define GENERAL:srilm-dir".
> >
> > operation-sequence-model = "yes"
> > operation-sequence-model-order = 5
> > operation-sequence-model-settings = ""
> >
> >
> > As far as I can tell, this is because the build-osm section of
> > experiment.meta references the variable $srilm-dir:
> >
> > build-osm
> > in: corpus word-alignment
> > out: osm-model
> > ignore-unless: operation-sequence-model
> > rerun-on-change: operation-sequence-model training-options
> > script giza-settings operation-sequence-model-settings
> > template: $moses-script-dir/OSM/OSM-Train.perl --corpus-f
> > IN0.$input-extension --corpus-e IN0.$output-extension --alignment
> > IN1.$alignment-symmetrization-method --order
> > $operation-sequence-model-order --out-dir OUT --moses-src-dir
> > $moses-src-dir --srilm-dir $srilm-dir
> > $operation-sequence-model-settings
> > default-name: model/OSM
> >
> >
> > I'm not terribly familiar with this code, but it seems that a solution
> > would be to specify --lmplz instead of --srilm-dir in the build-osm
> > section of experiment.meta, since OSM-Train.perl accepts --lmplz as an
> > alternative to --srilm-dir.
> >
> > Thanks,
> > Lane
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20160216/f13ea80c/attachment.html
> -------------- next part --------------
> An embedded and charset-unspecified text was scrubbed...
> Name: not available
> Url:
> http://mailman.mit.edu/mailman/private/moses-support/attachments/20160216/f13ea80c/attachment.pl
>
> ------------------------------
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> End of Moses-support Digest, Vol 112, Issue 31
> **********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160217/c87f2337/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 112, Issue 32
**********************************************

0 Response to "Moses-support Digest, Vol 112, Issue 32"

Post a Comment

Subscribe to: Post Comments (Atom)