Home » Uncategories » Moses-support Digest, Vol 115, Issue 33

Moses-support Digest, Vol 115, Issue 33

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: "Feature name SRILM is not registered." (Anna Garbar)
2. Re: "Feature name SRILM is not registered." (Kenneth Heafield)
3. extract.o.sorted.gz not being created in training (Purvik Shah)
4. Re: "Feature name SRILM is not registered." (Sa?o Kuntaric)


----------------------------------------------------------------------

Message: 1
Date: Sun, 29 May 2016 23:19:18 +0200
From: Anna Garbar <anna.garbar@gmail.com>
Subject: Re: [Moses-support] "Feature name SRILM is not registered."
To: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAHbd-+jwk=sEiTWzbC3VB5qm74_-80Z9bqjgV198xH2RVxziQg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Sa?o,

Thanks for your reply. Before recompiling moses with srilm, I also tried to
changed SRILM to KenLM im the moses.ini (under feature functions), but
received "Feature name KenLM is not registered." Or did I have to make any
other changes?

Best,
Anna

2016-05-29 23:10 GMT+02:00 Sa?o Kuntaric <saso.kuntaric@gmail.com>:

> Hi Anna,
>
> You are probably using KenLM as it's the default language model making
> tool. The factored tutorial however has the parameter for using SRILM. In
> the "lm 0:3:factored-corpus/surface.lm:0" part of the command, leave the
> last zero parameter ":0" out. Moses will then use the default KenLM tool.
>
> To the website authors: maybe it would be a good thing to mention this in
> the tutorial as I had the same issue.
>
> Best regards,
>
> Sa?o
>
> 2016-05-29 22:48 GMT+02:00 Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
> :
>
>> Hi,
>>
>> Since you are using SRILM have you compiled Moses with this flag "
>> --with-srilm=/path/to/srilm" (look here for various compilation options
>> http://www.statmt.org/moses/?n=Development.GetStarted).
>>
>> On Sun, May 29, 2016 at 9:48 PM, Anna Garbar <anna.garbar@gmail.com>
>> wrote:
>>
>>> Dear Moses Team,
>>>
>>> I am working through the factored tutorial (
>>> http://www.statmt.org/moses/?n=Moses.FactoredTutorial) and when trying
>>> out the pos model on a sample sentence ('putin beschreibt menschen .'),
>>> I receive
>>>
>>> Exception: moses\FF\Factory.cpp:321 in void
>>> Moses::FeatureRegistry::Construct(const string&, const string&) threw
>>> UnknownFeatureException because `i == registry_.end()'.
>>> Feature name SRILM is not registered.
>>>
>>> The last command was:
>>>
>>> /home/AG/mosesdecoder/bin/moses -f pos/model/moses.ini < in
>>>
>>> Where should I look for mistake?
>>>
>>> Thanks in advance,
>>> Anna
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> -Regards,
>> Rajen Chatterjee.
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> lp,
>
> Sa?o
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160529/3e90cb08/attachment-0001.html

------------------------------

Message: 2
Date: Sun, 29 May 2016 22:23:34 +0100
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] "Feature name SRILM is not registered."
To: moses-support@mit.edu
Message-ID: <574B5DD6.9070808@kheafield.com>
Content-Type: text/plain; charset=windows-1252

It's KENLM, not KenLM according to Hieu, who did name it after all.

Kenneth

On 05/29/2016 10:19 PM, Anna Garbar wrote:
> Hi Sa?o,
>
> Thanks for your reply. Before recompiling moses with srilm, I also tried
> to changed SRILM to KenLM im the moses.ini (under feature functions),
> but received "Feature name KenLM is not registered." Or did I have to
> make any other changes?
>
> Best,
> Anna
>
> 2016-05-29 23:10 GMT+02:00 Sa?o Kuntaric <saso.kuntaric@gmail.com
> <mailto:saso.kuntaric@gmail.com>>:
>
> Hi Anna,
>
> You are probably using KenLM as it's the default language model
> making tool. The factored tutorial however has the parameter for
> using SRILM. In the "lm 0:3:factored-corpus/surface.lm:0" part of
> the command, leave the last zero parameter ":0" out. Moses will then
> use the default KenLM tool.
>
> To the website authors: maybe it would be a good thing to mention
> this in the tutorial as I had the same issue.
>
> Best regards,
>
> Sa?o
>
> 2016-05-29 22:48 GMT+02:00 Rajen Chatterjee
> <rajen.k.chatterjee@gmail.com <mailto:rajen.k.chatterjee@gmail.com>>:
>
> Hi,
>
> Since you are using SRILM have you compiled Moses with this flag
> "--with-srilm=/path/to/srilm" (look here for various compilation
> options http://www.statmt.org/moses/?n=Development.GetStarted).
>
> On Sun, May 29, 2016 at 9:48 PM, Anna Garbar
> <anna.garbar@gmail.com <mailto:anna.garbar@gmail.com>> wrote:
>
> Dear Moses Team,
>
> I am working through the factored tutorial
> (http://www.statmt.org/moses/?n=Moses.FactoredTutorial) and
> when trying out the pos model on a sample sentence ('putin
> beschreibt menschen .'), I receive
>
> Exception: moses\FF\Factory.cpp:321 in void
> Moses::FeatureRegistry::Construct(const string&, const
> string&) threw UnknownFeatureException because `i ==
> registry_.end()'.
> Feature name SRILM is not registered.
>
> The last command was:
>
> /home/AG/mosesdecoder/bin/moses -f pos/model/moses.ini < in
>
> Where should I look for mistake?
>
> Thanks in advance,
> Anna
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> -Regards,
> Rajen Chatterjee.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> --
> lp,
>
> Sa?o
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 3
Date: Mon, 30 May 2016 14:19:59 +0530
From: Purvik Shah <spurvik07@gmail.com>
Subject: [Moses-support] extract.o.sorted.gz not being created in
training
To: moses-support@mit.edu
Message-ID:
<CALyMehrteK6vHkmB1jeMdQzqUF63RBWHX5uvstyCM=G929_t0A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi!

I'm training the system using this command.

nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir train \
-corpus ~/corpus/news-commentary-v8.fr-en.clean \
-f fr -e en -alignment grow-diag-final-and -reordering msd-bidirectional-fe \
-lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8 \
-external-bin-dir ~/mosesdecoder/tools >& training.out &


It fails to create moses.ini
I checked the log and apparently it fails to create extract.o.sorted.gz
in /working/train/model directory.
It also produces following errors.

gunzip -c /home/purvik/working/train/model/tmp.6608/extract.0000000.inv.gz
/home/purvik/working/train/model/tmp.6608/extract.0000001.inv.gz
/home/purvik/working/train/model/tmp.6608/extract.0000002.inv.gz
/home/purvik/working/train/model/tmp.6608/extract.0000003.inv.gz |
LC_ALL=C sort -T /home/purvik/working/train/model/tmp.6608 2>>
/dev/stderr | gzip -c >
/home/purvik/working/train/model/extract.inv.sorted.gz 2>> /dev/stderr
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000000.gz: No such
file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000001.gz: No such
file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000002.gz: No such
file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000003.gz: No such
file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000000.inv.gz: No
such file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000001.inv.gz: No
such file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000002.inv.gz: No
such file or directory
gzip: /home/purvik/working/train/model/tmp.6608/extract.0000003.inv.gz: No
such file or directory

training.out is at this link
https://drive.google.com/file/d/0B_dJFhc_wcd5SGxSeDFFTlM1X2c/view?usp=sharing
Thanks!
--
Purvik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160530/1f7b3dc7/attachment-0001.html

------------------------------

Message: 4
Date: Mon, 30 May 2016 10:52:50 +0200
From: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Subject: Re: [Moses-support] "Feature name SRILM is not registered."
To: Kenneth Heafield <moses@kheafield.com>
Cc: moses-support@mit.edu
Message-ID:
<CANsquDpnQWffzkUkivjcQDXwwR7qr-XxZDZso+LBY3+WNR4DZA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Anna,

I wouldn't change it manually, especially since you have multiple lms in
factored training and you can get the "old and new format don't mix" error.
Just use the command that Kenneth now changed on the website and you'll be
fine.

p.s. No problem Kenneth, we don't mind giving you extra work :-).

Best regards,

Sa?o

2016-05-29 23:23 GMT+02:00 Kenneth Heafield <moses@kheafield.com>:

> It's KENLM, not KenLM according to Hieu, who did name it after all.
>
> Kenneth
>
> On 05/29/2016 10:19 PM, Anna Garbar wrote:
> > Hi Sa?o,
> >
> > Thanks for your reply. Before recompiling moses with srilm, I also tried
> > to changed SRILM to KenLM im the moses.ini (under feature functions),
> > but received "Feature name KenLM is not registered." Or did I have to
> > make any other changes?
> >
> > Best,
> > Anna
> >
> > 2016-05-29 23:10 GMT+02:00 Sa?o Kuntaric <saso.kuntaric@gmail.com
> > <mailto:saso.kuntaric@gmail.com>>:
> >
> > Hi Anna,
> >
> > You are probably using KenLM as it's the default language model
> > making tool. The factored tutorial however has the parameter for
> > using SRILM. In the "lm 0:3:factored-corpus/surface.lm:0" part of
> > the command, leave the last zero parameter ":0" out. Moses will then
> > use the default KenLM tool.
> >
> > To the website authors: maybe it would be a good thing to mention
> > this in the tutorial as I had the same issue.
> >
> > Best regards,
> >
> > Sa?o
> >
> > 2016-05-29 22:48 GMT+02:00 Rajen Chatterjee
> > <rajen.k.chatterjee@gmail.com <mailto:rajen.k.chatterjee@gmail.com
> >>:
> >
> > Hi,
> >
> > Since you are using SRILM have you compiled Moses with this flag
> > "--with-srilm=/path/to/srilm" (look here for various compilation
> > options http://www.statmt.org/moses/?n=Development.GetStarted).
> >
> > On Sun, May 29, 2016 at 9:48 PM, Anna Garbar
> > <anna.garbar@gmail.com <mailto:anna.garbar@gmail.com>> wrote:
> >
> > Dear Moses Team,
> >
> > I am working through the factored tutorial
> > (http://www.statmt.org/moses/?n=Moses.FactoredTutorial) and
> > when trying out the pos model on a sample sentence ('putin
> > beschreibt menschen .'), I receive
> >
> > Exception: moses\FF\Factory.cpp:321 in void
> > Moses::FeatureRegistry::Construct(const string&, const
> > string&) threw UnknownFeatureException because `i ==
> > registry_.end()'.
> > Feature name SRILM is not registered.
> >
> > The last command was:
> >
> > /home/AG/mosesdecoder/bin/moses -f pos/model/moses.ini < in
> >
> > Where should I look for mistake?
> >
> > Thanks in advance,
> > Anna
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > --
> > -Regards,
> > Rajen Chatterjee.
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > --
> > lp,
> >
> > Sa?o
> >
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



--
lp,

Sa?o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160530/3c3d5317/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 115, Issue 33
**********************************************

0 Response to "Moses-support Digest, Vol 115, Issue 33"

Post a Comment

Subscribe to: Post Comments (Atom)