Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Moses Error (Hieu Hoang)
2. Re: Language Model Training failed (Nicola Bertoldi)
3. Re: Language Model Training failed (Seth Jarrett)
4. Re: Language Model Training failed (Barry Haddow)
5. Re: Language Model Training failed (Janez Kadivec)
----------------------------------------------------------------------
Message: 1
Date: Wed, 05 Mar 2014 07:50:16 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses Error
To: Rohit More <rohit4690@gmail.com>, moses-support@mit.edu
Message-ID: <5316D738.5080006@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
You seem to be missing the bz2 library on your computer.
This webpage describes how you can add it to your computer, depending on
what Linux distro you are useing
http://www.statmt.org/moses/?n=Development.GetStarted
On 04/03/2014 17:41, Rohit More wrote:
> Command used:
>
> ./bjam --with-srilm=/home/rohit/Study/moses/tools/srilm/
> --with-giza=/home/rohit/Study/moses/tools/bin/ >build.log
>
> --
> Rohit More.
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140305/27382294/attachment-0001.htm
------------------------------
Message: 2
Date: Wed, 5 Mar 2014 08:51:17 +0000
From: Nicola Bertoldi <bertoldi@fbk.eu>
Subject: Re: [Moses-support] Language Model Training failed
To: Janez Kadivec <jankad@zop-cr.com>
Cc: moses-support support <moses-support@mit.edu>
Message-ID: <990269C4-DA7A-4ABA-840B-CA06E3745E17@fbk.eu>
Content-Type: text/plain; charset="us-ascii"
Hi Janez,
Seth syggested you the right fix
I just checked the IRSTLM documentation
http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=Estimating_gigantic_models
and the correct notation is reported there.
Could you please tell me from where do you get the "wrong" information
So that I correct it.
Nicola
(on behalf of IRSTLM development team)
On Mar 5, 2014, at 1:36 AM, Seth Jarrett wrote:
First four commands were executed successfuly. The last one failed. Here
is the result after entering the following command line:zzz <at>
zzz-laptop:~/lm$ ~/moses/irstlm/bin/compile-lm --text
news-commentary-v8.fr-en.lm.en.gz news-commentary-v8.fr-en.arpa.en
inpfile: news-commentary-v8.fr-en.arpa.enloading up to the LM level 1000
(if any)
dub: 10000000Failed to open news-commentary-v8.fr-en.arpa.en!zzz <at>
zzz-laptop:~/lm$ ----------------Where we made a mistake? I see the
xxx.arpa.en listed as input file. Shouldn't be the xxx.arpa.en file an
output file?Best regards!
I was having the same problem when following the steps in the baseline
instructions but I was able to get it to work by adding "yes" after --text.
Try this:
~/moses/irstlm/bin/compile-lm --text yes news-commentary-v8.fr-en.lm.en.gz
news-commentary-v8.fr-en.arpa.en
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
------------------------------
Message: 3
Date: Wed, 5 Mar 2014 08:56:46 +0000 (UTC)
From: Seth Jarrett <seth@idanet.co.jp>
Subject: Re: [Moses-support] Language Model Training failed
To: moses-support@mit.edu
Message-ID: <loom.20140305T095337-575@post.gmane.org>
Content-Type: text/plain; charset=us-ascii
> I just checked the IRSTLM documentation
>
http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=Estimating_gigantic_models
> and the correct notation is reported there.
>
> Could you please tell me from where do you get the "wrong" information
> So that I correct it.
>
> Nicola
> (on behalf of IRSTLM development team)
Hi Nicola,
The problem is in the instructions on the Moses/Baseline page in the IRSTLM
section:
http://www.statmt.org/moses/?n=Moses.Baseline#irstlm
I found the correct notation using "compile-lm --help" so it looks like only
the baseline page has this problem. But it's pretty important for beginners!
Seth
------------------------------
Message: 4
Date: Wed, 05 Mar 2014 09:43:32 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Language Model Training failed
To: Nicola Bertoldi <bertoldi@fbk.eu>, Janez Kadivec
<jankad@zop-cr.com>
Cc: moses-support support <moses-support@mit.edu>
Message-ID: <5316F1C4.2030705@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi Nicola
When I tried with irstlm 5.80.03, the version mentioned on the Moses
baseline page (http://www.statmt.org/moses/?n=Moses.Baseline), it did
not like the "yes". Has there been a change in irstlm? I can check again.
There has been some history with this argument. You can see in the wiki
history of the Moses baseline page that the "yes" was added (because
some users reported problems) then removed (because other users reported
problems). Clarification of what works with what version of irstlm would
be very useful,
cheers - Barry
On 05/03/14 08:51, Nicola Bertoldi wrote:
> Hi Janez,
>
> Seth syggested you the right fix
>
> I just checked the IRSTLM documentation
> http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=Estimating_gigantic_models
> and the correct notation is reported there.
>
> Could you please tell me from where do you get the "wrong" information
> So that I correct it.
>
>
> Nicola
> (on behalf of IRSTLM development team)
>
>
>
> On Mar 5, 2014, at 1:36 AM, Seth Jarrett wrote:
>
> First four commands were executed successfuly. The last one failed. Here
> is the result after entering the following command line:zzz <at>
> zzz-laptop:~/lm$ ~/moses/irstlm/bin/compile-lm --text
> news-commentary-v8.fr-en.lm.en.gz news-commentary-v8.fr-en.arpa.en
>
> inpfile: news-commentary-v8.fr-en.arpa.enloading up to the LM level 1000
> (if any)
> dub: 10000000Failed to open news-commentary-v8.fr-en.arpa.en!zzz <at>
> zzz-laptop:~/lm$ ----------------Where we made a mistake? I see the
> xxx.arpa.en listed as input file. Shouldn't be the xxx.arpa.en file an
> output file?Best regards!
>
>
> I was having the same problem when following the steps in the baseline
> instructions but I was able to get it to work by adding "yes" after --text.
>
> Try this:
>
> ~/moses/irstlm/bin/compile-lm --text yes news-commentary-v8.fr-en.lm.en.gz
> news-commentary-v8.fr-en.arpa.en
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu<mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 5
Date: Wed, 5 Mar 2014 10:43:52 +0100
From: Janez Kadivec <jankad@zop-cr.com>
Subject: Re: [Moses-support] Language Model Training failed
To: Nicola Bertoldi <bertoldi@fbk.eu>
Cc: moses-support support <moses-support@mit.edu>
Message-ID:
<CA+viJseu1i93zpZSk59syhEytgvGa92LuQ7SSz8eXk8RLnz4-Q@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hi,
thank you for your help. The added "yes" parameter resolved the situation.
We are following the Moses baseline, published in the official Moses web
site:
http://www.statmt.org/moses/?n=moses.baseline
Please correct the last command in the Language Model Traning section. The
command is marked with red color.
mkdir ~/lm
cd ~/lm
~/irstlm/bin/add-start-end.sh \
< ~/corpus/news-commentary-v8.fr-en.true.en \
> news-commentary-v8.fr-en.sb.en
export IRSTLM=$HOME/irstlm; ~/irstlm/bin/build-lm.sh \
-i news-commentary-v8.fr-en.sb.en \
-t ./tmp -p -s improved-kneser-ney -o
news-commentary-v8.fr-en.lm.en* ~/irstlm/bin/compile-lm --text
news-commentary-v8.fr-en.lm.en.gz \
news-commentary-v8.fr-en.arpa.en*
....................
We followed the same baseline. In the Training the Translation System we
found the next inconsistency:
We installed the Moses and part of it was also Giza++ installed under the
...mosesdecoder\giza++1.0.7.
We executed the following commands from the Training the Translation System
section:
mkdir ~/working
cd ~/working
nohup nice ~/mosesdecoder/scripts/training/train-model.perl -root-dir train \
-corpus ~/corpus/news-commentary-v8.fr-en.clean \
-f fr -e en -alignment grow-diag-final-and -reordering msd-bidirectional-fe \
-lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8 \
-external-bin-dir ~/mosesdecoder/tools >& training.out &
During the execution of the last command (marked with green color) here was
an error saying that the mkcls utility is not found.
It's not found because in the "initial" installation there is no tools
subdirectory. We renamed the Giza++107 directory to "tools".
The command was executed instantly with no results. So we removed the last
character "&" from the command. It's working now for about half an hour. ;)
In my opinion there are two things that need to be somehow described or
corrected in the Moses baseline:
1. Notify the user about the location of the Giza++ utilities
(mosesdecoder/tools or mosesdecoder/giza++) and need to rename the folders
to the one used in command.
2. Remove the last "&" char in the command, listed in the baseline.
Have a nice rest of the day.
Janez
Seth syggested you the right fix
>
> I just checked the IRSTLM documentation
>
> http://sourceforge.net/apps/mediawiki/irstlm/index.php?title=Estimating_gigantic_models
> and the correct notation is reported there.
>
> Could you please tell me from where do you get the "wrong" information
> So that I correct it.
>
>
> Nicola
> (on behalf of IRSTLM development team)
>
>
>
> On Mar 5, 2014, at 1:36 AM, Seth Jarrett wrote:
>
> First four commands were executed successfuly. The last one failed. Here
> is the result after entering the following command line:zzz <at>
> zzz-laptop:~/lm$ ~/moses/irstlm/bin/compile-lm --text
> news-commentary-v8.fr-en.lm.en.gz news-commentary-v8.fr-en.arpa.en
>
> inpfile: news-commentary-v8.fr-en.arpa.enloading up to the LM level 1000
> (if any)
> dub: 10000000Failed to open news-commentary-v8.fr-en.arpa.en!zzz <at>
> zzz-laptop:~/lm$ ----------------Where we made a mistake? I see the
> xxx.arpa.en listed as input file. Shouldn't be the xxx.arpa.en file an
> output file?Best regards!
>
>
> I was having the same problem when following the steps in the baseline
> instructions but I was able to get it to work by adding "yes" after --text.
>
> Try this:
>
> ~/moses/irstlm/bin/compile-lm --text yes news-commentary-v8.fr-en.lm.en.gz
> news-commentary-v8.fr-en.arpa.en
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu<mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
Call
Send SMS
Add to Skype
You'll need Skype CreditFree via Skype
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140305/936f843a/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 89, Issue 9
********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 89, Issue 9"
Post a Comment