Home » Uncategories » Moses-support Digest, Vol 96, Issue 12

Moses-support Digest, Vol 96, Issue 12

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Factored Model | <s> Error ??? (Philipp Koehn)
2. Using berkeleyparsed2mosesxml.perl on Stanford Parser output
(Jeremy Gwinnup)
3. Factored LM / <s></s> (Marwa Refaie)
4. Re: Factored LM / <s></s> (Ondrej Bojar)
5. Error in Build_Binary (Fatemeh Eskandari)


----------------------------------------------------------------------

Message: 1
Date: Wed, 8 Oct 2014 13:18:32 -0400
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Factored Model | <s> Error ???
To: Marwa Refaie <basmallah@hotmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDAVo+JP6vPJKK3_1vMRH9d++-7wfF7Ua3OKT6VBv+ecBQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

you should first run tokenization, then add your factors,
to avoid this problem.

-phi

On Wed, Oct 8, 2014 at 12:17 PM, Marwa Refaie <basmallah@hotmail.com> wrote:
> When i applied moses tokeniser to the text file, all the '|' pipe as factor
> separator convert to "&#124" .... so can't produce the moses.ini as error ::
>
>
> ERROR: Couldn't find factor 1 in token "artificial" in
> c:/mosesdecoder-master/try/ai/sep/fsmt/data/UNpos.tok.en LINE 1
>
> Any advice please ??
>
>
>
>
>
> Marwa N. Refaie
>
>
>
>> Date: Tue, 7 Oct 2014 21:13:36 -0400
>> From: moses@kheafield.com
>> To: moses-support@mit.edu
>> Subject: Re: [Moses-support] Factored Model / <s> Error ???
>>
>> Well, does your data contain <s> in a position other than the first
>> word? If so you should be escaping it e.g. with the Moses tokenizer.
>>
>> On 10/07/14 20:32, Marwa Refaie wrote:
>> >
>> >
>> > Thank's for help ,, I fix all what mentioned now I'm stuck in this error
>> > ::
>> >
>> > Start loading text SCFG phrase table. Moses format : [1.000] seconds
>> > Reading
>> > /cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.
>> >
>> > 0,1-0,1.gz
>> >
>> > ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80
>> >
>> > ---85---90---95--100
>> > Either your data contains <s> in a position other than the first word or
>> > your la
>> > nguage model is missing <s>. Did you
>> > build your ARPA using IRSTLM and forget to
>> > run
>> > add-start-end.sh?
>> > [1]+ Done
>> > c:/mosesdecoder-master/scripts/training/train-mode
>> >
>> > l.perl -external-bin-dir c:/mosesdecoder-master/tools/bin -root-dir
>> > work -corpus
>> >
>> > c:/mosesdecoder-master/try/ai/sep/fsmt/data/UNpos.lo -f en -e ar
>> > -Translation-f
>> > actors 0,1-0,1 -lm
>> > 0:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/su
>> >
>> > rface.lm -lm
>> > 1:5:/cygdrive/c:/mosesdecoder-master/try/ai/sep/fsmt/model/pos.lm &
>> >
>> > >training.out
>> > Aborted (core dumped)
>> >
>> >
>>
>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>> > As mentioned I build non factored model , it worked well, but when I
>> > start use the pos & surface language model for my training data I got
>> > these errors ??
>> > I used the SRILM.
>> >
>> >
>> >
>> >
>> >
>> > /*Marwa N. Refaie*/
>> >
>> >
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 2
Date: Wed, 8 Oct 2014 15:09:35 -0400
From: Jeremy Gwinnup <jeremy@gwinnup.org>
Subject: [Moses-support] Using berkeleyparsed2mosesxml.perl on
Stanford Parser output
To: moses-support@mit.edu
Message-ID: <296D0E7E-DC17-43A8-BA17-120365C5FBC7@gwinnup.org>
Content-Type: text/plain; charset=us-ascii

All,

Has anyone used berkeleyparsed2mosesxml.perl to transform output of the Stanford Parser to the Moses tree format? Are there any gotchas I need to be aware of?

Thanks!
-Jeremy


------------------------------

Message: 3
Date: Thu, 9 Oct 2014 00:10:12 +0000
From: Marwa Refaie <basmallah@hotmail.com>
Subject: [Moses-support] Factored LM / <s></s>
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <DUB118-W49854AFAAB784E0B854E6DBAA00@phx.gbl>
Content-Type: text/plain; charset="windows-1256"

How I should fix this error ?? Tokenizing didn't differ !! how to normalize data or set sentence boundaries ???

Start loading text SCFG phrase table. Moses format : [1.000] secondsReading /cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table. 0,1-0,1.gz----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80 ---85---90---95--100Either your data contains <s> in a position other than the first word or your la nguage model is missing <s>. Did you build your ARPA using IRSTLM and forget to run add-start-end.sh?

Marwa N. Refaie


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141009/c8e960ad/attachment-0001.htm

------------------------------

Message: 4
Date: Thu, 09 Oct 2014 07:40:28 +0200
From: Ondrej Bojar <bojar@ufal.mff.cuni.cz>
Subject: Re: [Moses-support] Factored LM / <s></s>
To: Marwa Refaie <basmallah@hotmail.com>, "moses-support@mit.edu"
<moses-support@mit.edu>
Message-ID: <daad645b-878a-4d8c-9b3b-21d2a46be114@email.android.com>
Content-Type: text/plain; charset=UTF-8

Dear Marwa,

Try cutting the bad data in half and then in half again, etc. to get a very small input that still suffers from the error. Then you'll probably realize what is the problem or you can at least send it to the mailing list.

Cheers, O.


On October 9, 2014 2:10:12 AM CEST, Marwa Refaie <basmallah@hotmail.com> wrote:
>How I should fix this error ?? Tokenizing didn't differ !! how to
>normalize data or set sentence boundaries ???
>
>Start loading text SCFG phrase table. Moses format : [1.000]
>secondsReading
>/cygdrive/c/mosesdecoder-master/try/ai/sep/fsmt/work/model/phrase-table.
>0,1-0,1.gz----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80
>---85---90---95--100Either your data contains <s> in a position other
>than the first word or your la
>nguage model is missing <s>. Did you build your ARPA using IRSTLM and
>forget to
> run add-start-end.sh?
>
>Marwa N. Refaie
>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Moses-support mailing list
>Moses-support@mit.edu
>http://mailman.mit.edu/mailman/listinfo/moses-support

--
Ondrej Bojar (mailto:obo@cuni.cz / bojar@ufal.mff.cuni.cz)
http://www.cuni.cz/~obo




------------------------------

Message: 5
Date: Thu, 9 Oct 2014 13:39:28 +0330
From: Fatemeh Eskandari <fatemeh.eskandari.69@gmail.com>
Subject: [Moses-support] Error in Build_Binary
To: moses-support@mit.edu
Message-ID:
<CAOoq+SbOG0M4Aco86g4BkZ84f1q+FZPYuThWccTQQWEGuyQZWQ@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Dear Supporter,

I have started working with Moses recently. there is an issue in
build_binary in LM training part. I performed instructions just like as the
baseline page:

http://www.statmt.org/moses/?n=Moses.Baseline

but when I use the below command, I get the warning "Too many arguments".

/irstlm/bin/compile-lm \
--text yes \
news-commentary-v8.fr-en.lm.en.gz \
news-commentary-v8.fr-en.arpa.en

when I ignore the "--text yes" argument I faced with an ERROR "Did you
forget to pass --text yes to compile-lm?" in the next steps as follow:

~/mosesdecoder/bin/build_binary \
news-commentary-v8.fr-en.arpa.en \
news-commentary-v8.fr-en.blm.en


would you please guide me what should I do?

--
Best Regards,
Fatemeh Eskandari
--
M.Sc.
student
in
Artificial Intelligence
,
Computer Science and Engineering Dept.
Shiraz University, Iran
eskandari@cse.shirazu.ac.ir
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141009/c0bebb69/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 96, Issue 12
*********************************************

0 Response to "Moses-support Digest, Vol 96, Issue 12"

Post a Comment

Subscribe to: Post Comments (Atom)