Moses-support Digest, Vol 97, Issue 69

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Error in EMS for factor based SMT (Rajen Chatterjee)
2. Re: Error in EMS for factor based SMT (Hieu Hoang)
3. Re: Error in EMS for factor based SMT (Rajen Chatterjee)


----------------------------------------------------------------------

Message: 1
Date: Sat, 22 Nov 2014 11:51:18 +0100
From: Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
Subject: Re: [Moses-support] Error in EMS for factor based SMT
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAC4-+Nxt762WMX9U9Cc+aff4=s13MH6kphcT=KRGH=u8T+SVZg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Hieu,

It is "b" word1|word1-STEM|word1-POS word2|word2-STEM|word2-POS
word3|word3-STEM|word3-POS
I have already set factorized-stem=/path/to/src_corpus (where src_corpus
has sentences in above 'b' format)

On Sat, Nov 22, 2014 at 11:45 AM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:

> what does your source corpus file look like?
> (a) word1 word2 word3
> or
> (b) word1|word1-STEM|word1-POS word2|word2-STEM|word2-POS
> word3|word3-STEM|word3-POS
>
> the EMS expects (a), it then requires the scripts to create (b). If you
> already have (b), then let me know. You have to set some variable to bypass
> the scripts
>
>
> On 21 November 2014 13:08, Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
> wrote:
>
>> Hi Hieu Hoang,
>>
>> I am using EMS for factored based SMT.
>> I already have a factored corpus and I have used
>> factorized-stem=/path/to/corpus, but still I am getting this error "ERROR:
>> you need to define INPUT-FACTOR:lemma:factor-script"
>>
>> *Training Option:*
>> input-factors = word lemma pos
>> output-factors = word
>> alignment-factors = "lemma -> word"
>> translation-factors = "word -> word"
>> reordering-factors = "word -> word"
>> #generation-factors = "word -> pos"
>> decoding-steps = "t0"
>>
>> *Factor Definition:*
>> [INPUT-FACTOR]
>> temp-dir = $working-dir/training/factor
>>
>> #[OUTPUT-FACTOR]
>> #temp-dir = $working-dir/training/factor
>>
>> As I am already using factorized-stem then why it is asking for a
>> factor-script ?
>>
>>
>> --
>> -Regards,
>> Rajen Chatterjee.
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>


--
-Regards,
Rajen Chatterjee.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141122/be8b6828/attachment-0001.htm

------------------------------

Message: 2
Date: Sat, 22 Nov 2014 11:02:05 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] Error in EMS for factor based SMT
To: Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbhkEAQaf1HWFY+Rs=oA6JT-8=O=XeyQ-eTs9_pgj94mKA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

you may have to set this instead
[TRAINING]
corpus = /path/to/files
However, I'm not sure. If it doesn't work, you'll have to look at the error
carefully. Also, look at
scripts/ems/experiment.meta
to see what each EMS step expects as input and output


On 22 November 2014 10:51, Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
wrote:

> Hi Hieu,
>
> It is "b" word1|word1-STEM|word1-POS word2|word2-STEM|word2-POS
> word3|word3-STEM|word3-POS
> I have already set factorized-stem=/path/to/src_corpus (where src_corpus
> has sentences in above 'b' format)
>
> On Sat, Nov 22, 2014 at 11:45 AM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>
>> what does your source corpus file look like?
>> (a) word1 word2 word3
>> or
>> (b) word1|word1-STEM|word1-POS word2|word2-STEM|word2-POS
>> word3|word3-STEM|word3-POS
>>
>> the EMS expects (a), it then requires the scripts to create (b). If you
>> already have (b), then let me know. You have to set some variable to bypass
>> the scripts
>>
>>
>> On 21 November 2014 13:08, Rajen Chatterjee <rajen.k.chatterjee@gmail.com
>> > wrote:
>>
>>> Hi Hieu Hoang,
>>>
>>> I am using EMS for factored based SMT.
>>> I already have a factored corpus and I have used
>>> factorized-stem=/path/to/corpus, but still I am getting this error "ERROR:
>>> you need to define INPUT-FACTOR:lemma:factor-script"
>>>
>>> *Training Option:*
>>> input-factors = word lemma pos
>>> output-factors = word
>>> alignment-factors = "lemma -> word"
>>> translation-factors = "word -> word"
>>> reordering-factors = "word -> word"
>>> #generation-factors = "word -> pos"
>>> decoding-steps = "t0"
>>>
>>> *Factor Definition:*
>>> [INPUT-FACTOR]
>>> temp-dir = $working-dir/training/factor
>>>
>>> #[OUTPUT-FACTOR]
>>> #temp-dir = $working-dir/training/factor
>>>
>>> As I am already using factorized-stem then why it is asking for a
>>> factor-script ?
>>>
>>>
>>> --
>>> -Regards,
>>> Rajen Chatterjee.
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Hieu Hoang
>> Research Associate
>> University of Edinburgh
>> http://www.hoang.co.uk/hieu
>>
>>
>
>
> --
> -Regards,
> Rajen Chatterjee.
>



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141122/776eb857/attachment-0001.htm

------------------------------

Message: 3
Date: Sat, 22 Nov 2014 12:09:52 +0100
From: Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
Subject: Re: [Moses-support] Error in EMS for factor based SMT
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAC4-+Nwrh_rmA6DOVx5SjHeNxEAy5A1ObgaB13KFVfrQMuUptg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

ok, I will try to troubleshoot. If I get a solution will post it.
Thanks

On Sat, Nov 22, 2014 at 12:02 PM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:

> you may have to set this instead
> [TRAINING]
> corpus = /path/to/files
> However, I'm not sure. If it doesn't work, you'll have to look at the
> error carefully. Also, look at
> scripts/ems/experiment.meta
> to see what each EMS step expects as input and output
>
>
> On 22 November 2014 10:51, Rajen Chatterjee <rajen.k.chatterjee@gmail.com>
> wrote:
>
>> Hi Hieu,
>>
>> It is "b" word1|word1-STEM|word1-POS word2|word2-STEM|word2-POS
>> word3|word3-STEM|word3-POS
>> I have already set factorized-stem=/path/to/src_corpus (where src_corpus
>> has sentences in above 'b' format)
>>
>> On Sat, Nov 22, 2014 at 11:45 AM, Hieu Hoang <Hieu.Hoang@ed.ac.uk> wrote:
>>
>>> what does your source corpus file look like?
>>> (a) word1 word2 word3
>>> or
>>> (b) word1|word1-STEM|word1-POS word2|word2-STEM|word2-POS
>>> word3|word3-STEM|word3-POS
>>>
>>> the EMS expects (a), it then requires the scripts to create (b). If you
>>> already have (b), then let me know. You have to set some variable to bypass
>>> the scripts
>>>
>>>
>>> On 21 November 2014 13:08, Rajen Chatterjee <
>>> rajen.k.chatterjee@gmail.com> wrote:
>>>
>>>> Hi Hieu Hoang,
>>>>
>>>> I am using EMS for factored based SMT.
>>>> I already have a factored corpus and I have used
>>>> factorized-stem=/path/to/corpus, but still I am getting this error "ERROR:
>>>> you need to define INPUT-FACTOR:lemma:factor-script"
>>>>
>>>> *Training Option:*
>>>> input-factors = word lemma pos
>>>> output-factors = word
>>>> alignment-factors = "lemma -> word"
>>>> translation-factors = "word -> word"
>>>> reordering-factors = "word -> word"
>>>> #generation-factors = "word -> pos"
>>>> decoding-steps = "t0"
>>>>
>>>> *Factor Definition:*
>>>> [INPUT-FACTOR]
>>>> temp-dir = $working-dir/training/factor
>>>>
>>>> #[OUTPUT-FACTOR]
>>>> #temp-dir = $working-dir/training/factor
>>>>
>>>> As I am already using factorized-stem then why it is asking for a
>>>> factor-script ?
>>>>
>>>>
>>>> --
>>>> -Regards,
>>>> Rajen Chatterjee.
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Research Associate
>>> University of Edinburgh
>>> http://www.hoang.co.uk/hieu
>>>
>>>
>>
>>
>> --
>> -Regards,
>> Rajen Chatterjee.
>>
>
>
>
> --
> Hieu Hoang
> Research Associate
> University of Edinburgh
> http://www.hoang.co.uk/hieu
>
>


--
-Regards,
Rajen Chatterjee.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141122/8461ad4a/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 97, Issue 69
*********************************************

Tweet