Moses-support Digest, Vol 115, Issue 10

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Data for building a factored model (Sa?o Kuntaric)
2. Permission denied mgiza (Sanjanashree Palanivel)
3. mgiza installation (Sanjanashree Palanivel)
4. transliteration (Sanjanashree Palanivel)


----------------------------------------------------------------------

Message: 1
Date: Fri, 6 May 2016 20:23:07 +0200
From: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Subject: Re: [Moses-support] Data for building a factored model
To: Philipp Koehn <phi@jhu.edu>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CANsquDpn6gxE4PDO-tZ5-MLwrx1OWmF9xa3yg8k8UwKKPtCo8A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

Thank you Philipp for all the useful info, I will take a closer look at the
mentioned scripts.

I do have one follow-up question. Like I said, I really enjoyed working
with the factored corpora in the example. How were those created? Is there
a tool I can use to create similar ones?

Best regards,

Sa?o

2016-05-06 0:08 GMT+02:00 Philipp Koehn <phi@jhu.edu>:

> Hi,
>
> life is easier with factored models, if you use the experiment.perl set-up,
> where you just have to specify the factor set-up and scripts that generate
> factors.
>
> These scripts take the tokenized text and replace each word with a factor
> (e.g., replace each word with the POS tag).
>
> The POS LM is trained on such a corpus - each word is replaced by a
> POS tag, and then the standard LM training process is run over it.
>
> See $MOSES/scripts/ems/example/config.factored for an example.
>
> -phi
>
> On Wed, May 4, 2016 at 3:30 PM, Sa?o Kuntaric <saso.kuntaric@gmail.com>
> wrote:
> > Hello again,
> >
> > I believe I can wrap my head around the theoretical part, but the English
> > and German corpora in the Moses factored model tutorial
> > (http://www.statmt.org/moses/?n=Moses.FactoredTutorial) look beautifully
> > factored, so my question is how were the original corpora processed? Was
> a
> > specific tagger used and was there any manual/script postprocessing done?
> >
> > And since I am already bugging everyone, how is the language model pos.lm
> > created? Is it extracted from a file, created manually or in another way?
> >
> > Thank you in advance for all the replies.
> >
> > Best regards,
> >
> > Sa?o
> >
> > 2016-05-02 19:45 GMT+02:00 Marwa Refaie <basmallah@hotmail.com>:
> >>
> >> Corpus for translation model should be on 2 parallel files in the format
> >> Word | pos | Lema .... For example , by a file for each language. You
> can
> >> prepare files using word net , Stanford , or any tagger & stemmer as
> can
> >> deal with your language pairs. May be before enter the files to moses
> you
> >> should adjust the text files by a python script (write it your self)
> >>
> >> For language model ... You must build it as follows
> >> Verb noun noun
> >> Noun Det adj
> >> ....... Depending on the target language only ,, Then build it as usual
> >> n-gram lm.
> >>
> >> Sent from my iPad
> >>
> >> > On May 2, 2016, at 10:11, Sa?o Kuntaric <saso.kuntaric@gmail.com>
> wrote:
> >> >
> >> > Hi all,
> >> >
> >> > I am having some issues producing the corpora in the correct format
> for
> >> > Moses to execute factored training.
> >> >
> >> > I am looking at the factored tutorial on the Moses website and I am
> >> > wondering, how to get such consistent corpora for two languages. What
> tools
> >> > are being used and can they be trained for specific languages
> (Slovenian in
> >> > my example). Are such tools available for download or is such data
> produced
> >> > with custom scripts?
> >> >
> >> > --
> >> > Best regards,
> >> >
> >> > Sa?o
> >> > _______________________________________________
> >> > Moses-support mailing list
> >> > Moses-support@mit.edu
> >> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > --
> > lp,
> >
> > Sa?o
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
>



--
lp,

Sa?o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160506/ed3a4a71/attachment-0001.html

------------------------------

Message: 2
Date: Sat, 7 May 2016 10:25:04 +0530
From: Sanjanashree Palanivel <sanjanashree@gmail.com>
Subject: [Moses-support] Permission denied mgiza
To: moses-support@mit.edu
Message-ID:
<CAAc_kp4ehwMaAR1PJY+6AupLSus5cAmN+cEC0WohaavwQK9nVQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

why do i get permission denied when i try to run mgiza in
train-model.perl moses. what i



--
Thanks and regards,

Sanjanasri J.P
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160507/4127b088/attachment-0001.html

------------------------------

Message: 3
Date: Sat, 7 May 2016 11:27:44 +0530
From: Sanjanashree Palanivel <sanjanashree@gmail.com>
Subject: [Moses-support] mgiza installation
To: moses-support@mit.edu
Message-ID:
<CAAc_kp6EGgM2etn84Uf-+jZqdHj3EzUfbafCzFCpgdysNyzr6A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

Please forget my previous post, i get permission denied when i try to run
mgiza in train-model.perl moses, so i tried installing again with "cmake
.". I get the following . I do not understand that

sanjana@sanjana-HP-ProDesk-400-G2-MT:~/Documents/SMT/mgiza/mgizapp$ cmake .
> -- The C compiler identification is GNU 4.8.4
> -- The CXX compiler identification is GNU 4.8.4
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- You have not set the install dir, default to './inst', if
> you want to set it, use cmake -DCMAKE_INSTALL_PREFIX to do so
>
> --
Thanks and regards,

Sanjanasri J.P
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160507/f1b4b61a/attachment-0001.html

------------------------------

Message: 4
Date: Sat, 7 May 2016 11:54:07 +0530
From: Sanjanashree Palanivel <sanjanashree@gmail.com>
Subject: [Moses-support] transliteration
To: moses-support@mit.edu
Message-ID:
<CAAc_kp79mzcMyjmSHOnPnCCNC_sy-a9tKOnzm4W3EK3TRkGwRg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,


I tried training models using mgiza and it working properly . But, when
i do it with transliteration, I am getting the following error, Please help

Executing:
/home/sanjana/Documents/SMT/mosesdecoder/tools/m_giza_tools/mkcls -c50 -n2
-p/home/sanjana/Documents/SMT/Transliterationsam/training/corpus.en
-V/home/sanjana/Documents/SMT/Transliterationsam/training/prepared/en.vcb.classes
opt
ERROR: Execution of:
/home/sanjana/Documents/SMT/mosesdecoder/tools/m_giza_tools/mkcls -c50 -n2
-p/home/sanjana/Documents/SMT/Transliterationsam/training/corpus.en
-V/home/sanjana/Documents/SMT/Transliterationsam/training/prepared/en.vcb.classes
opt
died with signal 11, with coredump

--
Thanks and regards,

Sanjanasri J.P
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160507/2a51f84b/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 115, Issue 10
**********************************************

Tweet