Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: please help me (Hieu Hoang)
2. Re: New to Moses, Need Information regarding (Nadir Durrani)
----------------------------------------------------------------------
Message: 1
Date: Wed, 31 Jan 2018 10:46:34 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] please help me
To: seid mehamed <seidmda@gmail.com>, moses-support@mit.edu
Message-ID: <33a57296-c1f6-8720-6d86-38b617154b29@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
it runs fine for me. Check that you've spelt everything correctly in the
moses.ini file
?? ...../mosesserver --server-port 8080? -f phrase-model/moses.ini
Defined parameters (per moses.ini or switch):
??? config: phrase-model/moses.ini
??? feature: KENLM name=LM factor=0 order=3 num-features=1
path=lm/europarl.srilm.gz Distortion WordPenalty UnknownWordPenalty
PhraseDictionaryMemory input-factor=0 output-factor=0
path=phrase-model/phrase-table num-features=1 table-limit=10
??? input-factors: 0
??? mapping: T 0
??? n-best-list: nbest.txt 100
??? server:
??? server-port: 8080
??? weight: WordPenalty0= 0 LM= 1 Distortion0= 1
PhraseDictionaryMemory0= 1
line=KENLM name=LM factor=0 order=3 num-features=1 path=lm/europarl.srilm.gz
Loading the LM will be faster if you build a binary file.
Reading lm/europarl.srilm.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
**The ARPA file is missing <unk>.? Substituting log10 probability -100.000.
**************************************************************************************************
FeatureFunction: LM start: 0 end: 0
line=Distortion
FeatureFunction: Distortion0 start: 1 end: 1
line=WordPenalty
FeatureFunction: WordPenalty0 start: 2 end: 2
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 3 end: 3
line=PhraseDictionaryMemory input-factor=0 output-factor=0
path=phrase-model/phrase-table num-features=1 table-limit=10
FeatureFunction: PhraseDictionaryMemory0 start: 4 end: 4
Loading LM
Loading Distortion0
Loading WordPenalty0
Loading UnknownWordPenalty0
Loading PhraseDictionaryMemory0
Start loading text phrase table. Moses format : [0.473] seconds
Reading phrase-model/phrase-table
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
RUN SERVER at pid 0
[moses/server/Server.cpp:49] Listening on port 8080
On 31/01/18 05:11, seid mehamed wrote:
> --server-port 8080
--
Hieu Hoang
http://moses-smt.org/
------------------------------
Message: 2
Date: Wed, 31 Jan 2018 15:53:49 +0300
From: Nadir Durrani <nadir.durrani@nu.edu.pk>
Subject: Re: [Moses-support] New to Moses, Need Information regarding
To: moses-support@mit.edu
Message-ID:
<CAFDj2Q1CBApziuz9XYcS4PdeW_vTNnCqxW6cr=B3oh9_08d2Pg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
The command actually trains the whole pipeline including mining a
transliteration corpus from the parallel corpus. The language model for the
transliteration module is trained on characters (target-side of the mined
transliteration corpus). But as Anoop mentioned, if you have a
transliteration corpus, you can bypass the mining step and run the
remaining pipeline OR concatenate mined corpus with yours. The mined
transliteration corpus sometimes get useful pairs which are not exactly
transliterations, but they help. See the following related paper for details
https://www.aclweb.org/anthology/E/E14/E14-4029.pdf
Btw the `post-decoding-transliteration.pl` script uses regular language
model (trained on words) to select the best transliteration, given the
context.
Nadir
On Wed, Jan 31, 2018 at 1:37 PM, <moses-support-request@mit.edu> wrote:
> Send Moses-support mailing list submissions to
> moses-support@mit.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/moses-support
> or, via email, send a message with subject or body 'help' to
> moses-support-request@mit.edu
>
> You can reach the person managing the list at
> moses-support-owner@mit.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Moses-support digest..."
>
>
> Today's Topics:
>
> 1. New to Moses, Need Information regarding Transliteration
> (nikhil t.v.s)
> 2. Re: Improving Accuracy Level (Raj Dabre)
> 3. Re: New to Moses, Need Information regarding Transliteration
> (Anoop (?????))
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 31 Jan 2018 15:06:08 +0530
> From: "nikhil t.v.s" <t.v.s.nikhil123@gmail.com>
> Subject: [Moses-support] New to Moses, Need Information regarding
> Transliteration
> To: moses-support@mit.edu
> Message-ID:
> <CACxbnusgtLGjdApYLBE_vDN91wWLoE+ZU8mP18+
> c7O8cMjM44A@mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
> Have some questions regarding transliteration training using Moses.
> There is minimal documentation.
>
>
> ../mosesdecoder/scripts/Transliteration/train-transliteration-module.pl \
> --corpus-f <foreign text> --corpus-e <target text> \ --alignment <path to
> aligned text> \ --moses-src-dir <moses decoder path> --external-bin-dir
> <external tools> \ --input-extension <input extension>--output-extension
> <output-extension> \ --srilm-dir <sri lm binary path> --out-dir <path to
> generate output files>
>
>
> Is the lm trained over words or characters?
> A parallel corpus of source language word and destination language word
> will suffice?
> Should the corpus be character segmented for training?
>
>
> Regards,
> T. Venkata Sai Nikhil,
> International Institute of Information Technology -Hyderabad.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.mit.edu/mailman/private/moses-support/
> attachments/20180131/12b1f64c/attachment-0001.html
>
> ------------------------------
>
> Message: 2
> Date: Wed, 31 Jan 2018 18:56:23 +0900
> From: Raj Dabre <prajdabre@gmail.com>
> Subject: Re: [Moses-support] Improving Accuracy Level
> To: Emmanuel Dennis <emmanueldennisb@gmail.com>
> Cc: moses-support@mit.edu
> Message-ID:
> <CAB3gfjB_DbMC4PvmqpOE_YjVWC9qpO9ZUQJTWH9VxkVvPGo_Fw@
> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi,
>
> You might want to look up "N-best list re-scoring using neural features"
>
> On Mon, Jan 29, 2018 at 6:39 PM, Emmanuel Dennis <
> emmanueldennisb@gmail.com>
> wrote:
>
> > Hi!
> >
> > What are some of the deep learning/big data strategies that can be used
> to
> > improve the accuracy level of a developed statistical machine translation
> > system?
> >
> >
> > Your assistance will be highly appreciated.
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
>
>
> --
> Raj Dabre.
> Doctoral Student,
> Graduate School of Informatics,
> Kyoto University.
> CSE MTech, IITB., 2011-2014
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.mit.edu/mailman/private/moses-support/
> attachments/20180131/320677fe/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Wed, 31 Jan 2018 10:36:41 +0000
> From: Anoop (?????) <anoop.kunchukuttan@gmail.com>
> Subject: Re: [Moses-support] New to Moses, Need Information regarding
> Transliteration
> To: "nikhil t.v.s" <t.v.s.nikhil123@gmail.com>
> Cc: moses-support@mit.edu
> Message-ID:
> <CADXxMYfeCfakjYw0ftn1-t=8Z5ZAQscii6XLJY4oEm=kD8EQow@
> mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Nikhil,
>
> The command you have mentioned is to be used for transliteration mining
> from a parallel translation corpus. If you already have a parallel
> transliteration corpus, you can ignore this command and use the standard
> Moses pipeline to train the model. The parallel transliteration corpus
> should be character segmented, and you should train the LM over characters.
>
> Anoop.
>
> On Wed 31 Jan, 2018, 15:08 nikhil t.v.s, <t.v.s.nikhil123@gmail.com>
> wrote:
>
> > Hi,
> > Have some questions regarding transliteration training using Moses.
> > There is minimal documentation.
> >
> >
> > ../mosesdecoder/scripts/Transliteration/train-transliteration-module.pl
> \
> > --corpus-f <foreign text> --corpus-e <target text> \ --alignment <path to
> > aligned text> \ --moses-src-dir <moses decoder path> --external-bin-dir
> > <external tools> \ --input-extension <input extension>--output-extension
> > <output-extension> \ --srilm-dir <sri lm binary path> --out-dir <path to
> > generate output files>
> >
> >
> > Is the lm trained over words or characters?
> > A parallel corpus of source language word and destination language word
> > will suffice?
> > Should the corpus be character segmented for training?
> >
> >
> > Regards,
> > T. Venkata Sai Nikhil,
> > International Institute of Information Technology -Hyderabad.
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.mit.edu/mailman/private/moses-support/
> attachments/20180131/e907e98d/attachment.html
>
> ------------------------------
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> End of Moses-support Digest, Vol 135, Issue 34
> **********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20180131/5454b214/attachment-0001.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 135, Issue 35
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 135, Issue 35"
Post a Comment