Home » Uncategories » Moses-support Digest, Vol 126, Issue 37

Moses-support Digest, Vol 126, Issue 37

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Looking for a tool for training csv delimited and aligned
data (Allen Smith)
2. Unable to translate the in file became out file!
(Despina Mouratidi)
3. Re: Unable to translate the in file became out file! (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 26 Apr 2017 12:20:07 -0500
From: Allen Smith <allen.w.smith1@gmail.com>
Subject: Re: [Moses-support] Looking for a tool for training csv
delimited and aligned data
To: doc <raymond.doctor@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<CALiVeRGtfJ=faWoDuft2tOBb0PHjd8ydVjv9ky72dyw69m7t3A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

You are quite welcome. It looks like GIZApp isn't what you're looking for -
sequitur-g2p is a lot closer.

-Allen

On Tue, Apr 25, 2017 at 9:57 PM, doc <raymond.doctor@gmail.com> wrote:

> Many thanks for your kind interest.
> Basically I aim was to find if a training tool exists which can train data
> with the following format:
> abc=def
> Where the right hand is the input and the left hand is the output.
> I chose English to UK IPA as an example. I have around 80,000+ strings of
> English to IPA. An example is given below:
> ?em=?m
> ?neath=ni??
> ?shun=??n
> ?twas=tw?z
> ?twen=twi?n
> ?twen-decks=?twi?n-deks
> ?twere=tw??r
> ?twill=tw?l
> ?twixt=tw?kst
> ?twould=tw?d
> ?un=?n
> A=ei?
> A?s=e?z
> A-bomb=?e?-b?m
> If I had an unknown word like say
> superpose
> the tool after training should be able to predict
> su?p??p??z
> or something like that. Stress does not matter
> I wonder if someone in the MOSES team could come up with a small tool
> which can be used for such kind of work. As a linguist, I can assure you
> that such a tool would be immensely popular and render great service to the
> community.
> I need not add that I will download GIZApp and try. I am trying out
> SciKit, but the results are not very encouraging.
> Many thanks
>
>
>
> On Wed, Apr 26, 2017 at 3:23 AM, Allen Smith <allen.w.smith1@gmail.com>
> wrote:
>
>> Are you wanting to train something to align letters and sounds, or to
>> figure out the sounds given the letters? (As it happens, I've been working
>> on using GIZApp to do the former.)
>>
>> -Allen (Allen W. Smith, Ph.D.)
>>
>> On Tue, Apr 25, 2017 at 4:01 AM, doc <raymond.doctor@gmail.com> wrote:
>>
>>> Hello,
>>> I am looking for a tool for training data using either Statistical
>>> methods or even CNN/RNN.
>>> Basically the tool would allow the user to train simple data and then
>>> once trained, it could be deployed to predict unknown data
>>> As an example, I have around 80,000 words in English converted to IPA
>>> [text aligned in CSV format] and would like to train the tool using the
>>> data, to predict in the case of new words.
>>> Using Moses is like using a surgeon's scalpel to saw wood
>>> And since I work in a Windows environment installing Moses is not very
>>> easy
>>> In any case a large number of linguists like me would prefer to have a
>>> tool with ease of use.I am sure that if such a tool is made available, it
>>> will be one of the most popular tools.
>>>
>>> Thanks in advance for any help.
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170426/14a9469a/attachment-0001.html

------------------------------

Message: 2
Date: Thu, 27 Apr 2017 12:41:10 +0300
From: Despina Mouratidi <dmouratidi@gmail.com>
Subject: [Moses-support] Unable to translate the in file became out
file!
To: Moses-support@mit.edu
Message-ID:
<CAJJG54=naXrEfk+WJhNtR33A3+bzftWYCfktp11vZ6ZTBs2vZg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello guys...

I am trying to translate with moses but when I am typing
./moses -f ~/Desktop/moses/train_ken/model/moses.ini < ~/Desktop/in >
~/Desktop/out in my directory the out file became the in file.

Thus,
Defined parameters (per moses.ini or switch):
config: /home/despina/Desktop/moses/train_ken/model/moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/despina/Desktop/moses/train_ken/model/phrase-table
LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/despina/Desktop/moses/train_ken/model/reordering Distortion
IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/
corpus.en-fr.blm.fr order=3
input-factors: 0
mapping: 0 T 0
weight: LexicalReordering0= 0.0693229 0.0693229 0.0693229 0.0693229
0.0693229 0.0693229 Distortion0= 0.00636699 LM0= 0.115539 WordPenalty0=
-0.231077 PhrasePenalty0= 0.046216 TranslationModel0= 0.046216 0.046216
0.046216 0.046216 UnknownWordPenalty0= 1
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
path=/home/despina/Desktop/moses/train_ken/model/phrase-table
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/despina/Desktop/moses/train_ken/model/reordering
Initializing Lexical Reordering Feature..
FeatureFunction: LexicalReordering0 start: 7 end: 12
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/
corpus.en-fr.blm.fr order=3
FeatureFunction: LM0 start: 14 end: 14
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Loading Distortion0
Loading LM0
Language Model Type of /home/despina/Desktop/moses/lm2/corpus.en-fr.blm.fr
is 1
Language Model Type is 1
starting to use OOV words [<unk>]
OOV code is 0
OOV code is 0
LM0 LanguageModelIRST::Load() m_unknownId=0
Loading TranslationModel0
Created input-output object : [0.172] seconds
Translating: hello
Line 0: Initialize search took 0.000 seconds total
Line 0: Collecting options took 0.000 seconds at moses/Manager.cpp Line 141
Line 0: Search took 0.000 seconds
BEST TRANSLATION: hello|UNK|UNK|UNK [1] [total=-105.310]
core=(-100.000,-1.000,1.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-48.354)

Line 0: Decision rule took 0.000 seconds total
Line 0: Additional reporting took 0.000 seconds total
Line 0: Translation took 0.000 seconds total
reset caches
reset mmap
Name:moses VmPeak:565984 kB VmRSS:49036 kB RSSMax:370304 kB
user:0.136 sys:0.104 CPU:0.240 real:0.238


Thanks in advance
Despina
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170427/f02f7c9f/attachment-0001.html

------------------------------

Message: 3
Date: Thu, 27 Apr 2017 12:09:56 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Unable to translate the in file became
out file!
To: Despina Mouratidi <dmouratidi@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhMrU1OUbTLM0-MDsCkpZSfHERLuLgKgQX2kf0R2nB3gA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

it looks like the words in your input is are not in your phrase-table. By
default they will just be copied to the output as-is.



* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 27 April 2017 at 10:41, Despina Mouratidi <dmouratidi@gmail.com> wrote:

> Hello guys...
>
> I am trying to translate with moses but when I am typing
> ./moses -f ~/Desktop/moses/train_ken/model/moses.ini < ~/Desktop/in >
> ~/Desktop/out in my directory the out file became the in file.
>
> Thus,
> Defined parameters (per moses.ini or switch):
> config: /home/despina/Desktop/moses/train_ken/model/moses.ini
> distortion-limit: 6
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/despina/Desktop/moses/train_ken/model/phrase-table
> LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/despina/Desktop/moses/train_ken/model/reordering Distortion
> IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/
> corpus.en-fr.blm.fr order=3
> input-factors: 0
> mapping: 0 T 0
> weight: LexicalReordering0= 0.0693229 0.0693229 0.0693229 0.0693229
> 0.0693229 0.0693229 Distortion0= 0.00636699 LM0= 0.115539 WordPenalty0=
> -0.231077 PhrasePenalty0= 0.046216 TranslationModel0= 0.046216 0.046216
> 0.046216 0.046216 UnknownWordPenalty0= 1
> line=UnknownWordPenalty
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
> line=WordPenalty
> FeatureFunction: WordPenalty0 start: 1 end: 1
> line=PhrasePenalty
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
> line=PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/despina/Desktop/moses/train_ken/model/phrase-table
> FeatureFunction: TranslationModel0 start: 3 end: 6
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/despina/Desktop/moses/train_ken/model/reordering
> Initializing Lexical Reordering Feature..
> FeatureFunction: LexicalReordering0 start: 7 end: 12
> line=Distortion
> FeatureFunction: Distortion0 start: 13 end: 13
> line=IRSTLM name=LM0 factor=0 path=/home/despina/Desktop/moses/lm2/
> corpus.en-fr.blm.fr order=3
> FeatureFunction: LM0 start: 14 end: 14
> Loading UnknownWordPenalty0
> Loading WordPenalty0
> Loading PhrasePenalty0
> Loading LexicalReordering0
> Loading Distortion0
> Loading LM0
> Language Model Type of /home/despina/Desktop/moses/lm2/corpus.en-fr.blm.fr
> is 1
> Language Model Type is 1
> starting to use OOV words [<unk>]
> OOV code is 0
> OOV code is 0
> LM0 LanguageModelIRST::Load() m_unknownId=0
> Loading TranslationModel0
> Created input-output object : [0.172] seconds
> Translating: hello
> Line 0: Initialize search took 0.000 seconds total
> Line 0: Collecting options took 0.000 seconds at moses/Manager.cpp Line 141
> Line 0: Search took 0.000 seconds
> BEST TRANSLATION: hello|UNK|UNK|UNK [1] [total=-105.310]
> core=(-100.000,-1.000,1.000,0.000,0.000,0.000,0.000,0.000,0.
> 000,0.000,0.000,0.000,0.000,0.000,-48.354)
> Line 0: Decision rule took 0.000 seconds total
> Line 0: Additional reporting took 0.000 seconds total
> Line 0: Translation took 0.000 seconds total
> reset caches
> reset mmap
> Name:moses VmPeak:565984 kB VmRSS:49036 kB RSSMax:370304 kB
> user:0.136 sys:0.104 CPU:0.240 real:0.238
>
>
> Thanks in advance
> Despina
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170427/6984413c/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 126, Issue 37
**********************************************

Related Posts :

0 Response to "Moses-support Digest, Vol 126, Issue 37"

Post a Comment

Subscribe to: Post Comments (Atom)