Moses-support Digest, Vol 127, Issue 2

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. CreateProbingPT2 exception (Mike Ladwig)
2. Re: (no subject) (G/her G/libanos)


----------------------------------------------------------------------

Message: 1
Date: Tue, 2 May 2017 01:06:27 -0400
From: Mike Ladwig <mdladwig@gmail.com>
Subject: [Moses-support] CreateProbingPT2 exception
To: Moses Support <moses-support@mit.edu>
Message-ID:
<CAB3VaD2xOAiDh6aVByFMZn4VEnok+xbfomWWbKRpz4jjfAkrfw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Got an exception creating a PT2 with yesterday's master:

Binarize phrase and reordering model in probing table format:
/home/mike/stelae5/mosesdecoder/scripts/generic/binarize4moses2.perl
--phrase-table=/home/mike/stelae-projects/de-en/phrasemodel/model/phrase-table.gz
--lex-ro=/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz
--output-dir=/home/mike/stelae-projects/de-en/phrasemodel/PT2
--num-lex-scores=6
Executing: gzip -dc
/home/mike/stelae-projects/de-en/phrasemodel/model/phrase-table.gz |
/home/mike/stelae5/mosesdecoder/scripts/generic/../../contrib/sigtest-filter/filter-pt
-n 0 | gzip -c >
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.gz
sh:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../contrib/sigtest-filter/filter-pt:
No such file or directory
Executing:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/processLexicalTableMin
-in
/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz
-out /home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro -T .
-threads all
Used options:
Text reordering table will be read from:
/home/mike/stelae-projects/de-en/phrasemodel/model/reordering-table.wbe-msd-bidirectional-fe.gz
Output reordering table will be written to:
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr
Step size for source landmark phrases: 2^10=1024
Phrase fingerprint size: 16 bits / P(fp)=1.52588e-05
Single Huffman code set for score components: no
Using score quantization: no
Running with 24 threads

Pass 1/2: Creating phrase index + Counting scores
..................................................[5000000]
..................................................[10000000]
..................................................[15000000]
..................................................[20000000]
..................................................[25000000]
..................................................[30000000]
..................................................[35000000]
..................................................[40000000]
..................................................[45000000]
..................................................[50000000]
..................................................[55000000]
..................................................[60000000]
..................................................[65000000]
..................................................[70000000]
..................................................[75000000]
....................

Intermezzo: Calculating Huffman code sets
Creating Huffman codes for 32003 scores
Creating Huffman codes for 16732 scores
Creating Huffman codes for 31335 scores
Creating Huffman codes for 32076 scores
Creating Huffman codes for 15096 scores
Creating Huffman codes for 31659 scores

Pass 2/2: Compressing scores
..................................................[5000000]
..................................................[10000000]
..................................................[15000000]
..................................................[20000000]
..................................................[25000000]
..................................................[30000000]
..................................................[35000000]
..................................................[40000000]
..................................................[45000000]
..................................................[50000000]
..................................................[55000000]
..................................................[60000000]
..................................................[65000000]
..................................................[70000000]
..................................................[75000000]
....................

Saving to
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr
Done
Executing:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/addLexROtoPT
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.gz
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/lex-ro.minlexr |
gzip -c >
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.withLexRO.gz
Executing: ln -s pt.withLexRO.gz
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz
Executing:
/home/mike/stelae5/mosesdecoder/scripts/generic/../../bin/CreateProbingPT2
--num-scores 4 --log-prob --input-pt
/home/mike/stelae-projects/de-en/phrasemodel/tmp.15419/pt.txt.gz
--output-dir /home/mike/stelae-projects/de-en/phrasemodel/PT2
--num-lex-scores 6
Starting...
Reading phrase table finished, writing remaining files to disk.
terminate called after throwing an instance of 'util::ProbingSizeException'
what(): ./util/probing_hash_table.hh:150 in
util::ProbingHashTable<EntryT, HashT, EqualT, ModT>::Entry*
util::ProbingHashTable<EntryT, HashT, EqualT, ModT>::Insert(const T&) [with
T = probingpt::Entry; EntryT = probingpt::Entry; HashT = boost::hash<long
unsigned int>; EqualT = std::equal_to<long unsigned int>; ModT =
util::DivMod; util::ProbingHashTable<EntryT, HashT, EqualT,
ModT>::MutableIterator = probingpt::Entry*; util::ProbingHashTable<EntryT,
HashT, EqualT, ModT>::Entry = probingpt::Entry] threw ProbingSizeException
because `++entries_ >= buckets_'.
Hash table with 1 buckets is full.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170502/8e4703b0/attachment-0001.html

------------------------------

Message: 2
Date: Tue, 2 May 2017 00:58:39 -0700
From: "G/her G/libanos" <gerizaba@gmail.com>
Subject: Re: [Moses-support] (no subject)
To: Mathias M?ller <mathias.mueller@uzh.ch>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CALR6RoS1+qamvepPGDXm+oLn+aL3LU8yF-FJc2DQMgoJG20_tQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

hello there



I am doing well and your contribution is high



my adviser asks me to align the training system using phrase rather than
word based using Giza++



what package can I use to do phrase based alignment



thanks...

On Thu, Apr 13, 2017 at 1:37 AM, G/her G/libanos <gerizaba@gmail.com> wrote:

> hello there ....
>
> how is every thing
>
>
>
> I want to uses the m4loc to split the corpus into train, testing and
> tuning randomly
>
> but I didn't get how to apply it.
>
>
>
> the idea get from this paper
>
> Phoneme-based English-Amharic Statistical Machine Translation
>
>
> thanks..
>
>
>
> On Mon, Mar 27, 2017 at 3:42 AM, G/her G/libanos <gerizaba@gmail.com>
> wrote:
>
>> hello there...
>>
>> I want to ask you some thing
>>
>> 1. in language modeling for the monolingual data using IRSLM what value n
>> of the N-gram is more value?
>>
>> 2. in the training of the parallel corpus how I can isolate the testing
>> and tuning data randomly from the total corpus, since choosing myself is
>> not recommended?
>>
>>
>> thanks....
>>
>>
>> G/her from bahirdar university
>>
>> On Sat, Mar 11, 2017 at 1:06 AM, G/her G/libanos <gerizaba@gmail.com>
>> wrote:
>>
>>> hello there I would like thanks your contribution in my work...
>>> now I have one problem to create web-based translation system
>>> when I check using the Netcat by following the manual in the
>>> https://github.com/moses-smt/mosesdecoder/tree/master/contri
>>> b/iSenWeb/Introduction
>>> it display these result and the web site doesn't work
>>>
>>> thanks...
>>>
>>> On Sat, Feb 25, 2017 at 1:09 PM, Mathias M?ller <mathias.mueller@uzh.ch>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Did you perhaps forget the leading "/" in the file path to an existing
>>>> installation of XML-RPC?
>>>>
>>>> Regards
>>>> Mathias
>>>>
>>>>
>>>>
>>>> On Sat, Feb 25, 2017 at 10:46 AM, G/her G/libanos <gerizaba@gmail.com>
>>>> wrote:
>>>>
>>>>> hello there I am doing on the Moses server but on installing the
>>>>> xmlrpc create an error
>>>>> what I can do
>>>>>
>>>>> thanks for every thing you doing all
>>>>>
>>>>> On Wed, Feb 8, 2017 at 3:07 AM, Marwa Refaie <basmallah@hotmail.com>
>>>>> wrote:
>>>>>
>>>>>> For web server dig little on those links I think could help
>>>>>>
>>>>>> http://www.statmt.org/moses/?n=Moses.WebTranslation
>>>>>>
>>>>>> https://github.com/moses-smt/mosesdecoder/tree/master/contri
>>>>>> b/iSenWeb/Introduction
>>>>>>
>>>>>> *Marwa N Refaie*
>>>>>> On 7 Feb 2017, at 21:05, G/her G/libanos <gerizaba@gmail.com> wrote:
>>>>>>>
>>>>>>> the Moses decoder works for our system on Ubuntu using terminal
>>>>>>> but we want to make user interactive whether in the form of web page
>>>>>>> or application based
>>>>>>> could you help me any information to done that
>>>>>>> 10qu...
>>>>>>>
>>>>>>> On Mon, Dec 12, 2016 at 8:43 AM, G/her G/libanos <gerizaba@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> we create the translation model and we need to uses the window
>>>>>>>> application using
>>>>>>>> http://www.statmt.org/moses/?n=Moses.Packages
>>>>>>>> but when we import our model into the mainWindow application it
>>>>>>>> create an error
>>>>>>>> what we can do.
>>>>>>>> since it works for EN into Fr
>>>>>>>> 10qu for ...
>>>>>>>>
>>>>>>>> On Mon, Dec 12, 2016 at 7:17 PM, G/her G/libanos <
>>>>>>>> gerizaba@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> hello there...
>>>>>>>>> First of all I would like to thanks for your response for the
>>>>>>>>> previous comments on the question I rise
>>>>>>>>>
>>>>>>>>> next my system trained by 1000 parallel sentence and I try to
>>>>>>>>> calculate the Bleu score of the translation system using the vedio on TAUS
>>>>>>>>> and I got this one
>>>>>>>>>
>>>>>>>>> On Fri, Dec 9, 2016 at 5:17 PM, G/her G/libanos <
>>>>>>>>> gerizaba@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> hello there....
>>>>>>>>>> 1. we done our work using baseline and the system work for the
>>>>>>>>>> train data
>>>>>>>>>> but we need to localized the source code in the tokinization to
>>>>>>>>>> leave abrivated word ?/? as in english Adm. as Notbreaking_prefix.en that
>>>>>>>>>> is use the dot(.)
>>>>>>>>>>
>>>>>>>>>> 2. where we to change the code in c++ or in the perl part of the
>>>>>>>>>> codes
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 3. we need to uses the translation system in window, since the
>>>>>>>>>> end users are not expert in ubuntu and we see the how to change the window
>>>>>>>>>> see the Moses GUI and which part of the our file will be load to the model
>>>>>>>>>> only or all the train including the train data and can we modify the GUI of
>>>>>>>>>> the Moses.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 10qs for every thing
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> .......education is door of one's life.......
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> .......education is door of one's life.......
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> .......education is door of one's life.......
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> .......education is door of one's life.......
>>>
>>
>>
>>
>> --
>>
>> .......education is door of one's life.......
>>
>
>
>
> --
>
> .......education is door of one's life.......
>



--

.......education is door of one's life.......
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170502/ded46bbc/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 127, Issue 2
*********************************************

Tweet