Home » Uncategories » Moses-support Digest, Vol 127, Issue 25

Moses-support Digest, Vol 127, Issue 25

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: EMS fails with Transliteration option (Hieu Hoang)
2. Re: Segfault from moses while tuning (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 17 May 2017 11:43:50 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] EMS fails with Transliteration option
To: Rofael Behnam <rofaelemil@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbiOHbN5fnmbWys=-roZzi6iu9p6JhMOrVAuGO1c70Z68g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

If you are using the EMS, this should be in your EMS config file
[GENERAL]
....
external-bin-dir = path


* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 17 May 2017 at 11:39, Rofael Behnam <rofaelemil@gmail.com> wrote:

>
>
> Hello,
> I run EMS using experiment.perl, is this will be allowed?
>
> I appreciate your understanding.
> Thanks in advance.
>
> Best Regards,
> *Rofael Emil Fayez Behnam*
> Bachelor of Science in Computer and Systems Engineering,
> Faculty of Engineering,
> Alexandria University,
> Egypt.
>
> On 17 May 2017 at 12:57, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> delete the code you added.
>>
>> When running the script, add the argument
>> -external-bin-dir [path]
>> where [path] is the directory that contains the executables mkcls, GIZA++
>> or mgiza, and snt2cooc
>>
>> * Looking for MT/NLP opportunities *
>> Hieu Hoang
>> http://moses-smt.org/
>>
>>
>> On 17 May 2017 at 09:10, Rofael Behnam <rofaelemil@gmail.com> wrote:
>>
>>> Hello,
>>> Kindly find train-model.perl attached.
>>> I only added this line
>>> my $_EXTERNAL_BINDIR = '/home/rbehnam/mosesdecoder/external-bin-dir';
>>> at line 149 to try to solve the error during running EMS with
>>> transliteration option.
>>>
>>> I appreciate your understanding.
>>> Thanks in advance.
>>>
>>> Best Regards,
>>> *Rofael Emil Fayez Behnam*
>>> Bachelor of Science in Computer and Systems Engineering,
>>> Faculty of Engineering,
>>> Alexandria University,
>>> Egypt.
>>>
>>> On 17 May 2017 at 10:55, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>
>>>> what is the exact command you run to get the error? Can you please send
>>>> me the file
>>>> /home/rbehnam/mosesdecoder/scripts/training/train-model.perl
>>>>
>>>> * Looking for MT/NLP opportunities *
>>>> Hieu Hoang
>>>> http://moses-smt.org/
>>>>
>>>>
>>>> On 17 May 2017 at 08:45, Rofael Behnam <rofaelemil@gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>> I have downloaded it from https://github.com/moses-smt/mosesdecoder
>>>>> This month. I only use Moses, I didn't change anything in code.
>>>>>
>>>>> I appreciate your understanding.
>>>>> Thanks in advance.
>>>>>
>>>>> Best Regards,
>>>>> *Rofael Emil Fayez Behnam*
>>>>> Bachelor of Science in Computer and Systems Engineering,
>>>>> Faculty of Engineering,
>>>>> Alexandria University,
>>>>> Egypt.
>>>>>
>>>>> On 17 May 2017 at 00:59, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>>>
>>>>>> approximately when and where did you download the Moses code from? Or
>>>>>> have you changed the code?
>>>>>>
>>>>>> Line 149 in your script doesn't seem to correspond to line 149 in the
>>>>>> script on github
>>>>>>
>>>>>>
>>>>>> * Looking for MT/NLP opportunities *
>>>>>> Hieu Hoang
>>>>>> http://moses-smt.org/
>>>>>>
>>>>>>
>>>>>> On 16 May 2017 at 14:05, Rofael Behnam <rofaelemil@gmail.com> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>> Hope all is good.
>>>>>>> I got that error
>>>>>>> Error
>>>>>>> *****
>>>>>>> "my" variable $_EXTERNAL_BINDIR masks earlier declaration in same
>>>>>>> scope at /home/rbehnam/mosesdecoder/scripts/training/train-model.perl
>>>>>>> line 149.
>>>>>>> Using SCRIPTS_ROOTDIR: /home/rbehnam/mosesdecoder/scripts
>>>>>>> Using multi-thread GIZA
>>>>>>> using gzip
>>>>>>> Use of uninitialized value $GIZA in -x at
>>>>>>> /home/rbehnam/mosesdecoder/scripts/training/train-model.perl line
>>>>>>> 491.
>>>>>>> ERROR: Cannot find mkcls, GIZA++/mgiza, & snt2cooc.out/snt2cooc in
>>>>>>> /home/rbehnam/mosesdecoder/external-bin-dir.
>>>>>>> You MUST specify the parameter -external-bin-dir at
>>>>>>> /home/rbehnam/mosesdecoder/scripts/training/train-model.perl line
>>>>>>> 491.
>>>>>>> *****************************
>>>>>>> The files: mkcls, GIZA++, & snt2cooc.out, are in external-bin-dir
>>>>>>> and EMS works well but on adding transliteration option, this error occurs,
>>>>>>> although train-model.perl is also called with EMS without Transliteration
>>>>>>> option.
>>>>>>>
>>>>>>> Please, help me use transliteration option with EMS.
>>>>>>> Thanks in advance.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> *Rofael Emil Fayez Behnam*
>>>>>>> Bachelor of Science in Computer and Systems Engineering,
>>>>>>> Faculty of Engineering,
>>>>>>> Alexandria University,
>>>>>>> Egypt.
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Moses-support mailing list
>>>>>>> Moses-support@mit.edu
>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170517/82767e12/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 17 May 2017 11:53:03 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Segfault from moses while tuning
To: Per Starb?ck <starback@stp.lingfil.uu.se>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbincgremFcgcN1cUZAEMfJtAK_j3ZAn8DpKHmLURYN7tQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

thanks. It may not be the phrase-based/hiero mixup as I had suggested.

Maybe the system ran out of RAM.

Before you run tuning, you should also binarize the phrase-table and
lexicalised reordering model, this can be done in the filtering process by
providing the absolute path of the binarizer, eg
-Binarizer "/home/s0565741/workspace/github/hh/bin/processPhraseTableMin
This shows you an example

http://www.statmt.org/moses/RELEASE-3.0/models/de-en/steps/1/TUNING_filter.1

Binarizing reduces RAM usage by only loading translation rules for the
particular sentence it is translating



* Looking for MT/NLP opportunities *
Hieu Hoang
http://moses-smt.org/


On 17 May 2017 at 11:42, Per Starb?ck <starback@stp.lingfil.uu.se> wrote:

> Hieu Hoang writes:
> > It may be you're mixing up the phrase-based with the syntax/hiero model.
> Can you please
> > attach the file
> > filtered/moses.ini
> > and perhaps the first few lines of the phrase-table that's used
>
> Sure! Here is mert-work/filtered/moses.ini, again just with the
> beginning of absolute paths replaced with "...":
>
> ======================================================================
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
>
> [distortion-limit]
> 6
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryMemory name=TranslationModel0 num-features=4
> path=.../mert-work/filtered/phrase-table.0-0.1.1.gz input-factor=0
> output-factor=0
> LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=.../mert-work/filtered/reordering-table.wbe-msd-bidirectional-fe
> Distortion
> KENLM lazyken=0 name=LM0 factor=0 path=.../lm/news-commentrary-v8.de-en.blm.en
> order=3
>
> # dense weights for feature functions
> [weight]
> UnknownWordPenalty0= 1
> WordPenalty0= -1
> PhrasePenalty0= 0.2
> TranslationModel0= 0.2 0.2 0.2 0.2
> LexicalReordering0= 0.3 0.3 0.3 0.3 0.3 0.3
> Distortion0= 0.3
> LM0= 0.5
> ======================================================================
>
> And here the beginning of mert-work/filtered/phrase-table.0-0.1.1.gz
> uncompressed:
>
> ======================================================================
> $ ||| $100 they earn. ||| 0.25 0.0252101 0.025 1.54085e-10 ||| 0-0 ||| 4
> 40 1 ||| |||
> $ ||| $100 they ||| 0.25 0.0252101 0.025 2.56808e-05 ||| 0-0 ||| 4 40 1
> ||| |||
> $ ||| $100 ||| 0.00819672 0.0252101 0.025 0.0084507 ||| 0-0 ||| 122 40 1
> ||| |||
> $ ||| $100,000 apiece in campaign contributions in ||| 0.5 1 0.025
> 1.16584e-20 ||| 0-1 ||| 2 40 1 ||| |||
> $ ||| $100,000 apiece in campaign contributions ||| 0.5 1 0.025
> 6.46129e-19 ||| 0-1 ||| 2 40 1 ||| |||
> $ ||| $100,000 apiece in campaign ||| 0.5 1 0.025 1.54207e-14 ||| 0-1 |||
> 2 40 1 ||| |||
> ======================================================================
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170517/1343f64e/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 127, Issue 25
**********************************************

0 Response to "Moses-support Digest, Vol 127, Issue 25"

Post a Comment

Subscribe to: Post Comments (Atom)