Moses-support Digest, Vol 121, Issue 6

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: error in mosesdecoder tuning (Hasan Sait ARSLAN)
2. Re: error in mosesdecoder tuning (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Wed, 2 Nov 2016 12:45:26 +0200
From: Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
Subject: Re: [Moses-support] error in mosesdecoder tuning
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CAJj7HD01OBo1j-1A9uvDLnr6zGa0xcc-bavjUq+fPvn=PCQbuQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Now currently I have a bigger problem. I have trained my data for 5 days,
and the folder "train" is 39 G, but there is no any phrases saved on phrase
table. It is annoying. What should I do now? I hope I won't need to rerun
everything from the scratch

2016-11-02 12:26 GMT+02:00 Barry Haddow <bhaddow@staffmail.ed.ac.uk>:

> Hi Hasan
>
> The error message should be written into filterphrases.err, inside your
> working directory,
>
> cheers - Barry
>
>
> On 02/11/16 10:02, Hasan Sait ARSLAN wrote:
>
> Hi Hieu,
>
> I did in the way, you want. Plus, I am sure the path and file names are
> correctly spelt.
> But still, I get the same error.
>
> 2016-11-01 21:41 GMT+02:00 Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
> :
>
>> Thank you Hieu, I will do it, and give you feedback about it.
>>
>> Kind Regards,
>>
>> 2016-11-01 21:37 GMT+02:00 Hieu Hoang <hieuhoang@gmail.com>:
>>
>>> the command you execute looks ok. the problem is likely to be with your
>>> using tuning data.
>>>
>>> You should look at your tuning data, make sure you've spelt the path and
>>> filename correctly, use a very small subset (eg. start with 1 parallel
>>> sentence) and increase the number of sentences until you find the problem
>>>
>>>
>>>
>>> On 01/11/2016 12:30, Hasan Sait ARSLAN wrote:
>>>
>>> Hello,
>>>
>>> I have trained dataset for 5 days, and for 2-3 days I am dealing with
>>> the bug in tuning, but still couldn't solve the problem.
>>>
>>> I use the following command to run:
>>>
>>>
>>>
>>> */home/sait/mosesdecoder/scripts/training/mert-moses.pl
>>> <http://mert-moses.pl> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>> <http://dev.clean.tr> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.en
>>> /home/sait/mosesdecoder/bin/moses
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini --working-dir
>>> /home/sait/Kairit/Task3/working_tr-en/mert --decoder-flags "--threads 32" *
>>> Then I get the following error:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *Using SCRIPTS_ROOTDIR: /home/sait/mosesdecoder/scripts Assuming
>>> --mertdir=/home/sait/mosesdecoder/bin filtering the phrase tables... T
>>> nov 1 19:17:39 EET 2016 exec:
>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>> <http://filter-model-given-input.pl> ./filtered
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr <http://dev.clean.tr>
>>> Executing:
>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>> <http://filter-model-given-input.pl> ./filtered
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr <http://dev.clean.tr> >
>>> filterphrases.out 2> filterphrases.err Exit code: 255 ERROR: Failed to run
>>> '/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>> <http://filter-model-given-input.pl> ./filtered
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr <http://dev.clean.tr>'.
>>> at /home/sait/mosesdecoder/scripts/training/mert-moses.pl
>>> <http://mert-moses.pl> line 1748. *
>>> I have searched over the internet about the error, but unfortunately I
>>> couldn't find any beneficial solution.
>>>
>>> It is so annoying, and eats my time, please help me to solve my problem.
>>>
>>> Thanks,
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161102/8c2db101/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 02 Nov 2016 10:55:38 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] error in mosesdecoder tuning
To: Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <5819C62A.30302@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"

Hi Hasan

If your phrase table is empty, that would explain why tuning didn't
work. Something went wrong earlier in the process. Could you post your
log file from train_model.perl,

cheers - Barry

On 02/11/16 10:45, Hasan Sait ARSLAN wrote:
> Now currently I have a bigger problem. I have trained my data for 5
> days, and the folder "train" is 39 G, but there is no any phrases
> saved on phrase table. It is annoying. What should I do now? I hope I
> won't need to rerun everything from the scratch
>
> 2016-11-02 12:26 GMT+02:00 Barry Haddow <bhaddow@staffmail.ed.ac.uk
> <mailto:bhaddow@staffmail.ed.ac.uk>>:
>
> Hi Hasan
>
> The error message should be written into filterphrases.err, inside
> your working directory,
>
> cheers - Barry
>
>
> On 02/11/16 10:02, Hasan Sait ARSLAN wrote:
>> Hi Hieu,
>>
>> I did in the way, you want. Plus, I am sure the path and file
>> names are correctly spelt.
>> But still, I get the same error.
>>
>> 2016-11-01 21:41 GMT+02:00 Hasan Sait ARSLAN
>> <hasan.sait.arslan@gmail.com <mailto:hasan.sait.arslan@gmail.com>>:
>>
>> Thank you Hieu, I will do it, and give you feedback about it.
>>
>> Kind Regards,
>>
>> 2016-11-01 21:37 GMT+02:00 Hieu Hoang <hieuhoang@gmail.com
>> <mailto:hieuhoang@gmail.com>>:
>>
>> the command you execute looks ok. the problem is likely
>> to be with your using tuning data.
>>
>> You should look at your tuning data, make sure you've
>> spelt the path and filename correctly, use a very small
>> subset (eg. start with 1 parallel sentence) and increase
>> the number of sentences until you find the problem
>>
>>
>>
>> On 01/11/2016 12:30, Hasan Sait ARSLAN wrote:
>>> Hello,
>>>
>>> I have trained dataset for 5 days, and for 2-3 days I am
>>> dealing with the bug in tuning, but still couldn't solve
>>> the problem.
>>>
>>> I use the following command to run:
>>>
>>> */home/sait/mosesdecoder/scripts/training/mert-moses.pl
>>> <http://mert-moses.pl>
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>> <http://dev.clean.tr>
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.en
>>> /home/sait/mosesdecoder/bin/moses
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> --working-dir /home/sait/Kairit/Task3/working_tr-en/mert
>>> --decoder-flags "--threads 32"
>>>
>>> *
>>> Then I get the following error:
>>>
>>> *Using SCRIPTS_ROOTDIR: /home/sait/mosesdecoder/scripts
>>> Assuming --mertdir=/home/sait/mosesdecoder/bin
>>> filtering the phrase tables... T nov 1 19:17:39 EET 2016
>>> exec:
>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>> <http://filter-model-given-input.pl> ./filtered
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>> <http://dev.clean.tr>
>>> Executing:
>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>> <http://filter-model-given-input.pl> ./filtered
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>> <http://dev.clean.tr> > filterphrases.out 2>
>>> filterphrases.err
>>> Exit code: 255
>>> ERROR: Failed to run
>>> '/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>> <http://filter-model-given-input.pl> ./filtered
>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>> <http://dev.clean.tr>'. at
>>> /home/sait/mosesdecoder/scripts/training/mert-moses.pl
>>> <http://mert-moses.pl> line 1748.
>>>
>>> *
>>> I have searched over the internet about the error, but
>>> unfortunately I couldn't find any beneficial solution.
>>>
>>> It is so annoying, and eats my time, please help me to
>>> solve my problem.
>>>
>>> Thanks,
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161102/ab617d45/attachment.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161102/ab617d45/attachment.pl

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 121, Issue 6
*********************************************

Tweet