Moses-support Digest, Vol 121, Issue 10

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: error in mosesdecoder tuning (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Wed, 02 Nov 2016 11:27:18 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] error in mosesdecoder tuning
To: Hasan Sait ARSLAN <hasan.sait.arslan@gmail.com>
Cc: moses-support@mit.edu
Message-ID: <5819CD96.5090507@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="utf-8"

Hi Hasan

It's not really possible to debug without a log file, so I think that
unfortunately you should start again.

I suggest you start with a much smaller corpus to try to debug your
setup. If you use 100,000 sentences it should train within hours.

cheers - Barry

On 02/11/16 11:13, Hasan Sait ARSLAN wrote:
> Hi Barry,
>
> Unfortunately I didn't keep the log file. Is it really a hopeless
> situation?
>
> 2016-11-02 13:10 GMT+02:00 Barry Haddow <bhaddow@staffmail.ed.ac.uk
> <mailto:bhaddow@staffmail.ed.ac.uk>>:
>
> Hi Hasan
>
> You should have run train_model.perl something like this:
>
> nohup nice /path/to/train_model.perl ARGS_FOR_TRAINING &> log &
>
> You just need the log file
>
> cheers - Barry
>
>
> On 02/11/16 11:00, Hasan Sait ARSLAN wrote:
>> Hi Barry,
>>
>> I don't have any idea about where it is saved.
>>
>> Cheers,
>>
>> 2016-11-02 12:55 GMT+02:00 Barry Haddow
>> <bhaddow@staffmail.ed.ac.uk <mailto:bhaddow@staffmail.ed.ac.uk>>:
>>
>> Hi Hasan
>>
>> If your phrase table is empty, that would explain why tuning
>> didn't work. Something went wrong earlier in the process.
>> Could you post your log file from train_model.perl,
>>
>> cheers - Barry
>>
>>
>> On 02/11/16 10:45, Hasan Sait ARSLAN wrote:
>>> Now currently I have a bigger problem. I have trained my
>>> data for 5 days, and the folder "train" is 39 G, but there
>>> is no any phrases saved on phrase table. It is annoying.
>>> What should I do now? I hope I won't need to rerun
>>> everything from the scratch
>>>
>>> 2016-11-02 12:26 GMT+02:00 Barry Haddow
>>> <bhaddow@staffmail.ed.ac.uk
>>> <mailto:bhaddow@staffmail.ed.ac.uk>>:
>>>
>>> Hi Hasan
>>>
>>> The error message should be written into
>>> filterphrases.err, inside your working directory,
>>>
>>> cheers - Barry
>>>
>>>
>>> On 02/11/16 10:02, Hasan Sait ARSLAN wrote:
>>>> Hi Hieu,
>>>>
>>>> I did in the way, you want. Plus, I am sure the path
>>>> and file names are correctly spelt.
>>>> But still, I get the same error.
>>>>
>>>> 2016-11-01 21:41 GMT+02:00 Hasan Sait ARSLAN
>>>> <hasan.sait.arslan@gmail.com
>>>> <mailto:hasan.sait.arslan@gmail.com>>:
>>>>
>>>> Thank you Hieu, I will do it, and give you feedback
>>>> about it.
>>>>
>>>> Kind Regards,
>>>>
>>>> 2016-11-01 21:37 GMT+02:00 Hieu Hoang
>>>> <hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>:
>>>>
>>>> the command you execute looks ok. the problem
>>>> is likely to be with your using tuning data.
>>>>
>>>> You should look at your tuning data, make sure
>>>> you've spelt the path and filename correctly,
>>>> use a very small subset (eg. start with 1
>>>> parallel sentence) and increase the number of
>>>> sentences until you find the problem
>>>>
>>>>
>>>>
>>>> On 01/11/2016 12:30, Hasan Sait ARSLAN wrote:
>>>>> Hello,
>>>>>
>>>>> I have trained dataset for 5 days, and for 2-3
>>>>> days I am dealing with the bug in tuning, but
>>>>> still couldn't solve the problem.
>>>>>
>>>>> I use the following command to run:
>>>>>
>>>>> */home/sait/mosesdecoder/scripts/training/mert-moses.pl
>>>>> <http://mert-moses.pl>
>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>>>> <http://dev.clean.tr>
>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.en
>>>>> /home/sait/mosesdecoder/bin/moses
>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>>>> --working-dir
>>>>> /home/sait/Kairit/Task3/working_tr-en/mert
>>>>> --decoder-flags "--threads 32"
>>>>>
>>>>> *
>>>>> Then I get the following error:
>>>>>
>>>>> *Using SCRIPTS_ROOTDIR:
>>>>> /home/sait/mosesdecoder/scripts
>>>>> Assuming --mertdir=/home/sait/mosesdecoder/bin
>>>>> filtering the phrase tables... T nov 1
>>>>> 19:17:39 EET 2016
>>>>> exec:
>>>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>> <http://filter-model-given-input.pl>
>>>>> ./filtered
>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>>>> <http://dev.clean.tr>
>>>>> Executing:
>>>>> /home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>> <http://filter-model-given-input.pl>
>>>>> ./filtered
>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>>>> <http://dev.clean.tr> > filterphrases.out 2>
>>>>> filterphrases.err
>>>>> Exit code: 255
>>>>> ERROR: Failed to run
>>>>> '/home/sait/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>> <http://filter-model-given-input.pl>
>>>>> ./filtered
>>>>> /home/sait/Kairit/Task3/working_tr-en/train/model/moses.ini
>>>>> /home/sait/Kairit/Task3/dataset_tur_en/dev.clean.tr
>>>>> <http://dev.clean.tr>'. at
>>>>> /home/sait/mosesdecoder/scripts/training/mert-moses.pl
>>>>> <http://mert-moses.pl> line 1748.
>>>>>
>>>>> *
>>>>> I have searched over the internet about the
>>>>> error, but unfortunately I couldn't find any
>>>>> beneficial solution.
>>>>>
>>>>> It is so annoying, and eats my time, please
>>>>> help me to solve my problem.
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>
>>>
>>> The University of Edinburgh is a charitable body,
>>> registered in
>>> Scotland, with registration number SC005336.
>>>
>>>
>>
>>
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161102/2360dc04/attachment.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20161102/2360dc04/attachment.pl

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 121, Issue 10
**********************************************

Tweet