Moses-support Digest, Vol 145, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: I can't get moses2 to translate text of sentences like I
did with moses (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sun, 25 Nov 2018 11:19:10 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] I can't get moses2 to translate text of
sentences like I did with moses
To: Jin Nan Lu <owenljn@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbicB_YJ9UT_YvoCPJcj48Z0EqVpC9KLn=9vjSCUoukRMg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

moses and moses2 are implemented slightly differently. They are not
guaranteed to work the same if there is some inconsistency in the data.

You can replace the line with
whateverHAHAHA 26150
and it will work for now.

In the next training run you should tweak your pre-processing script to
deal with the character. The Moses cleaning and tokenization scripts does
most of the cleaning, if you're not using them then you should make sure
your data is similarly cleaned

Hieu Hoang
http://statmt.org/hieu


On Sun, 25 Nov 2018 at 02:36, Jin Nan Lu <owenljn@gmail.com> wrote:

> Got it, thanks a lot! I have a question regarding this issue, why moses
> original version can use the model trained on the same training data? Is it
> possible to get moses2 work by removing tabs and consecutive spaces from
> .dat files? Thanks again!
> Hieu Hoang <hieuhoang@gmail.com>?2018?11?24? ????8:14???
>
>> In TargetVocab.dat, there is a line
>> 26150
>> The format of each line in this file is
>> word number
>> So i guess you need to clean your training corpus to remove tabs and
>> consecutive spaces
>>
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Sat, 24 Nov 2018 at 21:50, Jin Nan Lu <owenljn@gmail.com> wrote:
>>
>>> Sure, no problem. I just removed all the unnecessary files, the trained
>>> models are in *eng_fra/model* *folder*, the file needs to be translated
>>> is called *test.en *and it's placed in the *eng_fra folder. *The
>>> integrated phrase table and lexical reordering are placed in the *eng_fra/model/ProbingPT
>>> folder. * I've zipped the folder for you to download. Sorry for all the
>>> trouble and late reply, it took me some while. Below is the link to
>>> download from:
>>> part 1:
>>> https://drive.google.com/file/d/1SRy85diHKVneellQ6eVeXLJDHhhPyisH/view?usp=sharing
>>> part2:
>>> https://drive.google.com/file/d/1fMRMJeOFVKMs3lKHOAoCP2z2b4jfqDed/view?usp=sharing
>>>
>>> Thanks again,
>>> Owen
>>>
>>>
>>>
>>> On Sat, Nov 24, 2018 at 4:18 PM Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>
>>>> err, can you just give me a zip file of just the things I need to
>>>> reproduce the issue. It looks like you've given me everything, there's too
>>>> much for me to sort out
>>>>
>>>> Hieu Hoang
>>>> http://statmt.org/hieu
>>>>
>>>>
>>>> On Sat, 24 Nov 2018 at 21:09, Jin Nan Lu <owenljn@gmail.com> wrote:
>>>>
>>>>> Hello Hieu,
>>>>>
>>>>> I've uploaded the folder to Google drive, it contains the trained
>>>>> model as well as the test data, here's the download link:
>>>>> https://drive.google.com/drive/folders/1aJqQ485yOujFu70y_3Qpvp0Q1WeSEztp?usp=sharing
>>>>> I use moses2.ini as the configuration file, and the binarised phrase
>>>>> table as well as lexical re-ordering are placed under the model folder,
>>>>> their integrated version, the ProbingPT is also generated and placed in the
>>>>> model folder.
>>>>> I really appreciate your help, thank you!
>>>>>
>>>>> Best Regards,
>>>>> Owen
>>>>>
>>>>> On Fri, Nov 23, 2018 at 7:06 PM Hieu Hoang <hieuhoang@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> The ini file looks fine. The only way I can debug the problem is it
>>>>>> you can make your files available for download
>>>>>>
>>>>>> On Fri, 23 Nov 2018, 10:24 pm Jin Nan Lu <owenljn@gmail.com wrote:
>>>>>>
>>>>>>> Dear support team,
>>>>>>>
>>>>>>> Hope you are doing well.
>>>>>>> I have a serious problem when running moses2 that I can't fix, so
>>>>>>> I'm reaching out to you for help, I've attached my moses2.ini file as well
>>>>>>> as the problem snapshot, I guarantee you that mosesdecoder is correctly
>>>>>>> compiled and installed, and I've binarised the phrase table as well as
>>>>>>> lexical reordering file.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Owen
>>>>>>>
>>>>>>> [image: ??.JPG]
>>>>>>> _______________________________________________
>>>>>>> Moses-support mailing list
>>>>>>> Moses-support@mit.edu
>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20181125/2878aef4/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ??.JPG
Type: image/jpeg
Size: 135648 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20181125/2878aef4/attachment.jpe

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 145, Issue 14
**********************************************

Tweet