Moses-support Digest, Vol 145, Issue 16

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: I can't get moses2 to translate text of sentences like I
did with moses (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sun, 25 Nov 2018 23:34:45 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] I can't get moses2 to translate text of
sentences like I did with moses
To: Jin Nan Lu <owenljn@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbiaGay16wyQW88NQ35pcJQ8msjmsvyarTr4dwr=vRSkmQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

works ok for me.

here's my TargetVocab.dat for you to use
http://statmt.org/hieu/downloads/TargetVocab.dat


Hieu Hoang
http://statmt.org/hieu


On Sun, 25 Nov 2018 at 21:20, Jin Nan Lu <owenljn@gmail.com> wrote:

> Thank you! I just replaced that line with
> testtest 26150
> However, the same error still persists, could it be something else that
> caused this issue? Thanks.
>
>
> On Sun, Nov 25, 2018 at 6:19 AM Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> moses and moses2 are implemented slightly differently. They are not
>> guaranteed to work the same if there is some inconsistency in the data.
>>
>> You can replace the line with
>> whateverHAHAHA 26150
>> and it will work for now.
>>
>> In the next training run you should tweak your pre-processing script to
>> deal with the character. The Moses cleaning and tokenization scripts does
>> most of the cleaning, if you're not using them then you should make sure
>> your data is similarly cleaned
>>
>> Hieu Hoang
>> http://statmt.org/hieu
>>
>>
>> On Sun, 25 Nov 2018 at 02:36, Jin Nan Lu <owenljn@gmail.com> wrote:
>>
>>> Got it, thanks a lot! I have a question regarding this issue, why moses
>>> original version can use the model trained on the same training data? Is it
>>> possible to get moses2 work by removing tabs and consecutive spaces from
>>> .dat files? Thanks again!
>>> Hieu Hoang <hieuhoang@gmail.com>?2018?11?24? ????8:14???
>>>
>>>> In TargetVocab.dat, there is a line
>>>> 26150
>>>> The format of each line in this file is
>>>> word number
>>>> So i guess you need to clean your training corpus to remove tabs and
>>>> consecutive spaces
>>>>
>>>>
>>>> Hieu Hoang
>>>> http://statmt.org/hieu
>>>>
>>>>
>>>> On Sat, 24 Nov 2018 at 21:50, Jin Nan Lu <owenljn@gmail.com> wrote:
>>>>
>>>>> Sure, no problem. I just removed all the unnecessary files, the
>>>>> trained models are in *eng_fra/model* *folder*, the file needs to be
>>>>> translated is called *test.en *and it's placed in the *eng_fra
>>>>> folder. *The integrated phrase table and lexical reordering are
>>>>> placed in the *eng_fra/model/ProbingPT folder. * I've zipped the
>>>>> folder for you to download. Sorry for all the trouble and late reply, it
>>>>> took me some while. Below is the link to download from:
>>>>> part 1:
>>>>> https://drive.google.com/file/d/1SRy85diHKVneellQ6eVeXLJDHhhPyisH/view?usp=sharing
>>>>> part2:
>>>>> https://drive.google.com/file/d/1fMRMJeOFVKMs3lKHOAoCP2z2b4jfqDed/view?usp=sharing
>>>>>
>>>>> Thanks again,
>>>>> Owen
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Nov 24, 2018 at 4:18 PM Hieu Hoang <hieuhoang@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> err, can you just give me a zip file of just the things I need to
>>>>>> reproduce the issue. It looks like you've given me everything, there's too
>>>>>> much for me to sort out
>>>>>>
>>>>>> Hieu Hoang
>>>>>> http://statmt.org/hieu
>>>>>>
>>>>>>
>>>>>> On Sat, 24 Nov 2018 at 21:09, Jin Nan Lu <owenljn@gmail.com> wrote:
>>>>>>
>>>>>>> Hello Hieu,
>>>>>>>
>>>>>>> I've uploaded the folder to Google drive, it contains the trained
>>>>>>> model as well as the test data, here's the download link:
>>>>>>> https://drive.google.com/drive/folders/1aJqQ485yOujFu70y_3Qpvp0Q1WeSEztp?usp=sharing
>>>>>>> I use moses2.ini as the configuration file, and the binarised phrase
>>>>>>> table as well as lexical re-ordering are placed under the model folder,
>>>>>>> their integrated version, the ProbingPT is also generated and placed in the
>>>>>>> model folder.
>>>>>>> I really appreciate your help, thank you!
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Owen
>>>>>>>
>>>>>>> On Fri, Nov 23, 2018 at 7:06 PM Hieu Hoang <hieuhoang@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The ini file looks fine. The only way I can debug the problem is it
>>>>>>>> you can make your files available for download
>>>>>>>>
>>>>>>>> On Fri, 23 Nov 2018, 10:24 pm Jin Nan Lu <owenljn@gmail.com wrote:
>>>>>>>>
>>>>>>>>> Dear support team,
>>>>>>>>>
>>>>>>>>> Hope you are doing well.
>>>>>>>>> I have a serious problem when running moses2 that I can't fix, so
>>>>>>>>> I'm reaching out to you for help, I've attached my moses2.ini file as well
>>>>>>>>> as the problem snapshot, I guarantee you that mosesdecoder is correctly
>>>>>>>>> compiled and installed, and I've binarised the phrase table as well as
>>>>>>>>> lexical reordering file.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Owen
>>>>>>>>>
>>>>>>>>> [image: ??.JPG]
>>>>>>>>> _______________________________________________
>>>>>>>>> Moses-support mailing list
>>>>>>>>> Moses-support@mit.edu
>>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>>>
>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20181125/2da5b242/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ??.JPG
Type: image/jpeg
Size: 135648 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20181125/2da5b242/attachment.jpe

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 145, Issue 16
**********************************************

Tweet