Moses-support Digest, Vol 145, Issue 15

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: I can't get moses2 to translate text of sentences like I
did with moses (Jin Nan Lu)


----------------------------------------------------------------------

Message: 1
Date: Sun, 25 Nov 2018 16:19:46 -0500
From: Jin Nan Lu <owenljn@gmail.com>
Subject: Re: [Moses-support] I can't get moses2 to translate text of
sentences like I did with moses
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CADWayMzC308ziGeQbF3T+ZfFUGu2mO=KmnHGZSBrD6DerTk-0Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Thank you! I just replaced that line with
testtest 26150
However, the same error still persists, could it be something else that
caused this issue? Thanks.


On Sun, Nov 25, 2018 at 6:19 AM Hieu Hoang <hieuhoang@gmail.com> wrote:

> moses and moses2 are implemented slightly differently. They are not
> guaranteed to work the same if there is some inconsistency in the data.
>
> You can replace the line with
> whateverHAHAHA 26150
> and it will work for now.
>
> In the next training run you should tweak your pre-processing script to
> deal with the character. The Moses cleaning and tokenization scripts does
> most of the cleaning, if you're not using them then you should make sure
> your data is similarly cleaned
>
> Hieu Hoang
> http://statmt.org/hieu
>
>
> On Sun, 25 Nov 2018 at 02:36, Jin Nan Lu <owenljn@gmail.com> wrote:
>
>> Got it, thanks a lot! I have a question regarding this issue, why moses
>> original version can use the model trained on the same training data? Is it
>> possible to get moses2 work by removing tabs and consecutive spaces from
>> .dat files? Thanks again!
>> Hieu Hoang <hieuhoang@gmail.com>?2018?11?24? ????8:14???
>>
>>> In TargetVocab.dat, there is a line
>>> 26150
>>> The format of each line in this file is
>>> word number
>>> So i guess you need to clean your training corpus to remove tabs and
>>> consecutive spaces
>>>
>>>
>>> Hieu Hoang
>>> http://statmt.org/hieu
>>>
>>>
>>> On Sat, 24 Nov 2018 at 21:50, Jin Nan Lu <owenljn@gmail.com> wrote:
>>>
>>>> Sure, no problem. I just removed all the unnecessary files, the
>>>> trained models are in *eng_fra/model* *folder*, the file needs to be
>>>> translated is called *test.en *and it's placed in the *eng_fra folder.
>>>> *The integrated phrase table and lexical reordering are placed in the *eng_fra/model/ProbingPT
>>>> folder. * I've zipped the folder for you to download. Sorry for all
>>>> the trouble and late reply, it took me some while. Below is the link to
>>>> download from:
>>>> part 1:
>>>> https://drive.google.com/file/d/1SRy85diHKVneellQ6eVeXLJDHhhPyisH/view?usp=sharing
>>>> part2:
>>>> https://drive.google.com/file/d/1fMRMJeOFVKMs3lKHOAoCP2z2b4jfqDed/view?usp=sharing
>>>>
>>>> Thanks again,
>>>> Owen
>>>>
>>>>
>>>>
>>>> On Sat, Nov 24, 2018 at 4:18 PM Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>>
>>>>> err, can you just give me a zip file of just the things I need to
>>>>> reproduce the issue. It looks like you've given me everything, there's too
>>>>> much for me to sort out
>>>>>
>>>>> Hieu Hoang
>>>>> http://statmt.org/hieu
>>>>>
>>>>>
>>>>> On Sat, 24 Nov 2018 at 21:09, Jin Nan Lu <owenljn@gmail.com> wrote:
>>>>>
>>>>>> Hello Hieu,
>>>>>>
>>>>>> I've uploaded the folder to Google drive, it contains the trained
>>>>>> model as well as the test data, here's the download link:
>>>>>> https://drive.google.com/drive/folders/1aJqQ485yOujFu70y_3Qpvp0Q1WeSEztp?usp=sharing
>>>>>> I use moses2.ini as the configuration file, and the binarised phrase
>>>>>> table as well as lexical re-ordering are placed under the model folder,
>>>>>> their integrated version, the ProbingPT is also generated and placed in the
>>>>>> model folder.
>>>>>> I really appreciate your help, thank you!
>>>>>>
>>>>>> Best Regards,
>>>>>> Owen
>>>>>>
>>>>>> On Fri, Nov 23, 2018 at 7:06 PM Hieu Hoang <hieuhoang@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The ini file looks fine. The only way I can debug the problem is it
>>>>>>> you can make your files available for download
>>>>>>>
>>>>>>> On Fri, 23 Nov 2018, 10:24 pm Jin Nan Lu <owenljn@gmail.com wrote:
>>>>>>>
>>>>>>>> Dear support team,
>>>>>>>>
>>>>>>>> Hope you are doing well.
>>>>>>>> I have a serious problem when running moses2 that I can't fix, so
>>>>>>>> I'm reaching out to you for help, I've attached my moses2.ini file as well
>>>>>>>> as the problem snapshot, I guarantee you that mosesdecoder is correctly
>>>>>>>> compiled and installed, and I've binarised the phrase table as well as
>>>>>>>> lexical reordering file.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Owen
>>>>>>>>
>>>>>>>> [image: ??.JPG]
>>>>>>>> _______________________________________________
>>>>>>>> Moses-support mailing list
>>>>>>>> Moses-support@mit.edu
>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>>>>
>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20181125/eace336c/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ??.JPG
Type: image/jpeg
Size: 135648 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20181125/eace336c/attachment.jpe

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 145, Issue 15
**********************************************

Tweet