Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: --activate-features in mert-moses.perl not working?
(Rico Sennrich)
2. Re: Tune crashed in ems (Barry Haddow)
3. Re: Tune crashed in ems (karan sharma)
4. Re: --activate-features in mert-moses.perl not working?
(Hieu Hoang)
5. Re: Improving SMT using RBMT (Philipp Koehn)
----------------------------------------------------------------------
Message: 1
Date: Mon, 24 Feb 2014 10:21:09 +0000 (UTC)
From: Rico Sennrich <rico.sennrich@gmx.ch>
Subject: Re: [Moses-support] --activate-features in mert-moses.perl
not working?
To: moses-support@mit.edu
Message-ID: <loom.20140224T110930-632@post.gmane.org>
Content-Type: text/plain; charset=us-ascii
Marcin Junczys-Dowmunt <junczys@...> writes:
>
> And with "tuneable=false" it seems the features are being ignored during
> decoding, I understand this should not be happening. I get much worse
> translation results with an ini-file that has "tuneable=false" for all
> features than with the same ini without the option. The translation is
> also much faster with the options specified, so something is clearly not
> being evaluated.
>
> W dniu 23.02.2014 00:30, Marcin Junczys-Dowmunt pisze:
> > BTW. "tuneable=false" seems to be ignored by Kenlm, works with other
> > features though.
> >
The option currently doesn't work consistently because
FeatureFunction::SetParameter, which handles the setting of the tuneable
parameter, is overridden by some subclasses. I'll add a way to separate
global parameters (that are not overridden by subclassing) from others.
Can you say which features are being ignored during decoding? My guess is
that your problem also has to do with the feature-specific behaviour in
SetParameter.
best,
Rico
------------------------------
Message: 2
Date: Mon, 24 Feb 2014 10:26:18 +0000
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Tune crashed in ems
To: karan sharma <karan.sharma.bond@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <530B1E4A.8070009@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Hi Karan
If you look at the error file, you will see the message:
> Check nextPos != string::npos failed in moses/Phrase.cpp:209
In recent versions of Moses (v2.1 onwards) this has been replaced with a
more descriptive error message:
> Incorrect formatting of non-terminal. Should have 2 non-terms, eg.
> [X][X].
This means that there is a formatting error in your phrase table.
Possibly this is because there were special characters such as | or [ in
your corpus, and you did not escape them. Did you use the Moses
tokeniser? If you use your own tokeniser then you'll need to escape
special characters yourself using the escape-special-chars.perl script.
cheers - Barry
On 24/02/14 09:38, karan sharma wrote:
> Sir,
> I have attached the STDERR file.
> Could you please look into it.
>
> Thanks
>
>
> On Mon, Feb 24, 2014 at 2:49 PM, Barry Haddow
> <bhaddow@staffmail.ed.ac.uk <mailto:bhaddow@staffmail.ed.ac.uk>> wrote:
>
> Hi Karan
>
> There's not enough information here to diagnose the problem. Could
> you attach the tuning STDERR file -- there should be an earlier
> error message in it,
>
> cheers - Barry
>
>
> On 24/02/14 09:05, karan sharma wrote:
>
> Hey,
> I am training a phrase based model using EMS.
> The tuning process is crashing . The error I am getting is :
>
> Aborted (core dumped)
> Exit code: 134
>
>
> Can anyone help me with it .
>
> Thanks
> Karan Sharma
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 3
Date: Mon, 24 Feb 2014 16:07:35 +0530
From: karan sharma <karan.sharma.bond@gmail.com>
Subject: Re: [Moses-support] Tune crashed in ems
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAFc-37R1A+wDJGY2H-rGgTDJ6e4aOFawUeWUVa9R2P2UVn6Z9g@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Sir,
Thanks for the feedback. I have used my own tokenizer. I will exclude the
special characters and try again.
Thanks
On Mon, Feb 24, 2014 at 3:56 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk>wrote:
> Hi Karan
>
> If you look at the error file, you will see the message:
>
> Check nextPos != string::npos failed in moses/Phrase.cpp:209
>>
>
> In recent versions of Moses (v2.1 onwards) this has been replaced with a
> more descriptive error message:
>
> Incorrect formatting of non-terminal. Should have 2 non-terms, eg.
>> [X][X].
>>
>
> This means that there is a formatting error in your phrase table. Possibly
> this is because there were special characters such as | or [ in your
> corpus, and you did not escape them. Did you use the Moses tokeniser? If
> you use your own tokeniser then you'll need to escape special characters
> yourself using the escape-special-chars.perl script.
>
> cheers - Barry
>
>
> On 24/02/14 09:38, karan sharma wrote:
>
>> Sir,
>> I have attached the STDERR file.
>> Could you please look into it.
>>
>> Thanks
>>
>>
>> On Mon, Feb 24, 2014 at 2:49 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk<mailto:
>> bhaddow@staffmail.ed.ac.uk>> wrote:
>>
>> Hi Karan
>>
>> There's not enough information here to diagnose the problem. Could
>> you attach the tuning STDERR file -- there should be an earlier
>> error message in it,
>>
>> cheers - Barry
>>
>>
>> On 24/02/14 09:05, karan sharma wrote:
>>
>> Hey,
>> I am training a phrase based model using EMS.
>> The tuning process is crashing . The error I am getting is :
>>
>> Aborted (core dumped)
>> Exit code: 134
>>
>>
>> Can anyone help me with it .
>>
>> Thanks
>> Karan Sharma
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>> -- The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140224/126dfa4f/attachment-0001.htm
------------------------------
Message: 4
Date: Mon, 24 Feb 2014 10:56:16 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] --activate-features in mert-moses.perl
not working?
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <EAE033F9-4FBD-41D2-BC9A-6FB235689416@gmail.com>
Content-Type: text/plain; charset=us-ascii
Can you please send me your ini file where you set tuneable = false.
This param has to work, the unknown word penalty depends on it
Sent while bumping into things
> On 23 Feb 2014, at 12:13 am, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:
>
> And with "tuneable=false" it seems the features are being ignored during
> decoding, I understand this should not be happening. I get much worse
> translation results with an ini-file that has "tuneable=false" for all
> features than with the same ini without the option. The translation is
> also much faster with the options specified, so something is clearly not
> being evaluated.
>
> W dniu 23.02.2014 00:30, Marcin Junczys-Dowmunt pisze:
>> BTW. "tuneable=false" seems to be ignored by Kenlm, works with other
>> features though.
>>
>> W dniu 10.02.2014 21:15, Rico Sennrich pisze:
>>> Marcin Junczys-Dowmunt <junczys@...> writes:
>>>
>>>> Hi,
>>>> it seems --activate-features=STRING is not working in mert-moses.perl.
>>>> The script prints a message that the ignored features are not being
>>>> used, but then optimizes them anyway. I can see that the "enabled"
>>>> information in the feature data structure is not being used anywhere in
>>>> the script once it has been set (apart from printing the message).
>>> I don't know too much about the --activate-features option myself, but in
>>> recent Moses versions, you can add the option 'tuneable=false' to a feature
>>> function in the config. The effect is that the feature score(s) won't be
>>> reported to the n-best list, and MERT/MIRA/PRO won't even know that the
>>> feature exists. The weight from the original config will be used for all
>>> tuning iterations, and copied to the final config. You can now also specify
>>> the weight of sparse features in the config, and this will override the
>>> weight set in the weights file.
>>>
>>> best wishes,
>>> Rico
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
Message: 5
Date: Mon, 24 Feb 2014 09:07:17 -0500
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Improving SMT using RBMT
To: Kalyani Baruah <kajubaruah04@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDD20ZOtjZjhYsFsS9hEnJEY3okJ-VS+38vVS0sYU7-kWA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
this is very little training data, which will likely lead to poor
translation performance.
You may get better results, if you work with morphological analysers, additional
dictionaries and pre-reordering. If you want to call any this RBMT, go ahead.
-phi
On Mon, Feb 24, 2014 at 2:38 AM, Kalyani Baruah <kajubaruah04@gmail.com> wrote:
> Hi all, I have built a SMT system using the moses manual for a pair of about
> 2500 sentences for Assamese and English language.
> I want to improve the translation quality. Is it possible to use the Rule
> Based Machine Translation into my SMT system and make my system hybrid. If
> it is possible, please guide me how to do so.
>
> Regards,
>
>
>
> Kalyanee Kanchan Baruah
> Institute of Science and Technology,
> Gauhati University,Guwahati,India
> Phone- +91-9706242124
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 88, Issue 50
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 88, Issue 50"
Post a Comment