Moses-support Digest, Vol 100, Issue 15

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses R-3 fails with Exception: vector::_M_range_check
(Hieu Hoang)
2. Re: Moses R-3 fails with Exception: vector::_M_range_check
(Tom Hoar)
3. Re: Moses R-3 fails with Exception: vector::_M_range_check
(Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Wed, 04 Feb 2015 13:42:18 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses R-3 fails with Exception:
vector::_M_range_check
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>, Tom Hoar
<tahoar@precisiontranslationtools.com>
Cc: moses-support@mit.edu
Message-ID: <54D221BA.3090207@gmail.com>
Content-Type: text/plain; charset="utf-8"

not specifying .minlex/.minphr works. I vote for just let sleeping dogs
lie. We got better things to do

On 04/02/15 13:29, Marcin Junczys-Dowmunt wrote:
>
> The problem is that Reordering models have a base class that manages
> and delegates loading. If I remember correctly there is some kind of
> preference hard coded, if something exists that has the minlexr
> suffix, take if before *.binphr and *.gz. Some time ago the compact
> phrase table would not work with the suffix, so this was at least
> consistent. That changed in the meantime and now we have this current
> mess.
>
> What would be the best solution?
> A) fix suffix handling,
> B) define a ReorderingCompact feature
> C) both
>
> W dniu 2015-02-04 14:18, Tom Hoar napisa?(a):
>
>> Hi, Marcin,
>>
>> 1) comment out the reordering model (both, feature and weights):
>> works. Exception disappears.
>> 2) Remove file extension ".minlexr" from path= reference: works. file
>> loads and model works
>>
>> Additional test:
>> 3) Remove file extension ".minphr" from compact phrase table's path=
>> reference: works.
>>
>> Is it possible this exception has to do with the filename buffer vice
>> algorithm?
>>
>> FYI, I'm using the build-in boost library from Ubuntu 14.04 libboost 1.54
>> apt-get install libboost-all-dev
>>
>>
>>
>>
>> On 02/04/2015 07:04 PM, Marcin Junczys-Dowmunt wrote:
>>>
>>> Hi,
>>>
>>> urgh, I saw this sometimes, could you comment out the reordering
>>> model (both, feature and weights) and check if you still get that?
>>> If not, uncomment and try to remove the file suffix "minlexr" for
>>> the reordering model. If it still happens, update boost :) (someone
>>> had the same problem and updating boost fixed it miraculously, no
>>> idea why though)
>>>
>>> W dniu 2015-02-04 12:44, Tom Hoar napisa?(a):
>>>
>>> Hi everyone,
>>>
>>> I just build the new R-3. I'm also in the middle of updating our environment. I had a problem preparing for mert-moses.pl. So, I cut back to basics... manual edits to the moses.ini file and command line.
>>>
>>> I attached the moses.ini file and its output log. I edited the moses.ini file to use compact phrase and reordering files, and KenLM with lazy option. It fails with "Exception: vector::_M_range_check".
>>>
>>> Is something wrong or missing with my edits?
>>>
>>> Also, more results addressed separately in another.
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
Research Associate (until March 2015)
** searching for interesting commercial MT position **
University of Edinburgh
http://www.hoang.co.uk/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/29d1123a/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 04 Feb 2015 20:42:17 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Moses R-3 fails with Exception:
vector::_M_range_check
To: moses-support@mit.edu
Message-ID: <54D221B9.4080409@precisiontranslationtools.com>
Content-Type: text/plain; charset="utf-8"

I tested more as it is now and the current work-around (delete the
extension) has limits. If the reordering-table.minlexr and
reordering-table.gz are both present in the referenced folder and
there's no extension in the config, moses loads the text .gz model. If
you remove the .gz file from the folder, moses finds/loads the .minlexr
file. This creates uncertainty and possibly problems for some users.

I think any/all of your solutions proposals will solve uncertainty.
Requiring a fixed extension is probably the easiest. Defining a feature
(ReorderingCompact or LexicalReorderingCompact) would make lexical
configuration management more like the PhraseDictionary &
PhraseDictionaryCompact. There's a lot to be said for consistency.

Ultimately, I don't have a preference either way. Easy is sometimes
better :) For now, we'll use the work-around.



On 02/04/2015 08:29 PM, Marcin Junczys-Dowmunt wrote:
>
> The problem is that Reordering models have a base class that manages
> and delegates loading. If I remember correctly there is some kind of
> preference hard coded, if something exists that has the minlexr
> suffix, take if before *.binphr and *.gz. Some time ago the compact
> phrase table would not work with the suffix, so this was at least
> consistent. That changed in the meantime and now we have this current
> mess.
>
> What would be the best solution?
> A) fix suffix handling,
> B) define a ReorderingCompact feature
> C) both
>
> W dniu 2015-02-04 14:18, Tom Hoar napisa?(a):
>
>> Hi, Marcin,
>>
>> 1) comment out the reordering model (both, feature and weights):
>> works. Exception disappears.
>> 2) Remove file extension ".minlexr" from path= reference: works. file
>> loads and model works
>>
>> Additional test:
>> 3) Remove file extension ".minphr" from compact phrase table's path=
>> reference: works.
>>
>> Is it possible this exception has to do with the filename buffer vice
>> algorithm?
>>
>> FYI, I'm using the build-in boost library from Ubuntu 14.04 libboost 1.54
>> apt-get install libboost-all-dev
>>
>>
>>
>>
>> On 02/04/2015 07:04 PM, Marcin Junczys-Dowmunt wrote:
>>>
>>> Hi,
>>>
>>> urgh, I saw this sometimes, could you comment out the reordering
>>> model (both, feature and weights) and check if you still get that?
>>> If not, uncomment and try to remove the file suffix "minlexr" for
>>> the reordering model. If it still happens, update boost :) (someone
>>> had the same problem and updating boost fixed it miraculously, no
>>> idea why though)
>>>
>>> W dniu 2015-02-04 12:44, Tom Hoar napisa?(a):
>>>
>>> Hi everyone,
>>>
>>> I just build the new R-3. I'm also in the middle of updating our environment. I had a problem preparing for mert-moses.pl. So, I cut back to basics... manual edits to the moses.ini file and command line.
>>>
>>> I attached the moses.ini file and its output log. I edited the moses.ini file to use compact phrase and reordering files, and KenLM with lazy option. It fails with "Exception: vector::_M_range_check".
>>>
>>> Is something wrong or missing with my edits?
>>>
>>> Also, more results addressed separately in another.
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/0ddbacb0/attachment-0001.htm

------------------------------

Message: 3
Date: Wed, 04 Feb 2015 14:45:01 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Moses R-3 fails with Exception:
vector::_M_range_check
To: Tom Hoar <tahoar@precisiontranslationtools.com>
Cc: moses-support@mit.edu
Message-ID: <2e110afc52e25743d94e3a33a299b433@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"



That's weird. It used to find the *.minlexr before the *.gz

W dniu 2015-02-04 14:42, Tom Hoar napisa?(a):

> I tested more as it is now and the current work-around (delete the extension) has limits. If the reordering-table.minlexr and reordering-table.gz are both present in the referenced folder and there's no extension in the config, moses loads the text .gz model. If you remove the .gz file from the folder, moses finds/loads the .minlexr file. This creates uncertainty and possibly problems for some users.
>
> I think any/all of your solutions proposals will solve uncertainty. Requiring a fixed extension is probably the easiest. Defining a feature (ReorderingCompact or LexicalReorderingCompact) would make lexical configuration management more like the PhraseDictionary & PhraseDictionaryCompact. There's a lot to be said for consistency.
>
> Ultimately, I don't have a preference either way. Easy is sometimes better :) For now, we'll use the work-around.
>
> On 02/04/2015 08:29 PM, Marcin Junczys-Dowmunt wrote:
>
> The problem is that Reordering models have a base class that manages and delegates loading. If I remember correctly there is some kind of preference hard coded, if something exists that has the minlexr suffix, take if before *.binphr and *.gz. Some time ago the compact phrase table would not work with the suffix, so this was at least consistent. That changed in the meantime and now we have this current mess.
>
> What would be the best solution?
> A) fix suffix handling,
> B) define a ReorderingCompact feature
> C) both
>
> W dniu 2015-02-04 14:18, Tom Hoar napisa?(a): Hi, Marcin,
>
> 1) comment out the reordering model (both, feature and weights): works. Exception disappears.
> 2) Remove file extension ".minlexr" from path= reference: works. file loads and model works
>
> Additional test:
> 3) Remove file extension ".minphr" from compact phrase table's path= reference: works.
>
> Is it possible this exception has to do with the filename buffer vice algorithm?
>
> FYI, I'm using the build-in boost library from Ubuntu 14.04 libboost 1.54
> apt-get install libboost-all-dev
>
> On 02/04/2015 07:04 PM, Marcin Junczys-Dowmunt wrote:
>
> Hi,
>
> urgh, I saw this sometimes, could you comment out the reordering model (both, feature and weights) and check if you still get that? If not, uncomment and try to remove the file suffix "minlexr" for the reordering model. If it still happens, update boost :) (someone had the same problem and updating boost fixed it miraculously, no idea why though)
>
> W dniu 2015-02-04 12:44, Tom Hoar napisa?(a):
>
> Hi everyone,
>
> I just build the new R-3. I'm also in the middle of updating our environment. I had a problem preparing for mert-moses.pl. So, I cut back to basics... manual edits to the moses.ini file and command line.
>
> I attached the moses.ini file and its output log. I edited the moses.ini file to use compact phrase and reordering files, and KenLM with lazy option. It fails with "Exception: vector::_M_range_check".
>
> Is something wrong or missing with my edits?
>
> Also, more results addressed separately in another.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]



Links:
------
[1] http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/cd9f39a0/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 100, Issue 15
**********************************************

Tweet