Moses-support Digest, Vol 100, Issue 16

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses R-3 fails with Exception: vector::_M_range_check
(Tom Hoar)
2. Re: Moses R-3 fails with Exception: vector::_M_range_check
(Kenneth Heafield)
3. Re: Moses R-3 fails with Exception: vector::_M_range_check
(Marcin Junczys-Dowmunt)


----------------------------------------------------------------------

Message: 1
Date: Wed, 04 Feb 2015 20:47:35 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Moses R-3 fails with Exception:
vector::_M_range_check
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support@mit.edu
Message-ID: <54D222F7.6020103@precisiontranslationtools.com>
Content-Type: text/plain; charset="utf-8"

Sorry, I tested but the behavior wasn't really clear. Maybe it did load
the .minlexr first. Another test would be what happens if the raw
(uncompressed) text file is there without an extension. Either way, I
vote for declarations vice uncertainty.


On 02/04/2015 08:45 PM, Marcin Junczys-Dowmunt wrote:
>
> That's weird. It used to find the *.minlexr before the *.gz
>
> W dniu 2015-02-04 14:42, Tom Hoar napisa?(a):
>
>> I tested more as it is now and the current work-around (delete the
>> extension) has limits. If the reordering-table.minlexr and
>> reordering-table.gz are both present in the referenced folder and
>> there's no extension in the config, moses loads the text .gz model.
>> If you remove the .gz file from the folder, moses finds/loads the
>> .minlexr file. This creates uncertainty and possibly problems for
>> some users.
>>
>> I think any/all of your solutions proposals will solve uncertainty.
>> Requiring a fixed extension is probably the easiest. Defining a
>> feature (ReorderingCompact or LexicalReorderingCompact) would make
>> lexical configuration management more like the PhraseDictionary &
>> PhraseDictionaryCompact. There's a lot to be said for consistency.
>>
>> Ultimately, I don't have a preference either way. Easy is sometimes
>> better :) For now, we'll use the work-around.
>>
>>
>>
>> On 02/04/2015 08:29 PM, Marcin Junczys-Dowmunt wrote:
>>>
>>> The problem is that Reordering models have a base class that manages
>>> and delegates loading. If I remember correctly there is some kind of
>>> preference hard coded, if something exists that has the minlexr
>>> suffix, take if before *.binphr and *.gz. Some time ago the compact
>>> phrase table would not work with the suffix, so this was at least
>>> consistent. That changed in the meantime and now we have this
>>> current mess.
>>>
>>> What would be the best solution?
>>> A) fix suffix handling,
>>> B) define a ReorderingCompact feature
>>> C) both
>>>
>>> W dniu 2015-02-04 14:18, Tom Hoar napisa?(a):
>>>
>>> Hi, Marcin,
>>>
>>> 1) comment out the reordering model (both, feature and weights):
>>> works. Exception disappears.
>>> 2) Remove file extension ".minlexr" from path= reference: works.
>>> file loads and model works
>>>
>>> Additional test:
>>> 3) Remove file extension ".minphr" from compact phrase table's
>>> path= reference: works.
>>>
>>> Is it possible this exception has to do with the filename buffer
>>> vice algorithm?
>>>
>>> FYI, I'm using the build-in boost library from Ubuntu 14.04
>>> libboost 1.54
>>> apt-get install libboost-all-dev
>>>
>>>
>>>
>>>
>>> On 02/04/2015 07:04 PM, Marcin Junczys-Dowmunt wrote:
>>>
>>> Hi,
>>>
>>> urgh, I saw this sometimes, could you comment out the
>>> reordering model (both, feature and weights) and check if
>>> you still get that? If not, uncomment and try to remove the
>>> file suffix "minlexr" for the reordering model. If it still
>>> happens, update boost :) (someone had the same problem and
>>> updating boost fixed it miraculously, no idea why though)
>>>
>>> W dniu 2015-02-04 12:44, Tom Hoar napisa?(a):
>>>
>>> Hi everyone,
>>>
>>> I just build the new R-3. I'm also in the middle of updating our environment. I had a problem preparing for mert-moses.pl. So, I cut back to basics... manual edits to the moses.ini file and command line.
>>>
>>> I attached the moses.ini file and its output log. I edited the moses.ini file to use compact phrase and reordering files, and KenLM with lazy option. It fails with "Exception: vector::_M_range_check".
>>>
>>> Is something wrong or missing with my edits?
>>>
>>> Also, more results addressed separately in another.
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/778ba186/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 04 Feb 2015 08:54:32 -0500
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Moses R-3 fails with Exception:
vector::_M_range_check
To: moses-support@mit.edu
Message-ID: <54D22498.8060308@kheafield.com>
Content-Type: text/plain; charset=UTF-8

Vote:

- The file name specified exists and is loaded.
- There is no semantic meaning to the file name.
- Marcin's header (there is a header, right?) indicates it's a minphr.

On 02/04/2015 08:47 AM, Tom Hoar wrote:
> Sorry, I tested but the behavior wasn't really clear. Maybe it did load
> the .minlexr first. Another test would be what happens if the raw
> (uncompressed) text file is there without an extension. Either way, I
> vote for declarations vice uncertainty.
>
>
> On 02/04/2015 08:45 PM, Marcin Junczys-Dowmunt wrote:
>>
>> That's weird. It used to find the *.minlexr before the *.gz
>>
>> W dniu 2015-02-04 14:42, Tom Hoar napisa?(a):
>>
>>> I tested more as it is now and the current work-around (delete the
>>> extension) has limits. If the reordering-table.minlexr and
>>> reordering-table.gz are both present in the referenced folder and
>>> there's no extension in the config, moses loads the text .gz model.
>>> If you remove the .gz file from the folder, moses finds/loads the
>>> .minlexr file. This creates uncertainty and possibly problems for
>>> some users.
>>>
>>> I think any/all of your solutions proposals will solve uncertainty.
>>> Requiring a fixed extension is probably the easiest. Defining a
>>> feature (ReorderingCompact or LexicalReorderingCompact) would make
>>> lexical configuration management more like the PhraseDictionary &
>>> PhraseDictionaryCompact. There's a lot to be said for consistency.
>>>
>>> Ultimately, I don't have a preference either way. Easy is sometimes
>>> better :) For now, we'll use the work-around.
>>>
>>>
>>>
>>> On 02/04/2015 08:29 PM, Marcin Junczys-Dowmunt wrote:
>>>>
>>>> The problem is that Reordering models have a base class that manages
>>>> and delegates loading. If I remember correctly there is some kind of
>>>> preference hard coded, if something exists that has the minlexr
>>>> suffix, take if before *.binphr and *.gz. Some time ago the compact
>>>> phrase table would not work with the suffix, so this was at least
>>>> consistent. That changed in the meantime and now we have this
>>>> current mess.
>>>>
>>>> What would be the best solution?
>>>> A) fix suffix handling,
>>>> B) define a ReorderingCompact feature
>>>> C) both
>>>>
>>>> W dniu 2015-02-04 14:18, Tom Hoar napisa?(a):
>>>>
>>>> Hi, Marcin,
>>>>
>>>> 1) comment out the reordering model (both, feature and weights):
>>>> works. Exception disappears.
>>>> 2) Remove file extension ".minlexr" from path= reference: works.
>>>> file loads and model works
>>>>
>>>> Additional test:
>>>> 3) Remove file extension ".minphr" from compact phrase table's
>>>> path= reference: works.
>>>>
>>>> Is it possible this exception has to do with the filename buffer
>>>> vice algorithm?
>>>>
>>>> FYI, I'm using the build-in boost library from Ubuntu 14.04
>>>> libboost 1.54
>>>> apt-get install libboost-all-dev
>>>>
>>>>
>>>>
>>>>
>>>> On 02/04/2015 07:04 PM, Marcin Junczys-Dowmunt wrote:
>>>>
>>>> Hi,
>>>>
>>>> urgh, I saw this sometimes, could you comment out the
>>>> reordering model (both, feature and weights) and check if
>>>> you still get that? If not, uncomment and try to remove the
>>>> file suffix "minlexr" for the reordering model. If it still
>>>> happens, update boost :) (someone had the same problem and
>>>> updating boost fixed it miraculously, no idea why though)
>>>>
>>>>
>>>>
>>>> W dniu 2015-02-04 12:44, Tom Hoar napisa?(a):
>>>>
>>>> Hi everyone,
>>>>
>>>> I just build the new R-3. I'm also in the middle of updating our environment. I had a problem preparing for mert-moses.pl. So, I cut back to basics... manual edits to the moses.ini file and command line.
>>>>
>>>> I attached the moses.ini file and its output log. I edited the moses.ini file to use compact phrase and reordering files, and KenLM with lazy option. It fails with "Exception: vector::_M_range_check".
>>>>
>>>> Is something wrong or missing with my edits?
>>>>
>>>> Also, more results addressed separately in another.
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 3
Date: Wed, 04 Feb 2015 14:57:51 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Moses R-3 fails with Exception:
vector::_M_range_check
To: Kenneth Heafield <moses@kheafield.com>
Cc: moses-support@mit.edu
Message-ID: <6f46719c81ece7b713743d764eeeac69@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"



Nope, no header. Forgot to add one in the beginning and got a lot of
angry stares whenever I mentioned potential backwards-incompatibility
due to introducing one.

W dniu 2015-02-04 14:54, Kenneth Heafield napisa?(a):

> Vote:
>
> - The file name specified exists and is loaded.
> - There is no semantic meaning to the file name.
> - Marcin's header (there is a header, right?) indicates it's a minphr.
>
> On 02/04/2015 08:47 AM, Tom Hoar wrote:
> Sorry, I tested but the behavior wasn't really clear. Maybe it did load the .minlexr first. Another test would be what happens if the raw (uncompressed) text file is there without an extension. Either way, I vote for declarations vice uncertainty. On 02/04/2015 08:45 PM, Marcin Junczys-Dowmunt wrote: That's weird. It used to find the *.minlexr before the *.gz W dniu 2015-02-04 14:42, Tom Hoar napisa?(a): I tested more as it is now and the current work-around (delete the extension) has limits. If the reordering-table.minlexr and reordering-table.gz are both present in the referenced folder and there's no extension in the config, moses loads the text .gz model. If you remove the .gz file from the folder, moses finds/loads the .minlexr file. This creates uncertainty and possibly problems for some users. I think any/all of your solutions proposals will solve uncertainty. Requiring a fixed extension is probably the easiest. Defining a feature (ReorderingCompact or
LexicalReorderingCompact) would make lexical configuration management more like the PhraseDictionary & PhraseDictionaryCompact. There's a lot to be said for consistency. Ultimately, I don't have a preference either way. Easy is sometimes better :) For now, we'll use the work-around. On 02/04/2015 08:29 PM, Marcin Junczys-Dowmunt wrote: The problem is that Reordering models have a base class that manages and delegates loading. If I remember correctly there is some kind of preference hard coded, if something exists that has the minlexr suffix, take if before *.binphr and *.gz. Some time ago the compact phrase table would not work with the suffix, so this was at least consistent. That changed in the meantime and now we have this current mess. What would be the best solution? A) fix suffix handling, B) define a ReorderingCompact feature C) both W dniu 2015-02-04 14:18, Tom Hoar napisa?(a): Hi, Marcin, 1) comment out the reordering model (both, feature and weights): works. Except!
ion
disappears. 2) Remove file extension ".minlexr" from path= reference: works. file loads and model works Additional test: 3) Remove file extension ".minphr" from compact phrase table's path= reference: works. Is it possible this exception has to do with the filename buffer vice algorithm? FYI, I'm using the build-in boost library from Ubuntu 14.04 libboost 1.54 apt-get install libboost-all-dev On 02/04/2015 07:04 PM, Marcin Junczys-Dowmunt wrote: Hi, urgh, I saw this sometimes, could you comment out the reordering model (both, feature and weights) and check if you still get that? If not, uncomment and try to remove the file suffix "minlexr" for the reordering model. If it still happens, update boost :) (someone had the same problem and updating boost fixed it miraculously, no idea why though) W dniu 2015-02-04 12:44, Tom Hoar napisa?(a): Hi everyone, I just build the new R-3. I'm also in the middle of updating our environment. I had a problem preparing for mert-moses.pl. So, !
I cut
back to basics... manual edits to the moses.ini file and command line. I attached the moses.ini file and its output log. I edited the moses.ini file to use compact phrase and reordering files, and KenLM with lazy option. It fails with "Exception: vector::_M_range_check". Is something wrong or missing with my edits? Also, more results addressed separately in another. _______________________________________________ Moses-support mailing list Moses-support@mit.edu <mailto:Moses-support@mit.edu> http://mailman.mit.edu/mailman/listinfo/moses-support [1] _______________________________________________ Moses-support mailing list Moses-support@mit.edu http://mailman.mit.edu/mailman/listinfo/moses-support [1] _______________________________________________ Moses-support mailing list Moses-support@mit.edu <mailto:Moses-support@mit.edu> http://mailman.mit.edu/mailman/listinfo/moses-support [1] _______________________________________________ Moses-support mailing list Moses-support@mit.!
edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1] _______________________________________________ Moses-support mailing list Moses-support@mit.edu <mailto:Moses-support@mit.edu> http://mailman.mit.edu/mailman/listinfo/moses-support [1]
_______________________________________________ Moses-support mailing
list Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support [1]



Links:
------
[1] http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150204/74e3e0d1/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 100, Issue 16
**********************************************

Tweet