Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: CeateOnDiskPt throws 'Already Saved' Exception on phrase
table in sample_models/phrase-model (Hieu Hoang)
2. Re: CeateOnDiskPt throws 'Already Saved' Exception on phrase
table in sample_models/phrase-model (kwame porter robinson)
3. Re: CeateOnDiskPt throws 'Already Saved' Exception on phrase
table in sample_models/phrase-model (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Tue, 5 May 2015 17:33:23 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] CeateOnDiskPt throws 'Already Saved'
Exception on phrase table in sample_models/phrase-model
To: kwame porter robinson <k.porter.robinson@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbiqDk6KBw0SMBMKEnk_nwWpeo07vSGBurMjqOK3ZWhH5A@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
is your phrase-table sorted? You can sort is like
LC_ALL=C sort big-pt > big.pt.sorted
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
On 5 May 2015 at 17:22, kwame porter robinson <k.porter.robinson@gmail.com>
wrote:
> Hmm, so I tried that and received the same exception:
>
> $ CreateOnDiskPt 1 1 1 10 2 big-pt ondiskpt
>
> Starting : [0] seconds
> terminate called after throwing an instance of 'util::Exception'
> what(): OnDiskPt/PhraseNode.cpp:97 in void
> OnDiskPt::PhraseNode::Save(OnDiskPt::OnDiskWrapper&, size_t, size_t) threw
> util::Exception because `m_saved'.
> Already saved
> Aborted (core dumped)
>
> - Kwame
>
>
> On Tue, May 5, 2015 at 12:48 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>>
>>
>> On 05/05/2015 02:46, kwame porter robinson wrote:
>>
>> Hi,
>>
>> I am attempting to binarize a large ascii phrase table using
>> CreateOnDiskPt. Each phrase table row contains the source phrase, target
>> phrase and a single score. I am getting a 'Already Saved' exception for
>> phrase tables larger than 9 lines. I am using Moses release 2.1.
>>
>> The exception is reproduced below:
>>
>> 1) Using the 10 line phrase table below*, fromhttp://www.statmt.org/moses/download/sample-models.tgz.
>>
>> 2) Truncate the phrase-table with 'head phrase-table -n9 > small-pt'
>>
>> 3) Truncate another version of the phrase-table with 'head phrase-table
>> -n10 > big-pt'
>>
>> 4) CreateOnDisk works with 'CreateOnDiskPt 0 0 1 10 2 small-pt
>> myphrasetable'
>>
>> it should be
>> CreateOnDiskPt 1 1 ....
>> The 1st two arguments are the NUMBER of source and target factors.
>>
>> 5) CreateOnDisk throws the following exception with 'CreateOnDiskPt 0 0 1
>> 10 2 big-pt myphrasetable'
>>
>> ---
>> Starting : [0] seconds
>> terminate called after throwing an instance of 'util::Exception'
>> what(): OnDiskPt/PhraseNode.cpp:97 in void
>> OnDiskPt::PhraseNode::Save(OnDiskPt::OnDiskWrapper&, size_t, size_t) threw
>> util::Exception because `m_saved'.
>> Already saved
>> Aborted (core dumped)
>> ---
>>
>> Any thoughts on how to fix this? For hints I've looked at:https://www.mail-archive.com/moses-support%40mit.edu/msg11999.html andhttps://www.mail-archive.com/moses-support%40mit.edu/msg10602.html
>> but was unable to resolve this.
>>
>> Thank you for your time.
>>
>> * The phrase table
>> ----
>> der ||| the ||| 0.3 ||| |||
>> das ||| the ||| 0.4 ||| |||
>> das ||| it ||| 0.1 ||| |||
>> das ||| this ||| 0.1 ||| |||
>> die ||| the ||| 0.3 ||| |||
>> ist ||| is ||| 1.0 ||| |||
>> ist ||| 's ||| 1.0 ||| |||
>> das ist ||| it is ||| 0.2 ||| |||
>> das ist ||| this is ||| 0.8 ||| |||
>> es ist ||| it is ||| 0.8 ||| |||
>> ---
>>
>> - Kwame
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> --
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>
>>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150505/f5b8cdb4/attachment-0001.htm
------------------------------
Message: 2
Date: Tue, 5 May 2015 10:10:58 -0400
From: kwame porter robinson <k.porter.robinson@gmail.com>
Subject: Re: [Moses-support] CeateOnDiskPt throws 'Already Saved'
Exception on phrase table in sample_models/phrase-model
To: moses-support@mit.edu
Message-ID:
<CA+RrxgW313JEpDC2z0eX+vL5iHn4vEVHHW34xNnMGBnSUx4KOA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Ah, that was my error. Thanks for pointing that out. The exception is no
longer thrown.
Quick follow up question: does CreateOnDiskPt process phrase tables lazily?
(that is, in chunks so that the entire table is not loaded into memory).
On Tue, May 5, 2015 at 9:33 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
> is your phrase-table sorted? You can sort is like
> LC_ALL=C sort big-pt > big.pt.sorted
>
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 5 May 2015 at 17:22, kwame porter robinson <k.porter.robinson@gmail.com
> > wrote:
>
>> Hmm, so I tried that and received the same exception:
>>
>> $ CreateOnDiskPt 1 1 1 10 2 big-pt ondiskpt
>>
>> Starting : [0] seconds
>> terminate called after throwing an instance of 'util::Exception'
>> what(): OnDiskPt/PhraseNode.cpp:97 in void
>> OnDiskPt::PhraseNode::Save(OnDiskPt::OnDiskWrapper&, size_t, size_t) threw
>> util::Exception because `m_saved'.
>> Already saved
>> Aborted (core dumped)
>>
>> - Kwame
>>
>>
>> On Tue, May 5, 2015 at 12:48 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>>
>>>
>>> On 05/05/2015 02:46, kwame porter robinson wrote:
>>>
>>> Hi,
>>>
>>> I am attempting to binarize a large ascii phrase table using
>>> CreateOnDiskPt. Each phrase table row contains the source phrase, target
>>> phrase and a single score. I am getting a 'Already Saved' exception for
>>> phrase tables larger than 9 lines. I am using Moses release 2.1.
>>>
>>> The exception is reproduced below:
>>>
>>> 1) Using the 10 line phrase table below*, fromhttp://www.statmt.org/moses/download/sample-models.tgz.
>>>
>>> 2) Truncate the phrase-table with 'head phrase-table -n9 > small-pt'
>>>
>>> 3) Truncate another version of the phrase-table with 'head phrase-table
>>> -n10 > big-pt'
>>>
>>> 4) CreateOnDisk works with 'CreateOnDiskPt 0 0 1 10 2 small-pt
>>> myphrasetable'
>>>
>>> it should be
>>> CreateOnDiskPt 1 1 ....
>>> The 1st two arguments are the NUMBER of source and target factors.
>>>
>>> 5) CreateOnDisk throws the following exception with 'CreateOnDiskPt 0 0 1
>>> 10 2 big-pt myphrasetable'
>>>
>>> ---
>>> Starting : [0] seconds
>>> terminate called after throwing an instance of 'util::Exception'
>>> what(): OnDiskPt/PhraseNode.cpp:97 in void
>>> OnDiskPt::PhraseNode::Save(OnDiskPt::OnDiskWrapper&, size_t, size_t) threw
>>> util::Exception because `m_saved'.
>>> Already saved
>>> Aborted (core dumped)
>>> ---
>>>
>>> Any thoughts on how to fix this? For hints I've looked at:https://www.mail-archive.com/moses-support%40mit.edu/msg11999.html andhttps://www.mail-archive.com/moses-support%40mit.edu/msg10602.html
>>> but was unable to resolve this.
>>>
>>> Thank you for your time.
>>>
>>> * The phrase table
>>> ----
>>> der ||| the ||| 0.3 ||| |||
>>> das ||| the ||| 0.4 ||| |||
>>> das ||| it ||| 0.1 ||| |||
>>> das ||| this ||| 0.1 ||| |||
>>> die ||| the ||| 0.3 ||| |||
>>> ist ||| is ||| 1.0 ||| |||
>>> ist ||| 's ||| 1.0 ||| |||
>>> das ist ||| it is ||| 0.2 ||| |||
>>> das ist ||| this is ||| 0.8 ||| |||
>>> es ist ||| it is ||| 0.8 ||| |||
>>> ---
>>>
>>> - Kwame
>>>
>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Researcher
>>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>>
>>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150505/3ac01d3c/attachment-0001.htm
------------------------------
Message: 3
Date: Tue, 5 May 2015 18:17:25 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] CeateOnDiskPt throws 'Already Saved'
Exception on phrase table in sample_models/phrase-model
To: kwame porter robinson <k.porter.robinson@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjQ7tpL+qx-1eAi3ew+CoQxhOGc2oYN34XqgTG3t7WZMw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
yes, it process a bit at a time so u don't need a big pc to run it
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
On 5 May 2015 at 18:10, kwame porter robinson <k.porter.robinson@gmail.com>
wrote:
> Ah, that was my error. Thanks for pointing that out. The exception is no
> longer thrown.
>
> Quick follow up question: does CreateOnDiskPt process phrase tables
> lazily? (that is, in chunks so that the entire table is not loaded into
> memory).
>
> On Tue, May 5, 2015 at 9:33 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
>> is your phrase-table sorted? You can sort is like
>> LC_ALL=C sort big-pt > big.pt.sorted
>>
>>
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabi
>> http://www.hoang.co.uk/hieu
>>
>> On 5 May 2015 at 17:22, kwame porter robinson <
>> k.porter.robinson@gmail.com> wrote:
>>
>>> Hmm, so I tried that and received the same exception:
>>>
>>> $ CreateOnDiskPt 1 1 1 10 2 big-pt ondiskpt
>>>
>>> Starting : [0] seconds
>>> terminate called after throwing an instance of 'util::Exception'
>>> what(): OnDiskPt/PhraseNode.cpp:97 in void
>>> OnDiskPt::PhraseNode::Save(OnDiskPt::OnDiskWrapper&, size_t, size_t) threw
>>> util::Exception because `m_saved'.
>>> Already saved
>>> Aborted (core dumped)
>>>
>>> - Kwame
>>>
>>>
>>> On Tue, May 5, 2015 at 12:48 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On 05/05/2015 02:46, kwame porter robinson wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am attempting to binarize a large ascii phrase table using
>>>> CreateOnDiskPt. Each phrase table row contains the source phrase, target
>>>> phrase and a single score. I am getting a 'Already Saved' exception for
>>>> phrase tables larger than 9 lines. I am using Moses release 2.1.
>>>>
>>>> The exception is reproduced below:
>>>>
>>>> 1) Using the 10 line phrase table below*, fromhttp://www.statmt.org/moses/download/sample-models.tgz.
>>>>
>>>> 2) Truncate the phrase-table with 'head phrase-table -n9 > small-pt'
>>>>
>>>> 3) Truncate another version of the phrase-table with 'head phrase-table
>>>> -n10 > big-pt'
>>>>
>>>> 4) CreateOnDisk works with 'CreateOnDiskPt 0 0 1 10 2 small-pt
>>>> myphrasetable'
>>>>
>>>> it should be
>>>> CreateOnDiskPt 1 1 ....
>>>> The 1st two arguments are the NUMBER of source and target factors.
>>>>
>>>> 5) CreateOnDisk throws the following exception with 'CreateOnDiskPt 0 0 1
>>>> 10 2 big-pt myphrasetable'
>>>>
>>>> ---
>>>> Starting : [0] seconds
>>>> terminate called after throwing an instance of 'util::Exception'
>>>> what(): OnDiskPt/PhraseNode.cpp:97 in void
>>>> OnDiskPt::PhraseNode::Save(OnDiskPt::OnDiskWrapper&, size_t, size_t) threw
>>>> util::Exception because `m_saved'.
>>>> Already saved
>>>> Aborted (core dumped)
>>>> ---
>>>>
>>>> Any thoughts on how to fix this? For hints I've looked at:https://www.mail-archive.com/moses-support%40mit.edu/msg11999.html andhttps://www.mail-archive.com/moses-support%40mit.edu/msg10602.html
>>>> but was unable to resolve this.
>>>>
>>>> Thank you for your time.
>>>>
>>>> * The phrase table
>>>> ----
>>>> der ||| the ||| 0.3 ||| |||
>>>> das ||| the ||| 0.4 ||| |||
>>>> das ||| it ||| 0.1 ||| |||
>>>> das ||| this ||| 0.1 ||| |||
>>>> die ||| the ||| 0.3 ||| |||
>>>> ist ||| is ||| 1.0 ||| |||
>>>> ist ||| 's ||| 1.0 ||| |||
>>>> das ist ||| it is ||| 0.2 ||| |||
>>>> das ist ||| this is ||| 0.8 ||| |||
>>>> es ist ||| it is ||| 0.8 ||| |||
>>>> ---
>>>>
>>>> - Kwame
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>> --
>>>> Hieu Hoang
>>>> Researcher
>>>> New York University, Abu Dhabihttp://www.hoang.co.uk/hieu
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150505/9f369c0e/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 103, Issue 10
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 103, Issue 10"
Post a Comment