Moses-support Digest, Vol 112, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Problem with processPhraseTableMin (Kenneth Heafield)


----------------------------------------------------------------------

Message: 1
Date: Thu, 4 Feb 2016 15:51:12 +0000
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Problem with processPhraseTableMin
To: moses-support@mit.edu
Message-ID: <56B37370.9010903@kheafield.com>
Content-Type: text/plain; charset=UTF-8

I can haz backtrace? It's clearly calculating a huge amount to allocate
somewhere which is leading to tcmalloc returning NULL but that's not
tcmalloc's fault.

On 02/04/2016 03:47 PM, Marcin Junczys-Dowmunt wrote:
> I've been using it with and without tcmalloc with no problems, the part
> where it crashes for is not multi-threading anyway. I guess it's the
> intel compiler, no idea why though.
>
> W dniu 2016-02-04 16:37, Jeremy Gwinnup napisa?(a):
>
>> Uli,
>>
>> I sent the phrase-table to Marcin yesterday to test - He was able to binarize the table successfully. Here, we've been compiling moses with the Intel compiler. We built the same checkout with gcc and using processPhraseTableMin from that build we were able to successfully binarize the phrase table.
>>
>> One thing I saw during testing these different configs was the intel-compiled version would output tcmalloc debug messages, but the gcc-compiled one would not. We're using tcmalloc-minimal for these builds. Should we be using the full version?
>>
>> Running moses ?version on both builds shows Boost 1.54, Xmlrpc-c 1.33.17 and CMPH (version unknown) linked in. We compile static binaries on a RHEL 6-based distro (Scientific Linux 6.7)
>>
>> -Jeremy
>>> Message: 2 Date: Thu, 4 Feb 2016 15:03:02 +0000 From: Ulrich Germann
>>> <ulrich.germann@gmail.com <mailto:ulrich.germann@gmail.com>> Subject:
>>> Re: [Moses-support] Problem with processPhraseTableMin To: Marcin
>>> Junczys-Dowmunt <junczys@amu.edu.pl <mailto:junczys@amu.edu.pl>> Cc:
>>> "moses-support@mit.edu <mailto:moses-support@mit.edu>"
>>> <moses-support@mit.edu <mailto:moses-support@mit.edu>> Message-ID:
>>> <CAHQSRUq_gtrCUBkzwMZpVMKYPORmyGsE4sW-4rYBS_jzML1tWA@mail.gmail.com
>>> <mailto:CAHQSRUq_gtrCUBkzwMZpVMKYPORmyGsE4sW-4rYBS_jzML1tWA@mail.gmail.com>>
>>> Content-Type: text/plain; charset="utf-8" I've had
>>> processPhraseTableMin crash when the phrase table contains duplicate
>>> entries (can't remember if there was an unreasonable memory
>>> allocation involved). Is Marcin using the exact same phrase table?
>>> Can you check if the phrase table has duplicate entries? To crash or
>>> not to crash could also depend on OS and libraries used. You can get
>>> the versions of libraries compiled into moses with moses --version
>>> I've had duplicate entries in the phrase table after running
>>> ptable-sigtest-filter, which is Marcin's implementation of Johnson et
>>> al.'s significance filtering that I pulled in from his WIPO branch;
>>> compile with --with-mm --with-mm-extras to get it compiled. - Uli
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 112, Issue 14
**********************************************

Tweet