Run globaldce periodically to collect dead objects

We give LLVM global objects ownership of the Python objects they mirror, but until recently there was no way to safely free unused global objects. Now that the ExecutionEngine supports it, this patch hooks up a mechanism.

This allows regrtest to complete under -j always after using ~1.6gb of memory.

On an instrumented test_llvm run, we get:

Number of globals collected by globaldce (n=338):
Min: 0
Median: 30
Mean: 30
Max: 101
Sum: 10346

Time for a globaldce run in ns (n=338):
Min: 462000
Median: 815500
Mean: 941295
Max: 5247000
Sum: 318158000

And for regrtest:

Number of globals collected by globaldce (n=22):
Min: 0
Median: 9
Mean: 72
Max: 290
Sum: 1588

Time for a globaldce run in ns (n=22):
Min: 726000
Median: 5039500
Mean: 10233772
Max: 34198000
Sum: 225143000

[Jeffrey Yasskin, 2009-11-03 23:06:43]

PTAL. If you have time, please try it out and tell me if anything crashes.

[Collin Winter, 2009-11-10 00:13:19]

Please document this in Python/llvm_notes.txt.

http://codereview.appspot.com/144077/diff/2001/2008
File Objects/_llvmfunctionobject.cc (left):

http://codereview.appspot.com/144077/diff/2001/2008#oldcode103
Objects/_llvmfunctionobject.cc:103: // need it again. function->empty() can be
used to test whether a function
Is there any way now to tell if the function has been cleared out?

http://codereview.appspot.com/144077/diff/2001/2008
File Objects/_llvmfunctionobject.cc (right):

http://codereview.appspot.com/144077/diff/2001/2008#newcode117
Objects/_llvmfunctionobject.cc:117: // worth it.  GlobalValues are Constants, so
this picks them up too.
80 cols

http://codereview.appspot.com/144077/diff/2001/2008#newcode129
Objects/_llvmfunctionobject.cc:129: // Build a single "call void
undef(all_constants)" instruction to
Do we have any way of verifying that LLVM isn't optimizing this away?

http://codereview.appspot.com/144077/diff/2001/2008#newcode136
Objects/_llvmfunctionobject.cc:136: const llvm::FunctionType *userT =
llvm::FunctionType::get(
Nit: userT -> user_t?

http://codereview.appspot.com/144077/diff/2001/2002
File Python/llvm_fbuilder.h (right):

http://codereview.appspot.com/144077/diff/2001/2002#newcode320
Python/llvm_fbuilder.h:320: // for PyObjects that may be immutable.
So we'd call GetGlobalFunction() on things that aren't functions?

http://codereview.appspot.com/144077/diff/2001/2011
File Util/Stats.cc (right):

http://codereview.appspot.com/144077/diff/2001/2011#newcode20
Util/Stats.cc:20: return 0;
Should this be a compile-time warning?

[Jeffrey Yasskin, 2009-11-18 21:58:59]

http://codereview.appspot.com/144077/diff/2001/2008
File Objects/_llvmfunctionobject.cc (left):

http://codereview.appspot.com/144077/diff/2001/2008#oldcode103
Objects/_llvmfunctionobject.cc:103: // need it again. function->empty() can be
used to test whether a function
On 2009/11/10 00:13:19, Collin Winter wrote:
> Is there any way now to tell if the function has been cleared out?

We could check for the body containing a single call instruction, calling undef,
but there's nothing simple.

http://codereview.appspot.com/144077/diff/2001/2008
File Objects/_llvmfunctionobject.cc (right):

http://codereview.appspot.com/144077/diff/2001/2008#newcode117
Objects/_llvmfunctionobject.cc:117: // worth it.  GlobalValues are Constants, so
this picks them up too.
On 2009/11/10 00:13:19, Collin Winter wrote:
> 80 cols

Oops.

http://codereview.appspot.com/144077/diff/2001/2008#newcode129
Objects/_llvmfunctionobject.cc:129: // Build a single "call void
undef(all_constants)" instruction to
On 2009/11/10 00:13:19, Collin Winter wrote:
> Do we have any way of verifying that LLVM isn't optimizing this away?

It would, but we don't run optimization passes except when trying to generate
code, and this would generate invalid code anyway.

I'm not sure how to catch if we change that.

http://codereview.appspot.com/144077/diff/2001/2008#newcode136
Objects/_llvmfunctionobject.cc:136: const llvm::FunctionType *userT =
llvm::FunctionType::get(
On 2009/11/10 00:13:19, Collin Winter wrote:
> Nit: userT -> user_t?

Done.

http://codereview.appspot.com/144077/diff/2001/2002
File Python/llvm_fbuilder.h (right):

http://codereview.appspot.com/144077/diff/2001/2002#newcode320
Python/llvm_fbuilder.h:320: // for PyObjects that may be immutable.
On 2009/11/10 00:13:19, Collin Winter wrote:
> So we'd call GetGlobalFunction() on things that aren't functions?

Um. I think I deleted the wrong comment in an intermediate change. Fixed.

http://codereview.appspot.com/144077/diff/2001/2011
File Util/Stats.cc (right):

http://codereview.appspot.com/144077/diff/2001/2011#newcode20
Util/Stats.cc:20: return 0;
On 2009/11/10 00:13:19, Collin Winter wrote:
> Should this be a compile-time warning?

It could be. Do we want to bug abbeyj whenever he builds the instrumented build?

[Collin Winter, 2009-11-18 22:04:42]

LGTM