code review 4446062: runtime: more graceful out-of-memory crash

runtime: more graceful out-of-memory crash

Used to fault trying to access l->list->next
when l->list == nil after MCentral_AllocList.
Now prints

runtime: out of memory: no room in arena for 65536-byte allocation (536870912 in use)
throw: out of memory

followed by stack trace.

Fixes issue 1650.

[rsc, 2011-04-26 04:50:38]

Hello r (cc: golang-dev@googlegroups.com),

I'd like you to review this change to
https://go.googlecode.com/hg/

[dave_cheney.net, 2011-04-26 05:10:40]

Thanks. I was using this as a stop gap.


diff -r a77eea2b52bd src/pkg/runtime/mcentral.c
--- a/src/pkg/runtime/mcentral.c	Tue Apr 26 00:57:03 2011 -0400
+++ b/src/pkg/runtime/mcentral.c	Tue Apr 26 01:09:44 2011 -0400
@@ -176,7 +176,7 @@
 	s = runtime·MHeap_Alloc(&runtime·mheap, npages, c->sizeclass, 0);
 	if(s == nil) {
 		// TODO(rsc): Log out of memory
-		runtime·lock(c);
+		runtime·throw("Heap exhausted");
 		return false;
 	}

90% of the time the godoc + ab test would cause this runtime error to be thrown.

[r, 2011-04-26 07:13:23]

LGTM

[rsc, 2011-04-26 12:25:48]

*** Submitted as http://code.google.com/p/go/source/detail?r=ebb5a2413cf7 ***

runtime: more graceful out-of-memory crash

Used to fault trying to access l->list->next
when l->list == nil after MCentral_AllocList.
Now prints

runtime: out of memory: no room in arena for 65536-byte allocation (536870912 in
use)
throw: out of memory

followed by stack trace.

Fixes issue 1650.

R=r, dfc
CC=golang-dev
http://codereview.appspot.com/4446062

[fiber, 2011-04-26 13:20:17]

Looking back at mem.go I was wondering how that uses 500+ M in the
first place. If I do get this correctly 32 bit (8g) apps are limited
to 500M currently. What are the longer-term perspectives for this?

-Sven

2011/4/26  <rsc@golang.org>:
> *** Submitted as
> http://code.google.com/p/go/source/detail?r=ebb5a2413cf7 ***
>
> runtime: more graceful out-of-memory crash
>
> Used to fault trying to access l->list->next
> when l->list == nil after MCentral_AllocList.
> Now prints
>
> runtime: out of memory: no room in arena for 65536-byte allocation
> (536870912 in use)
> throw: out of memory
>
> followed by stack trace.
>
> Fixes issue 1650.
>
> R=r, dfc
> CC=golang-dev
> http://codereview.appspot.com/4446062
>
>
> http://codereview.appspot.com/4446062/
>

[rsc, 2011-04-26 13:36:53]

On Tue, Apr 26, 2011 at 09:19, Sven Engelhardt <sven@engelhardt.to> wrote:
> Looking back at mem.go I was wondering how that uses 500+ M in the
> first place.

I think that the continual reallocations of the slice by each
goroutine are causing significant fragmentation, and since
the 10 goroutines are all running out of sync there is never
a time when the garbage collection can reclaim significant
chunks and coalesce them.  The program does almost nothing
but allocate memory.  It's going to run out no matter how
much you give it, at least until there is a compacting collector
(not planned).

> If I do get this correctly 32 bit (8g) apps are limited
> to 500M currently. What are the longer-term perspectives for this?

Thanks for asking.  It looks like that's a bug.  I had intended
that the heap grow up to 2 GB (and then you buy a 64-bit processor).
The test for out of memory is not in the right place.

Russ

[fiber, 2011-04-26 17:43:08]

Thanks for the swift response.

mem.go doesnt actually run out of memory on 64 bit for me, but rather
peaks somewhere around 800-900M. At least I can run it there for
several for minutes.

I guess I was merely looking for some hint or workaround, as I do have
some more useful programs that run into actual memory issues like this
one due to some high memory consumption during the initialization
phase, while the garbage collector doesn't really seem to be bringing
much relief (the same program that goes out of memory half-way during
initialization on 32 bit does not even reach 100M memory consumption
on 64bit).

Upping the limit to 2G may help for now. I'd yet argue that putting
this kind of hard limits is not really a good choice for a systems
programming language.

Anyway, I really enjoy the Go experience. Thanks to all for putting
some coolness back into compiled languages.

-Sven

2011/4/26 Russ Cox <rsc@golang.org>:
> On Tue, Apr 26, 2011 at 09:19, Sven Engelhardt <sven@engelhardt.to> wrote:
>> Looking back at mem.go I was wondering how that uses 500+ M in the
>> first place.
>
> I think that the continual reallocations of the slice by each
> goroutine are causing significant fragmentation, and since
> the 10 goroutines are all running out of sync there is never
> a time when the garbage collection can reclaim significant
> chunks and coalesce them.  The program does almost nothing
> but allocate memory.  It's going to run out no matter how
> much you give it, at least until there is a compacting collector
> (not planned).
>
>> If I do get this correctly 32 bit (8g) apps are limited
>> to 500M currently. What are the longer-term perspectives for this?
>
> Thanks for asking.  It looks like that's a bug.  I had intended
> that the heap grow up to 2 GB (and then you buy a 64-bit processor).
> The test for out of memory is not in the right place.
>
> Russ

[rsc, 2011-04-26 17:46:14]

> Upping the limit to 2G may help for now. I'd yet argue that putting
> this kind of hard limits is not really a good choice for a systems
> programming language.

As always there are tradeoffs.  2G means having 256 MB of
tracking bitmaps the way the current allocator works.
Going all the way up to 4G means having 512 MB of tracking.
That seemed unlikely to pay off in most cases.

The real fix is to use a 64-bit machine.

Russ