code review 119550043: runtime: implement transfer cache

runtime: implement transfer cache
Currently we do the following dance after sweeping a span:
1. lock mcentral
2. remove the span from a list
3. unlock mcentral
4. unmark span
5. lock mheap
6. insert the span into heap
7. unlock mheap
8. lock mcentral
9. observe empty list
10. unlock mcentral
11. lock mheap
12. grab the span
13. unlock mheap
14. mark span
15. lock mcentral
16. insert the span into empty list
17. unlock mcentral

This change short-circuits this sequence to nothing,
that is, we just cache and use the span after sweeping.

This gives us functionality similar (even better) to tcmalloc's transfer cache.

benchmark            old ns/op     new ns/op     delta
BenchmarkMalloc8     22.2          19.5          -12.16%
BenchmarkMalloc16    31.0          26.6          -14.19%

[dvyukov, 2014-08-07 17:36:05]

Hello golang-codereviews@googlegroups.com (cc: khr@golang.org, rlh@golang.org,
rsc@golang.org),

I'd like you to review this change to
https://dvyukov%40google.com@code.google.com/p/go/

[dvyukov, 2014-08-12 20:35:17]

ping

[khr, 2014-08-17 23:27:32]

On 2014/08/12 20:35:17, dvyukov wrote:
> ping

LGTM.

[khr, 2014-08-17 23:27:43]

https://codereview.appspot.com/119550043/diff/80001/src/pkg/runtime/mcentral.c
File src/pkg/runtime/mcentral.c (right):

https://codereview.appspot.com/119550043/diff/80001/src/pkg/runtime/mcentral....
src/pkg/runtime/mcentral.c:70: // the span could be moved to nonempty or heap,
retry
Don't we have to do something with s here?
At least the comment needs to be fixed, the span in never nonempty here.

https://codereview.appspot.com/119550043/diff/80001/src/pkg/runtime/mcentral....
src/pkg/runtime/mcentral.c:129: // the latest generation.
Comment on what preserve means here.

[dvyukov, 2014-08-18 12:52:17]

https://codereview.appspot.com/119550043/diff/80001/src/pkg/runtime/mcentral.c
File src/pkg/runtime/mcentral.c (right):

https://codereview.appspot.com/119550043/diff/80001/src/pkg/runtime/mcentral....
src/pkg/runtime/mcentral.c:70: // the span could be moved to nonempty or heap,
retry
On 2014/08/17 23:27:43, khr wrote:
> Don't we have to do something with s here?
> At least the comment needs to be fixed, the span in never nonempty here.

s is already in empty list, so we don't need to do anything with s here
updated the comment

https://codereview.appspot.com/119550043/diff/80001/src/pkg/runtime/mcentral....
src/pkg/runtime/mcentral.c:129: // the latest generation.
On 2014/08/17 23:27:43, khr wrote:
> Comment on what preserve means here.

Done.

[dvyukov, 2014-08-18 12:52:36]

*** Submitted as https://code.google.com/p/go/source/detail?r=b86ee06ef235 ***

runtime: implement transfer cache
Currently we do the following dance after sweeping a span:
1. lock mcentral
2. remove the span from a list
3. unlock mcentral
4. unmark span
5. lock mheap
6. insert the span into heap
7. unlock mheap
8. lock mcentral
9. observe empty list
10. unlock mcentral
11. lock mheap
12. grab the span
13. unlock mheap
14. mark span
15. lock mcentral
16. insert the span into empty list
17. unlock mcentral

This change short-circuits this sequence to nothing,
that is, we just cache and use the span after sweeping.

This gives us functionality similar (even better) to tcmalloc's transfer cache.

benchmark            old ns/op     new ns/op     delta
BenchmarkMalloc8     22.2          19.5          -12.16%
BenchmarkMalloc16    31.0          26.6          -14.19%

LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, rlh, rsc
https://codereview.appspot.com/119550043