code review 9776044: runtime: fix CPU underutilization

runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes issue 5586.

[dvyukov, 2013-06-05 20:43:08]

Hello golang-dev@googlegroups.com,

I'd like you to review this change to
https://dvyukov%40google.com@code.google.com/p/go/

[gobot, 2013-06-14 14:27:01]

R=rsc (assigned by dvyukov)

[rsc, 2013-06-21 19:39:29]

LGTM

"spinning" means that the M is waiting for a G, right?
If so, can we perhaps s/spinning/gwait/ in a separate CL?
spinning has a different meaning in lock implementations, and I always think of
that one.

[rsc, 2013-06-21 19:56:19]

I should not have reviewed this. My mistake. Do not submit until the
runtime CL moratorium is over.

[dvyukov, 2013-06-27 16:51:42]

On Fri, Jun 21, 2013 at 11:39 PM,  <rsc@golang.org> wrote:
> LGTM
>
> "spinning" means that the M is waiting for a G, right?
> If so, can we perhaps s/spinning/gwait/ in a separate CL?
> spinning has a different meaning in lock implementations, and I always
> think of that one.

This is not exactly "gwait" (as it was in the old scheduler). It means
that the M itself is actively look for new work, other M's can *not*
handoff new work to this M.
This is actually very close to the "spinning" in mutex implementations:

// mutex:
try_fast_acquire(mu);
mark_as_has_spinning(mu);
for(;;) {
  sleep(...);
  if(try_acqurie(mu)) {
    unmark_as_has_spinning(mu);
    break;
 }
}

// scheduler:
try_get_g_from_local_queue();
m->spinning = true;
for(...) {
  if(try_to_steal_from_another_P()) {
    m->spinning = false;
    break;
 }
}

[dvyukov, 2013-06-27 16:52:32]

*** Submitted as https://code.google.com/p/go/source/detail?r=1e280889f997 ***

runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes issue 5586.

R=rsc
CC=gobot, golang-dev
https://codereview.appspot.com/9776044

[remyoudompheng, 2013-07-20 20:11:14]

R=close