code review 7579044: runtime: integrated network poller for linux

runtime: integrated network poller for linux
vs tip:
BenchmarkTCP4OneShot                    172994        40485  -76.60%
BenchmarkTCP4OneShot-2                   96581        30028  -68.91%
BenchmarkTCP4OneShot-4                   52615        18454  -64.93%
BenchmarkTCP4OneShot-8                   26351        12289  -53.36%
BenchmarkTCP4OneShot-16                  12258        16093  +31.29%
BenchmarkTCP4OneShot-32                  13200        17045  +29.13%

BenchmarkTCP4OneShotTimeout             124814        42932  -65.60%
BenchmarkTCP4OneShotTimeout-2            99090        29040  -70.69%
BenchmarkTCP4OneShotTimeout-4            51860        18455  -64.41%
BenchmarkTCP4OneShotTimeout-8            26100        12073  -53.74%
BenchmarkTCP4OneShotTimeout-16           12198        16654  +36.53%
BenchmarkTCP4OneShotTimeout-32           13438        17143  +27.57%

BenchmarkTCP4Persistent                 115647         7782  -93.27%
BenchmarkTCP4Persistent-2                58024         4808  -91.71%
BenchmarkTCP4Persistent-4                24715         3674  -85.13%
BenchmarkTCP4Persistent-8                16431         2407  -85.35%
BenchmarkTCP4Persistent-16                2336         1875  -19.73%
BenchmarkTCP4Persistent-32                1689         1637   -3.08%

BenchmarkTCP4PersistentTimeout           79754         7859  -90.15%
BenchmarkTCP4PersistentTimeout-2         57708         5952  -89.69%
BenchmarkTCP4PersistentTimeout-4         26907         3823  -85.79%
BenchmarkTCP4PersistentTimeout-8         15036         2567  -82.93%
BenchmarkTCP4PersistentTimeout-16         2507         1903  -24.09%
BenchmarkTCP4PersistentTimeout-32         1717         1627   -5.24%

vs old scheduler:
benchmark                           old ns/op    new ns/op    delta
BenchmarkTCPOneShot                    192244        40485  -78.94%
BenchmarkTCPOneShot-2                   63835        30028  -52.96%
BenchmarkTCPOneShot-4                   35443        18454  -47.93%
BenchmarkTCPOneShot-8                   22140        12289  -44.49%
BenchmarkTCPOneShot-16                  16930        16093   -4.94%
BenchmarkTCPOneShot-32                  16719        17045   +1.95%

BenchmarkTCPOneShotTimeout             190495        42932  -77.46%
BenchmarkTCPOneShotTimeout-2            64828        29040  -55.20%
BenchmarkTCPOneShotTimeout-4            34591        18455  -46.65%
BenchmarkTCPOneShotTimeout-8            21989        12073  -45.10%
BenchmarkTCPOneShotTimeout-16           16848        16654   -1.15%
BenchmarkTCPOneShotTimeout-32           16796        17143   +2.07%

BenchmarkTCPPersistent                  81670         7782  -90.47%
BenchmarkTCPPersistent-2                26598         4808  -81.92%
BenchmarkTCPPersistent-4                15633         3674  -76.50%
BenchmarkTCPPersistent-8                18093         2407  -86.70%
BenchmarkTCPPersistent-16               17472         1875  -89.27%
BenchmarkTCPPersistent-32                7679         1637  -78.68%

BenchmarkTCPPersistentTimeout           83186         7859  -90.55%
BenchmarkTCPPersistentTimeout-2         26883         5952  -77.86%
BenchmarkTCPPersistentTimeout-4         15776         3823  -75.77%
BenchmarkTCPPersistentTimeout-8         18180         2567  -85.88%
BenchmarkTCPPersistentTimeout-16        17454         1903  -89.10%
BenchmarkTCPPersistentTimeout-32         7798         1627  -79.14%

[dvyukov, 2013-03-12 20:26:41]

Hello golang-dev@googlegroups.com,

I'd like you to review this change to
https://dvyukov%40google.com@code.google.com/p/go/

[dvyukov, 2013-03-12 20:28:19]

On 2013/03/12 20:26:41, dvyukov wrote:
> Hello mailto:golang-dev@googlegroups.com,
> 
> I'd like you to review this change to
> https://dvyukov%2540google.com%40code.google.com/p/go/


I've not yet figured out how to regenerate defs_linux_386.h (seems to require
kernel header).
And it depends on 7569043: runtime: integrated network poller for darwin
But other than that ready for review

[iant, 2013-03-12 21:27:35]

LGTM

[bradfitz, 2013-03-12 21:34:17]

https://codereview.appspot.com/7579044/diff/20001/src/pkg/runtime/netpoll_lin...
File src/pkg/runtime/netpoll_linux.c (right):

https://codereview.appspot.com/7579044/diff/20001/src/pkg/runtime/netpoll_lin...
src/pkg/runtime/netpoll_linux.c:23: // TODO(dvyukov): set CLOEXEC
I think (on an old enough kernel), the existing tests would fail (the os/exec fd
leak tests) without this. Could you try importing "net" there, commenting out
epollcreate1, and see if it fails?

[dave_cheney.net, 2013-03-12 22:47:05]

> I've not yet figured out how to regenerate defs_linux_386.h (seems to require
> kernel header).

I had a quick attempt at this on an i386 host ... it wasn't successful.

[dvyukov, 2013-03-13 12:40:11]

On 2013/03/12 21:34:17, bradfitz wrote:
>
https://codereview.appspot.com/7579044/diff/20001/src/pkg/runtime/netpoll_lin...
> File src/pkg/runtime/netpoll_linux.c (right):
> 
>
https://codereview.appspot.com/7579044/diff/20001/src/pkg/runtime/netpoll_lin...
> src/pkg/runtime/netpoll_linux.c:23: // TODO(dvyukov): set CLOEXEC
> I think (on an old enough kernel), the existing tests would fail (the os/exec
fd
> leak tests) without this. Could you try importing "net" there, commenting out
> epollcreate1, and see if it fails?

Yes, it fails:
--- FAIL: TestExtraFiles (0.06 seconds)
	exec_test.go:239: Run: exit status 1; stdout "leaked parent file. fd = 5; want
4

[dvyukov, 2013-03-13 14:50:04]

Note to myself: fix build tags, handle arm

[dvyukov, 2013-03-13 15:20:16]

Some cool numbers:

benchmark                            old ns/op    new ns/op    delta
BenchmarkTCP4OneShot                    172994        40485  -76.60%
BenchmarkTCP4OneShot-2                   96581        30028  -68.91%
BenchmarkTCP4OneShot-4                   52615        18454  -64.93%
BenchmarkTCP4OneShot-8                   26351        12289  -53.36%
BenchmarkTCP4OneShot-16                  12258        16093  +31.29%
BenchmarkTCP4OneShot-32                  13200        17045  +29.13%
BenchmarkTCP4OneShotTimeout             124814        42932  -65.60%
BenchmarkTCP4OneShotTimeout-2            99090        29040  -70.69%
BenchmarkTCP4OneShotTimeout-4            51860        18455  -64.41%
BenchmarkTCP4OneShotTimeout-8            26100        12073  -53.74%
BenchmarkTCP4OneShotTimeout-16           12198        16654  +36.53%
BenchmarkTCP4OneShotTimeout-32           13438        17143  +27.57%
BenchmarkTCP4Persistent                 115647         7782  -93.27%
BenchmarkTCP4Persistent-2                58024         4808  -91.71%
BenchmarkTCP4Persistent-4                24715         3674  -85.13%
BenchmarkTCP4Persistent-8                16431         2407  -85.35%
BenchmarkTCP4Persistent-16                2336         1875  -19.73%
BenchmarkTCP4Persistent-32                1689         1637   -3.08%
BenchmarkTCP4PersistentTimeout           79754         7859  -90.15%
BenchmarkTCP4PersistentTimeout-2         57708         5952  -89.69%
BenchmarkTCP4PersistentTimeout-4         26907         3823  -85.79%
BenchmarkTCP4PersistentTimeout-8         15036         2567  -82.93%
BenchmarkTCP4PersistentTimeout-16         2507         1903  -24.09%
BenchmarkTCP4PersistentTimeout-32         1717         1627   -5.24%


I have not looked at OneShot with GOMAXPROCS=16,32, but it establishes and
closes connections at insanely high race. Perhaps the slowdown is because old
impl does not bother to arm epoll if we connect->read->close, but new impl
always arms and disarms epoll.

[rsc, 2013-03-13 16:54:37]

LGTM

[dvyukov, 2013-03-14 07:27:39]

Updated build tags
Now it is only for linux,386 linux,amd64

[dvyukov, 2013-03-14 15:06:46]

*** Submitted as https://code.google.com/p/go/source/detail?r=f9c03cd9bd84 ***

runtime: integrated network poller for linux
vs tip:
BenchmarkTCP4OneShot                    172994        40485  -76.60%
BenchmarkTCP4OneShot-2                   96581        30028  -68.91%
BenchmarkTCP4OneShot-4                   52615        18454  -64.93%
BenchmarkTCP4OneShot-8                   26351        12289  -53.36%
BenchmarkTCP4OneShot-16                  12258        16093  +31.29%
BenchmarkTCP4OneShot-32                  13200        17045  +29.13%

BenchmarkTCP4OneShotTimeout             124814        42932  -65.60%
BenchmarkTCP4OneShotTimeout-2            99090        29040  -70.69%
BenchmarkTCP4OneShotTimeout-4            51860        18455  -64.41%
BenchmarkTCP4OneShotTimeout-8            26100        12073  -53.74%
BenchmarkTCP4OneShotTimeout-16           12198        16654  +36.53%
BenchmarkTCP4OneShotTimeout-32           13438        17143  +27.57%

BenchmarkTCP4Persistent                 115647         7782  -93.27%
BenchmarkTCP4Persistent-2                58024         4808  -91.71%
BenchmarkTCP4Persistent-4                24715         3674  -85.13%
BenchmarkTCP4Persistent-8                16431         2407  -85.35%
BenchmarkTCP4Persistent-16                2336         1875  -19.73%
BenchmarkTCP4Persistent-32                1689         1637   -3.08%

BenchmarkTCP4PersistentTimeout           79754         7859  -90.15%
BenchmarkTCP4PersistentTimeout-2         57708         5952  -89.69%
BenchmarkTCP4PersistentTimeout-4         26907         3823  -85.79%
BenchmarkTCP4PersistentTimeout-8         15036         2567  -82.93%
BenchmarkTCP4PersistentTimeout-16         2507         1903  -24.09%
BenchmarkTCP4PersistentTimeout-32         1717         1627   -5.24%

vs old scheduler:
benchmark                           old ns/op    new ns/op    delta
BenchmarkTCPOneShot                    192244        40485  -78.94%
BenchmarkTCPOneShot-2                   63835        30028  -52.96%
BenchmarkTCPOneShot-4                   35443        18454  -47.93%
BenchmarkTCPOneShot-8                   22140        12289  -44.49%
BenchmarkTCPOneShot-16                  16930        16093   -4.94%
BenchmarkTCPOneShot-32                  16719        17045   +1.95%

BenchmarkTCPOneShotTimeout             190495        42932  -77.46%
BenchmarkTCPOneShotTimeout-2            64828        29040  -55.20%
BenchmarkTCPOneShotTimeout-4            34591        18455  -46.65%
BenchmarkTCPOneShotTimeout-8            21989        12073  -45.10%
BenchmarkTCPOneShotTimeout-16           16848        16654   -1.15%
BenchmarkTCPOneShotTimeout-32           16796        17143   +2.07%

BenchmarkTCPPersistent                  81670         7782  -90.47%
BenchmarkTCPPersistent-2                26598         4808  -81.92%
BenchmarkTCPPersistent-4                15633         3674  -76.50%
BenchmarkTCPPersistent-8                18093         2407  -86.70%
BenchmarkTCPPersistent-16               17472         1875  -89.27%
BenchmarkTCPPersistent-32                7679         1637  -78.68%

BenchmarkTCPPersistentTimeout           83186         7859  -90.55%
BenchmarkTCPPersistentTimeout-2         26883         5952  -77.86%
BenchmarkTCPPersistentTimeout-4         15776         3823  -75.77%
BenchmarkTCPPersistentTimeout-8         18180         2567  -85.88%
BenchmarkTCPPersistentTimeout-16        17454         1903  -89.10%
BenchmarkTCPPersistentTimeout-32         7798         1627  -79.14%

R=golang-dev, iant, bradfitz, dave, rsc
CC=golang-dev
https://codereview.appspot.com/7579044

[bradfitz, 2013-03-14 15:28:30]

Yay!
 On Mar 14, 2013 8:06 AM, <dvyukov@google.com> wrote:

> *** Submitted as
>
https://code.google.com/p/go/**source/detail?r=f9c03cd9bd84<https://code.goog...
>
> runtime: integrated network poller for linux
> vs tip:
> BenchmarkTCP4OneShot                    172994        40485  -76.60%
> BenchmarkTCP4OneShot-2                   96581        30028  -68.91%
> BenchmarkTCP4OneShot-4                   52615        18454  -64.93%
> BenchmarkTCP4OneShot-8                   26351        12289  -53.36%
> BenchmarkTCP4OneShot-16                  12258        16093  +31.29%
> BenchmarkTCP4OneShot-32                  13200        17045  +29.13%
>
> BenchmarkTCP4OneShotTimeout             124814        42932  -65.60%
> BenchmarkTCP4OneShotTimeout-2            99090        29040  -70.69%
> BenchmarkTCP4OneShotTimeout-4            51860        18455  -64.41%
> BenchmarkTCP4OneShotTimeout-8            26100        12073  -53.74%
> BenchmarkTCP4OneShotTimeout-16           12198        16654  +36.53%
> BenchmarkTCP4OneShotTimeout-32           13438        17143  +27.57%
>
> BenchmarkTCP4Persistent                 115647         7782  -93.27%
> BenchmarkTCP4Persistent-2                58024         4808  -91.71%
> BenchmarkTCP4Persistent-4                24715         3674  -85.13%
> BenchmarkTCP4Persistent-8                16431         2407  -85.35%
> BenchmarkTCP4Persistent-16                2336         1875  -19.73%
> BenchmarkTCP4Persistent-32                1689         1637   -3.08%
>
> BenchmarkTCP4PersistentTimeout           79754         7859  -90.15%
> BenchmarkTCP4PersistentTimeout**-2         57708         5952  -89.69%
> BenchmarkTCP4PersistentTimeout**-4         26907         3823  -85.79%
> BenchmarkTCP4PersistentTimeout**-8         15036         2567  -82.93%
> BenchmarkTCP4PersistentTimeout**-16         2507         1903  -24.09%
> BenchmarkTCP4PersistentTimeout**-32         1717         1627   -5.24%
>
> vs old scheduler:
> benchmark                           old ns/op    new ns/op    delta
> BenchmarkTCPOneShot                    192244        40485  -78.94%
> BenchmarkTCPOneShot-2                   63835        30028  -52.96%
> BenchmarkTCPOneShot-4                   35443        18454  -47.93%
> BenchmarkTCPOneShot-8                   22140        12289  -44.49%
> BenchmarkTCPOneShot-16                  16930        16093   -4.94%
> BenchmarkTCPOneShot-32                  16719        17045   +1.95%
>
> BenchmarkTCPOneShotTimeout             190495        42932  -77.46%
> BenchmarkTCPOneShotTimeout-2            64828        29040  -55.20%
> BenchmarkTCPOneShotTimeout-4            34591        18455  -46.65%
> BenchmarkTCPOneShotTimeout-8            21989        12073  -45.10%
> BenchmarkTCPOneShotTimeout-16           16848        16654   -1.15%
> BenchmarkTCPOneShotTimeout-32           16796        17143   +2.07%
>
> BenchmarkTCPPersistent                  81670         7782  -90.47%
> BenchmarkTCPPersistent-2                26598         4808  -81.92%
> BenchmarkTCPPersistent-4                15633         3674  -76.50%
> BenchmarkTCPPersistent-8                18093         2407  -86.70%
> BenchmarkTCPPersistent-16               17472         1875  -89.27%
> BenchmarkTCPPersistent-32                7679         1637  -78.68%
>
> BenchmarkTCPPersistentTimeout           83186         7859  -90.55%
> BenchmarkTCPPersistentTimeout-**2         26883         5952  -77.86%
> BenchmarkTCPPersistentTimeout-**4         15776         3823  -75.77%
> BenchmarkTCPPersistentTimeout-**8         18180         2567  -85.88%
> BenchmarkTCPPersistentTimeout-**16        17454         1903  -89.10%
> BenchmarkTCPPersistentTimeout-**32         7798         1627  -79.14%
>
> R=golang-dev, iant, bradfitz, dave, rsc
> CC=golang-dev
>
https://codereview.appspot.**com/7579044<https://codereview.appspot.com/7579044>
>
>
>
https://codereview.appspot.**com/7579044/<https://codereview.appspot.com/7579...
>

[albert.strasheim, 2013-03-16 00:32:15]

On 2013/03/14 15:06:46, dvyukov wrote:
> *** Submitted as https://code.google.com/p/go/source/detail?r=f9c03cd9bd84 ***
> runtime: integrated network poller for linux

Howdy

I'm seeing some kind of regression related to UNIX socket pairs and
net.FileConn.

I have a test that makes a socket pair with syscall.Socketpair and then
exchanges another socket pair over the first pair.

When I call net.FileConn on the received socket I get a syscall.Errno with a
value of -EEXIST (note the minus).

The simplest test I wrote doesn't reproduce the problem, so I have to go dig a
test out of a larger piece of code.

Posting now, in case it rings a bell.

Cheers

Albert

[albert.strasheim, 2013-03-16 01:04:40]

On 2013/03/16 00:32:15, albert.strasheim wrote:
> On 2013/03/14 15:06:46, dvyukov wrote:
> > *** Submitted as https://code.google.com/p/go/source/detail?r=f9c03cd9bd84
***
> > runtime: integrated network poller for linux
> I'm seeing some kind of regression related to UNIX socket pairs and
> net.FileConn.
> I have a test that makes a socket pair with syscall.Socketpair and then
> exchanges another socket pair over the first pair.
> When I call net.FileConn on the received socket I get a syscall.Errno with a
> value of -EEXIST (note the minus).

https://code.google.com/p/go/issues/detail?id=5061