code review 5488057: math: regularize build

math: regularize build

This will be nicer to the automatic tools.
It requires a few more assembly stubs
but fewer Go files.

There are a few instances where it looks like
there are new blobs of code, but they are just
being copied out of deleted files.

There is no new code here.

Suppose you have a portable implementation for Sin
and a 386-specific assembly one.  The old way to
do this was to write three files

sin_decl.go
   func Sin(x float64) float64  // declaration only
sin_386.s
   assembly implementation

sin_port.go
   func Sin(x float64) float64 { ... }  // pure-Go impl

and then link in either sin_decl.go+sin_386.s or
just sin_port.go.  The Makefile actually did the magic
of linking in only the _port.go files for those without
assembly and only the _decl.go files for those with
assembly, or at least some of that magic.

The biggest problem with this, beyond being hard
to explain to the build system, is that once you do
explain it to the build system, godoc knows which
of sin_port.go or sin_decl.go are involved on a given
architecture, and it (correctly) ignores the other.
That means you have to put identical doc comments
in both files.

The new approach, which is more like what we did
in the later packages math/big and sync/atomic,
is to have

sin.go
   func Sin(x float64) float64  // decl only
   func sin(x float64) float64 {...}  // pure-Go impl

sin_386.s
   // assembly for Sin (ignores sin)
sin_amd64.s
   // assembly for Sin: jmp sin
sin_arm.s
   // assembly for Sin: jmp sin

Once we abandon Makefiles we can put all the assembly
stubs in one source file, so the number of files will
actually go down.

Chris asked whether the branches cost anything.
Given that they are branching to pure-Go implementations
that are not typically known for their speed, the single
direct branch is not going to be noticeable.  That is,
it's on the slow path.

An alternative would have been to preserve the old
"only write assembly files when there's an implementation"
and still have just one copy of the declaration of Sin
(and thus one doc comment) by doing:

sin.go
   func Sin(x float64) float64 { return sin(x) }

sin_decl.go
   func sin(x float64) float64 // declaration only
sin_386.s
   // assembly for sin

sin_port.go
   func sin(x float64) float64 { portable code }

In this version everyone would link in sin.go and
then either sin_decl.go+sin_386.s or sin_port.go.

This has an extra function call on all paths, including
the "fast path" to get to assembly, and it triples the
number of Go files involved compared to what I did
in this CL.  On the other hand you don't have to
write assembly stubs.  After starting down this path
I decided that the assembly stubs were the easier
approach.

As for generating the assembly stubs on the fly, much
of the goal here is to eliminate magic from the build
process, so that zero-configuration tools like goinstall
or the new go tool can handle this package.

[rsc, 2011-12-13 04:50:25]

Hello golang-dev@googlegroups.com,

I'd like you to review this change to
https://go.googlecode.com/hg/

[r2, 2011-12-13 06:30:45]

Can you please explain what you did, and why?

-rob

[cw, 2011-12-13 06:38:12]

I like the consistency of the filenames with this, but I wonder do the
introduced branches cost anything in practical performance? (My guess is on x86
it will be below the noise threshold, is that true for A8 arm cores as well?)

Can we  auto-generate the asm stubs that contain only a single branch/jmp?  It
seems we know the filenames, the functions and can check if there is an actual
implementation (filename_arch.s) and generate these on the fly

[iant, 2011-12-13 06:40:46]

LGTM

[rsc, 2011-12-13 18:17:23]

On Tue, Dec 13, 2011 at 01:30, Rob 'Commander' Pike <r@google.com> wrote:
> Can you please explain what you did, and why?

Suppose you have a portable implementation for Sin
and a 386-specific assembly one.  The old way to
do this was to write three files

sin_decl.go
    func Sin(x float64) float64  // declaration only
sin_386.s
    assembly implementation

sin_port.go
    func Sin(x float64) float64 { ... }  // pure-Go impl

and then link in either sin_decl.go+sin_386.s or
just sin_port.go.  The Makefile actually did the magic
of linking in only the _port.go files for those without
assembly and only the _decl.go files for those with
assembly, or at least some of that magic.

The biggest problem with this, beyond being hard
to explain to the build system, is that once you do
explain it to the build system, godoc knows which
of sin_port.go or sin_decl.go are involved on a given
architecture, and it (correctly) ignores the other.
That means you have to put identical doc comments
in both files.

The new approach, which is more like what we did
in the later packages math/big and sync/atomic,
is to have

sin.go
    func Sin(x float64) float64  // decl only
    func sin(x float64) float64 {...}  // pure-Go impl

sin_386.s
    // assembly for Sin (ignores sin)
sin_amd64.s
    // assembly for Sin: jmp sin
sin_arm.s
    // assembly for Sin: jmp sin

Once we abandon Makefiles we can put all the assembly
stubs in one source file, so the number of files will
actually go down.

Chris asked whether the branches cost anything.
Given that they are branching to pure-Go implementations
that are not typically known for their speed, the single
direct branch is not going to be noticeable.  That is,
it's on the slow path.

An alternative would have been to preserve the old
"only write assembly files when there's an implementation"
and still have just one copy of the declaration of Sin
(and thus one doc comment) by doing:

sin.go
    func Sin(x float64) float64 { return sin(x) }

sin_decl.go
    func sin(x float64) float64 // declaration only
sin_386.s
    // assembly for sin

sin_port.go
    func sin(x float64) float64 { portable code }

In this version everyone would link in sin.go and
then either sin_decl.go+sin_386.s or sin_port.go.

This has an extra function call on all paths, including
the "fast path" to get to assembly, and it triples the
number of Go files involved compared to what I did
in this CL.  On the other hand you don't have to
write assembly stubs.  After starting down this path
I decided that the assembly stubs were the easier
approach.

As for generating the assembly stubs on the fly, much
of the goal here is to eliminate magic from the build
process, so that zero-configuration tools like goinstall
or the new go tool can handle this package.

Russ

[r2, 2011-12-13 18:19:05]

thanks. can you put this text into the CL please? 

-rob

[rsc, 2011-12-13 18:22:12]

On Tue, Dec 13, 2011 at 13:19, Rob 'Commander' Pike <r@google.com> wrote:
> thanks. can you put this text into the CL please?

done

[r, 2011-12-13 18:22:33]

LGTM

[rsc, 2011-12-13 20:20:14]

*** Submitted as http://code.google.com/p/go/source/detail?r=af839530a31e ***

math: regularize build

This will be nicer to the automatic tools.
It requires a few more assembly stubs
but fewer Go files.

There are a few instances where it looks like
there are new blobs of code, but they are just
being copied out of deleted files.

There is no new code here.

Suppose you have a portable implementation for Sin
and a 386-specific assembly one.  The old way to
do this was to write three files

sin_decl.go
   func Sin(x float64) float64  // declaration only
sin_386.s
   assembly implementation

sin_port.go
   func Sin(x float64) float64 { ... }  // pure-Go impl

and then link in either sin_decl.go+sin_386.s or
just sin_port.go.  The Makefile actually did the magic
of linking in only the _port.go files for those without
assembly and only the _decl.go files for those with
assembly, or at least some of that magic.

The biggest problem with this, beyond being hard
to explain to the build system, is that once you do
explain it to the build system, godoc knows which
of sin_port.go or sin_decl.go are involved on a given
architecture, and it (correctly) ignores the other.
That means you have to put identical doc comments
in both files.

The new approach, which is more like what we did
in the later packages math/big and sync/atomic,
is to have

sin.go
   func Sin(x float64) float64  // decl only
   func sin(x float64) float64 {...}  // pure-Go impl

sin_386.s
   // assembly for Sin (ignores sin)
sin_amd64.s
   // assembly for Sin: jmp sin
sin_arm.s
   // assembly for Sin: jmp sin

Once we abandon Makefiles we can put all the assembly
stubs in one source file, so the number of files will
actually go down.

Chris asked whether the branches cost anything.
Given that they are branching to pure-Go implementations
that are not typically known for their speed, the single
direct branch is not going to be noticeable.  That is,
it's on the slow path.

An alternative would have been to preserve the old
"only write assembly files when there's an implementation"
and still have just one copy of the declaration of Sin
(and thus one doc comment) by doing:

sin.go
   func Sin(x float64) float64 { return sin(x) }

sin_decl.go
   func sin(x float64) float64 // declaration only
sin_386.s
   // assembly for sin

sin_port.go
   func sin(x float64) float64 { portable code }

In this version everyone would link in sin.go and
then either sin_decl.go+sin_386.s or sin_port.go.

This has an extra function call on all paths, including
the "fast path" to get to assembly, and it triples the
number of Go files involved compared to what I did
in this CL.  On the other hand you don't have to
write assembly stubs.  After starting down this path
I decided that the assembly stubs were the easier
approach.

As for generating the assembly stubs on the fly, much
of the goal here is to eliminate magic from the build
process, so that zero-configuration tools like goinstall
or the new go tool can handle this package.

R=golang-dev, r, cw, iant, r
CC=golang-dev
http://codereview.appspot.com/5488057