code review 7060044: cmd/ld: support for linking with host linker

cmd/ld: support for linking with host linker

A step toward a fix for issue 4069.

To allow linking with arbitrary host object files, add a linker mode
that can generate a host object file instead of an executable.
Then the host linker can be invoked to generate the final executable.

This CL adds a new -hostobj flag that instructs the linker to write
a host object file instead of an executable.

That is, this works:

        go tool 6g x.go
        go tool 6l -hostobj -o x.o x.6
        ld -e _rt0_amd64_linux x.o
        ./a.out

as does:

        go tool 8g x.go
        go tool 8l -hostld ignored -o x.o x.8
        ld -m elf_i386 -e _rt0_386_linux x.o
        ./a.out

Because 5l was never updated to use the standard relocation scheme,
it will take more work to get this working on ARM.

This is a checkpoint of the basic functionality. It does not work
with cgo yet, and cgo is the main reason for the change.
The command-line interface will likely change too.
The gc linker has other information that needs to be returned to
the caller for use when invoking the host linker besides the single
object file.

[rsc, 2013-01-06 22:28:43]

Hello iant (cc: golang-dev@googlegroups.com),

I'd like you to review this change to
https://go.googlecode.com/hg/

[iant, 2013-01-07 22:14:59]

LGTM

https://codereview.appspot.com/7060044/diff/1009/src/cmd/6l/obj.c
File src/cmd/6l/obj.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/6l/obj.c#newcode135
src/cmd/6l/obj.c:135: sysfatal("cannot use -hostobj with output header type");
This error message is rather obscure.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/8l/obj.c
File src/cmd/8l/obj.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/8l/obj.c#newcode141
src/cmd/8l/obj.c:141: sysfatal("cannot use -hostobj with output header type");
Obscure error message.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/data.c
File src/cmd/ld/data.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/data.c#newcode180
src/cmd/ld/data.c:180: if(isobj || archreloc(r, s, &o) < 0)
Checking isobj like this won't work for ARM or almost any processor other than
x86.  Probably easiest to have archreloc handle isobj in which case this line
can be reverted.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/data.c#newcode186
src/cmd/ld/data.c:186: if(thechar == '6')
Checking thechar == '6' here seems a bit ugly but it's OK with me if it's OK
with you.  The relevant distinction is SHT_REL or SHT_RELA, which you also check
elsewhere.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c
File src/cmd/ld/elf.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c#newcode779
src/cmd/ld/elf.c:779: // Also nothing to relocate in .shstrtab.
I would not be surprised if emitting empty reloc sections causes trouble
somewhere down the road.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c#newcode871
src/cmd/ld/elf.c:871: while(cpos()&7)
For 386 and ARM you only need to align to a 4 byte boundary, though I suppose
additional alignment doesn't hurt.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c#newcode1462
src/cmd/ld/elf.c:1462: // TODO(rsc): Do these go into .o files too?
Pretty sure these are only required in executables.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/lib.h
File src/cmd/ld/lib.h (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/lib.h#newcode140
src/cmd/ld/lib.h:140: EXTERN	char*	hostld;
Where is hostld used?

[rsc, 2013-01-31 03:00:58]

https://codereview.appspot.com/7060044/diff/1009/src/cmd/6l/obj.c
File src/cmd/6l/obj.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/6l/obj.c#newcode135
src/cmd/6l/obj.c:135: sysfatal("cannot use -hostobj with output header type");
On 2013/01/07 22:14:59, iant wrote:
> This error message is rather obscure.

Done.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/8l/obj.c
File src/cmd/8l/obj.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/8l/obj.c#newcode141
src/cmd/8l/obj.c:141: sysfatal("cannot use -hostobj with output header type");
On 2013/01/07 22:14:59, iant wrote:
> Obscure error message.

Done.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/data.c
File src/cmd/ld/data.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/data.c#newcode180
src/cmd/ld/data.c:180: if(isobj || archreloc(r, s, &o) < 0)
On 2013/01/07 22:14:59, iant wrote:
> Checking isobj like this won't work for ARM or almost any processor other than
> x86.  Probably easiest to have archreloc handle isobj in which case this line
> can be reverted.

archreloc handles relocations pulled in from .o files. Since we're not reading
.o files in this mode it shouldn't happen.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/data.c#newcode186
src/cmd/ld/data.c:186: if(thechar == '6')
On 2013/01/07 22:14:59, iant wrote:
> Checking thechar == '6' here seems a bit ugly but it's OK with me if it's OK
> with you.  The relevant distinction is SHT_REL or SHT_RELA, which you also
check
> elsewhere.

Acknowledged.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c
File src/cmd/ld/elf.c (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c#newcode779
src/cmd/ld/elf.c:779: // Also nothing to relocate in .shstrtab.
On 2013/01/07 22:14:59, iant wrote:
> I would not be surprised if emitting empty reloc sections causes trouble
> somewhere down the road.

Acknowledged (it's not trivial to suppress them).

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c#newcode871
src/cmd/ld/elf.c:871: while(cpos()&7)
On 2013/01/07 22:14:59, iant wrote:
> For 386 and ARM you only need to align to a 4 byte boundary, though I suppose
> additional alignment doesn't hurt.

Ack.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/elf.c#newcode1462
src/cmd/ld/elf.c:1462: // TODO(rsc): Do these go into .o files too?
On 2013/01/07 22:14:59, iant wrote:
> Pretty sure these are only required in executables.

Done.

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/lib.h
File src/cmd/ld/lib.h (right):

https://codereview.appspot.com/7060044/diff/1009/src/cmd/ld/lib.h#newcode140
src/cmd/ld/lib.h:140: EXTERN	char*	hostld;
On 2013/01/07 22:14:59, iant wrote:
> Where is hostld used?

Gone.

[iant, 2013-01-31 17:28:32]

For some reason I'm not seeing the changes, e.g., to 6l/obj.c.

[rsc, 2013-01-31 17:31:23]

Uploaded now. I was going to submit but merging with the shared object
changes has produced code that no longer links correctly in -hostobj mode.
I might check it in anyway just to checkpoint the review.

[iant2, 2013-01-31 17:55:43]

LGTM

On Thu, Jan 31, 2013 at 9:31 AM, Russ Cox <rsc@golang.org> wrote:
> Uploaded now. I was going to submit but merging with the shared object
> changes has produced code that no longer links correctly in -hostobj mode. I
> might check it in anyway just to checkpoint the review.
>

[rsc, 2013-01-31 22:11:37]

*** Submitted as https://code.google.com/p/go/source/detail?r=98034d036d03 ***

cmd/ld: support for linking with host linker

A step toward a fix for issue 4069.

To allow linking with arbitrary host object files, add a linker mode
that can generate a host object file instead of an executable.
Then the host linker can be invoked to generate the final executable.

This CL adds a new -hostobj flag that instructs the linker to write
a host object file instead of an executable.

That is, this works:

        go tool 6g x.go
        go tool 6l -hostobj -o x.o x.6
        ld -e _rt0_amd64_linux x.o
        ./a.out

as does:

        go tool 8g x.go
        go tool 8l -hostld ignored -o x.o x.8
        ld -m elf_i386 -e _rt0_386_linux x.o
        ./a.out

Because 5l was never updated to use the standard relocation scheme,
it will take more work to get this working on ARM.

This is a checkpoint of the basic functionality. It does not work
with cgo yet, and cgo is the main reason for the change.
The command-line interface will likely change too.
The gc linker has other information that needs to be returned to
the caller for use when invoking the host linker besides the single
object file.

R=iant, iant
CC=golang-dev
https://codereview.appspot.com/7060044

[dave_cheney.net, 2013-02-01 00:08:09]

This appears to have broken the non linux 386 builders.

On Fri, Feb 1, 2013 at 9:11 AM,  <rsc@golang.org> wrote:
> *** Submitted as
> https://code.google.com/p/go/source/detail?r=98034d036d03 ***
>
> cmd/ld: support for linking with host linker
>
> A step toward a fix for issue 4069.
>
> To allow linking with arbitrary host object files, add a linker mode
> that can generate a host object file instead of an executable.
> Then the host linker can be invoked to generate the final executable.
>
> This CL adds a new -hostobj flag that instructs the linker to write
> a host object file instead of an executable.
>
> That is, this works:
>
>         go tool 6g x.go
>         go tool 6l -hostobj -o x.o x.6
>         ld -e _rt0_amd64_linux x.o
>         ./a.out
>
> as does:
>
>         go tool 8g x.go
>         go tool 8l -hostld ignored -o x.o x.8
>         ld -m elf_i386 -e _rt0_386_linux x.o
>         ./a.out
>
> Because 5l was never updated to use the standard relocation scheme,
> it will take more work to get this working on ARM.
>
> This is a checkpoint of the basic functionality. It does not work
> with cgo yet, and cgo is the main reason for the change.
> The command-line interface will likely change too.
> The gc linker has other information that needs to be returned to
> the caller for use when invoking the host linker besides the single
> object file.
>
> R=iant, iant
> CC=golang-dev
> https://codereview.appspot.com/7060044
>
>
> https://codereview.appspot.com/7060044/
>
> --
>
> ---You received this message because you are subscribed to the Google Groups
> "golang-dev" group.
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>