code review 6060047: linnix: an inferno/nix hybrid for bootstrapping Nix on ...

linnix: an inferno/nix hybrid for bootstrapping Nix on foreign systems.
This is an initial implementation of a Nix bootstrap for foreign architectures. It's not yet functional but it contains the basic structure needed to bootstrap a Nix system, using the inferno toolchain and directory hierarchy.

Linnix follows the conventions set by Inferno native kernels. Nix include files live in /usr/linnix/Nix, while the operating system kernel lives in /usr/linnix/os.

The majority of the work up until this point has been decoupling the Nix kernel from plan9 dependencies and separating adding bourne shell equivalents of rc only mkfile code.

[npe, 2012-04-18 13:58:03]

Hello nixiedev@gmail.com, rminnich@gmail.com, charles.forsyth@gmail.com (cc:
nix-dev@googlegroups.com),

I'd like you to review this change to
https://noah.evans@code.google.com/p/nix-os/

[0intro, 2012-04-18 14:22:16]

>    [[ 875 additional files ]]

Wow, I'm pretty sure this CL is way to big for rietveld.

-- 
David du Colombier

[rminnich, 2012-04-26 05:44:25]

I hate to carp, but I'm going to have to. This is a 1000-file patch that by its
own admission does not work, meaning reviewers can not test it. 

I understand the goal is to make NIX more accessible to non-Plan 9 users. I
think that's a good goal. I just don't see how this approach gets us there. 

I'd like to propose a different approach. Shells all provide the same basic
primitives, as do make commands. They can spawn processes, redirect IO, set up
pipes. So take a standard NIX source tree, and aim for the lowest common
denominator. Create mkfiles that gmake can process as well as mk. Or at least
make mkfiles that look like:
</$objtype/mkfile
<make.common
and
include $objtype/makefile
include make.common

(This is what Russ proposed when I tried to put mkfiles into Go. He made it
clear it was not acceptable to have two entirely parallel sets of recipes,
unless the one simply included the other. He was right.)

Same with shell scripts. Get rid of constructs that are not common to sh and rc.
If you've got a for loop over 5 directories, which we do, then just break it out
into five sets of commands. Yes, ugly, but I think two parallel sets of
mkfiles/makefile and rc scripts/sh scripts is going to be much harder to deal
with. 

Failing that, I think this set of changes needs to come in gradually, in working
form, in a way that we can test it. This patch is more than I can understand.
Sorry, I'm slow. 

Every file you add should be considered another barrier to acceptance in other
communities. Hence, for every file you add, you might want to show how you are
removing a file. 

This is a neat idea, but it needs another iteration, and I think it has to be
added in an incremental manner in a way in which everything always keeps
working. That was an original goal of NIX anyway -- everything keeps working. 

Another approach, which I'm more comfortable with, would be to change 6? so they
build under both gcc and Plan 9 -- easy, been done, see Go! -- and then use
those to build the rest of the binaries. I still think that can work.

[npe, 2012-04-26 07:38:35]

That was the initial stab. There's a new version that works. If I get
permission to release to the wild, then I'll bitbucket it and then we
can think about it how to deal with it from there.

As for getting rid of the sh/rc dependencies, I think at the end of
the day you should probably include rc in the distro and then do
MKSHELL=rc (suggested by Russ).

At the end of the day I think I'd propose something more radical,
which is doing away with mk and shells entirely and doing the build
system in go. Much of the complexity in the Nix build system is in
mk/parse, which is not very accessible. Go templates make this a lot
easier. For dependency analysis and compiling much of the "go" command
could be repurposed there.

Noah


On Thu, Apr 26, 2012 at 7:44 AM,  <rminnich@gmail.com> wrote:
> I hate to carp, but I'm going to have to. This is a 1000-file patch that
> by its own admission does not work, meaning reviewers can not test it.
>
> I understand the goal is to make NIX more accessible to non-Plan 9
> users. I think that's a good goal. I just don't see how this approach
> gets us there.
>
> I'd like to propose a different approach. Shells all provide the same
> basic primitives, as do make commands. They can spawn processes,
> redirect IO, set up pipes. So take a standard NIX source tree, and aim
> for the lowest common denominator. Create mkfiles that gmake can process
> as well as mk. Or at least make mkfiles that look like:
> </$objtype/mkfile
> <make.common
> and
> include $objtype/makefile
> include make.common
>
> (This is what Russ proposed when I tried to put mkfiles into Go. He made
> it clear it was not acceptable to have two entirely parallel sets of
> recipes, unless the one simply included the other. He was right.)
>
> Same with shell scripts. Get rid of constructs that are not common to sh
> and rc. If you've got a for loop over 5 directories, which we do, then
> just break it out into five sets of commands. Yes, ugly, but I think two
> parallel sets of mkfiles/makefile and rc scripts/sh scripts is going to
> be much harder to deal with.
>
> Failing that, I think this set of changes needs to come in gradually, in
> working form, in a way that we can test it. This patch is more than I
> can understand. Sorry, I'm slow.
>
> Every file you add should be considered another barrier to acceptance in
> other communities. Hence, for every file you add, you might want to show
> how you are removing a file.
>
> This is a neat idea, but it needs another iteration, and I think it has
> to be added in an incremental manner in a way in which everything always
> keeps working. That was an original goal of NIX anyway -- everything
> keeps working.
>
> Another approach, which I'm more comfortable with, would be to change 6?
> so they build under both gcc and Plan 9 -- easy, been done, see Go! --
> and then use those to build the rest of the binaries. I still think that
> can work.
>
>
> http://codereview.appspot.com/6060047/

[rminnich, 2012-04-26 17:23:38]

On 2012/04/26 07:38:35, npe wrote:
> That was the initial stab. There's a new version that works. 

Then we definitely want to abandon this patch. It's huge and if it's just a stab
I think it will hurt too much.

> As for getting rid of the sh/rc dependencies, I think at the end of
> the day you should probably include rc in the distro and then do
> MKSHELL=rc (suggested by Russ).

sounds good. This seems as though it would be a non-hurtful change to the
current source. Why not start here, with that 
simple change, and see how far we get?

> 
> At the end of the day I think I'd propose something more radical,
> which is doing away with mk and shells entirely and doing the build
> system in go. Much of the complexity in the Nix build system is in
> mk/parse, which is not very accessible. Go templates make this a lot
> easier. For dependency analysis and compiling much of the "go" command
> could be repurposed there.

if we're doing that then why add these 1000 files? This is too large a change if
it's temporary IMHO.

ron

[npe, 2012-04-26 19:26:57]

*** Abandoned ***

[npe, 2012-04-26 19:37:50]

Good point. Gone.

It was easier initially to just do things the inferno way. I was able
to get around most of the mkfiles by just transliterating the shell
code into /bin/sh. I'm not sure if the plan9port rc or inferno rcsh is
the best way to go on the shell though.

Akshat, what is the critical path on go these days?

Noah


On Thu, Apr 26, 2012 at 7:23 PM,  <rminnich@gmail.com> wrote:
> On 2012/04/26 07:38:35, npe wrote:
>>
>> That was the initial stab. There's a new version that works.
>
>
> Then we definitely want to abandon this patch. It's huge and if it's
> just a stab I think it will hurt too much.
>
>
>> As for getting rid of the sh/rc dependencies, I think at the end of
>> the day you should probably include rc in the distro and then do
>> MKSHELL=rc (suggested by Russ).
>
>
> sounds good. This seems as though it would be a non-hurtful change to
> the current source. Why not start here, with that
> simple change, and see how far we get?
>
>
>
>> At the end of the day I think I'd propose something more radical,
>> which is doing away with mk and shells entirely and doing the build
>> system in go. Much of the complexity in the Nix build system is in
>> mk/parse, which is not very accessible. Go templates make this a lot
>> easier. For dependency analysis and compiling much of the "go" command
>> could be repurposed there.
>
>
> if we're doing that then why add these 1000 files? This is too large a
> change if it's temporary IMHO.
>
> ron
>
> http://codereview.appspot.com/6060047/

[akumar_mail.nanosouffle.net, 2012-04-26 21:09:11]

On 26 April 2012 12:37, Noah Evans <noah.evans@gmail.com> wrote:
> Akshat, what is the critical path on go these days?

Once my tsemacquire and fmt patches are
accepted to Plan 9, one only needs two
CLs: 6031056 and 5608059 for a Go that
builds and runs. I can go on about what
needs to be done, but for your purposes,
that should be all you need in order to use
the build system.


On 26 April 2012 13:41, Charles Forsyth <charles.forsyth@gmail.com> wrote:
> actually, the Inferno ones (which Noah was using, I think) do just that.

Which is probably why the Go team chose the
Inferno suite as a basis for their own.

> On 26 April 2012 01:44, <rminnich@gmail.com> wrote:
>>
>> Another approach, which I'm more comfortable with, would be to change 6?
>> so they build under both gcc and Plan 9
>
>

[npe, 2012-04-26 21:22:59]

Did you submit the patches through patch(1) already? If so how long
have you been waiting?

Charles might know a little bit about the Inferno compilers and how
they got into go. ;)

Noah


On Thu, Apr 26, 2012 at 11:08 PM, Akshat Kumar
<akumar@mail.nanosouffle.net> wrote:
> On 26 April 2012 12:37, Noah Evans <noah.evans@gmail.com> wrote:
>> Akshat, what is the critical path on go these days?
>
> Once my tsemacquire and fmt patches are
> accepted to Plan 9, one only needs two
> CLs: 6031056 and 5608059 for a Go that
> builds and runs. I can go on about what
> needs to be done, but for your purposes,
> that should be all you need in order to use
> the build system.
>
>
> On 26 April 2012 13:41, Charles Forsyth <charles.forsyth@gmail.com> wrote:
>> actually, the Inferno ones (which Noah was using, I think) do just that.
>
> Which is probably why the Go team chose the
> Inferno suite as a basis for their own.
>
>> On 26 April 2012 01:44, <rminnich@gmail.com> wrote:
>>>
>>> Another approach, which I'm more comfortable with, would be to change 6?
>>> so they build under both gcc and Plan 9
>>
>>

[akumar_mail.nanosouffle.net, 2012-04-26 21:35:40]

On 26 April 2012 14:22, Noah Evans <noah.evans@gmail.com> wrote:
> Did you submit the patches through patch(1) already? If so how long
> have you been waiting?

Yes, I did; I'm working on a few things
jmk needs. :-)

> Charles might know a little bit about the Inferno compilers and how
> they got into go. ;)
>
> Noah

I know quite well that he knows this. :-)
I was just giving an observation for others.

> On Thu, Apr 26, 2012 at 11:08 PM, Akshat Kumar
> <akumar@mail.nanosouffle.net> wrote:
>> On 26 April 2012 12:37, Noah Evans <noah.evans@gmail.com> wrote:
>>> Akshat, what is the critical path on go these days?
>>
>> Once my tsemacquire and fmt patches are
>> accepted to Plan 9, one only needs two
>> CLs: 6031056 and 5608059 for a Go that
>> builds and runs. I can go on about what
>> needs to be done, but for your purposes,
>> that should be all you need in order to use
>> the build system.
>>
>>
>> On 26 April 2012 13:41, Charles Forsyth <charles.forsyth@gmail.com> wrote:
>>> actually, the Inferno ones (which Noah was using, I think) do just that.
>>
>> Which is probably why the Go team chose the
>> Inferno suite as a basis for their own.
>>
>>> On 26 April 2012 01:44, <rminnich@gmail.com> wrote:
>>>>
>>>> Another approach, which I'm more comfortable with, would be to change 6?
>>>> so they build under both gcc and Plan 9
>>>
>>>

[npe, 2012-04-26 21:41:19]

Hah. What is your timeframe on the go stuff? That might change my
plans considerably.

Noah


On Thu, Apr 26, 2012 at 11:35 PM, Akshat Kumar
<akumar@mail.nanosouffle.net> wrote:
> On 26 April 2012 14:22, Noah Evans <noah.evans@gmail.com> wrote:
>> Did you submit the patches through patch(1) already? If so how long
>> have you been waiting?
>
> Yes, I did; I'm working on a few things
> jmk needs. :-)
>
>> Charles might know a little bit about the Inferno compilers and how
>> they got into go. ;)
>>
>> Noah
>
> I know quite well that he knows this. :-)
> I was just giving an observation for others.
>
>> On Thu, Apr 26, 2012 at 11:08 PM, Akshat Kumar
>> <akumar@mail.nanosouffle.net> wrote:
>>> On 26 April 2012 12:37, Noah Evans <noah.evans@gmail.com> wrote:
>>>> Akshat, what is the critical path on go these days?
>>>
>>> Once my tsemacquire and fmt patches are
>>> accepted to Plan 9, one only needs two
>>> CLs: 6031056 and 5608059 for a Go that
>>> builds and runs. I can go on about what
>>> needs to be done, but for your purposes,
>>> that should be all you need in order to use
>>> the build system.
>>>
>>>
>>> On 26 April 2012 13:41, Charles Forsyth <charles.forsyth@gmail.com> wrote:
>>>> actually, the Inferno ones (which Noah was using, I think) do just that.
>>>
>>> Which is probably why the Go team chose the
>>> Inferno suite as a basis for their own.
>>>
>>>> On 26 April 2012 01:44, <rminnich@gmail.com> wrote:
>>>>>
>>>>> Another approach, which I'm more comfortable with, would be to change 6?
>>>>> so they build under both gcc and Plan 9
>>>>
>>>>

[dho, 2012-04-26 21:48:57]

Op 26 april 2012 17:40 heeft Noah Evans <noah.evans@gmail.com> het
volgende geschreven:
> Hah. What is your timeframe on the go stuff? That might change my
> plans considerably.

I haven't been following the Plan 9 Go port much, but happy to help
with it if anything's needed.

--dho

> Noah
>
>
> On Thu, Apr 26, 2012 at 11:35 PM, Akshat Kumar
> <akumar@mail.nanosouffle.net> wrote:
>> On 26 April 2012 14:22, Noah Evans <noah.evans@gmail.com> wrote:
>>> Did you submit the patches through patch(1) already? If so how long
>>> have you been waiting?
>>
>> Yes, I did; I'm working on a few things
>> jmk needs. :-)
>>
>>> Charles might know a little bit about the Inferno compilers and how
>>> they got into go. ;)
>>>
>>> Noah
>>
>> I know quite well that he knows this. :-)
>> I was just giving an observation for others.
>>
>>> On Thu, Apr 26, 2012 at 11:08 PM, Akshat Kumar
>>> <akumar@mail.nanosouffle.net> wrote:
>>>> On 26 April 2012 12:37, Noah Evans <noah.evans@gmail.com> wrote:
>>>>> Akshat, what is the critical path on go these days?
>>>>
>>>> Once my tsemacquire and fmt patches are
>>>> accepted to Plan 9, one only needs two
>>>> CLs: 6031056 and 5608059 for a Go that
>>>> builds and runs. I can go on about what
>>>> needs to be done, but for your purposes,
>>>> that should be all you need in order to use
>>>> the build system.
>>>>
>>>>
>>>> On 26 April 2012 13:41, Charles Forsyth <charles.forsyth@gmail.com> wrote:
>>>>> actually, the Inferno ones (which Noah was using, I think) do just that.
>>>>
>>>> Which is probably why the Go team chose the
>>>> Inferno suite as a basis for their own.
>>>>
>>>>> On 26 April 2012 01:44, <rminnich@gmail.com> wrote:
>>>>>>
>>>>>> Another approach, which I'm more comfortable with, would be to change 6?
>>>>>> so they build under both gcc and Plan 9
>>>>>
>>>>>