Be "smarter" when injecting ';' into the parser's input...

Be "smarter" when injecting ';' into the parser's input stream
in order to avoid unnecessary restrictions on source code format.
In order to more accurately implement the apparent desire in the
description of ';' injection that ';' are only optional at the end
of statements the injection of ';' for newlines in "for", "if" and
"switch" statements may be disallowed.

Patch set 1 disallows them and patch set 2 allows them.

[pwil3058, 2010-06-17 05:57:21]

Two patch sets have been uploaded.

They are identical except that the first disallows ';' injection in "if", "for"
and "switch" headers and the second allows them.

[pwil3058, 2010-06-20 02:04:29]

The test/syntax/ff*.go files added by this patch will need to be modified in
order to prevent "hg gofmt" reformatting them and making them pointless.  As I
see it all that is necessary is the addition of a syntax error so I think adding
a ';' on a fresh line at the end of each file should do the trick by triggering
the "empty top level statement" error.

[rsc, 2010-06-23 02:55:28]

Regarding the code:

We tried various context-sensitive semicolon elision
before the public release of Go, and they were dissatisfying mainly
because understanding a failure mode required understanding
complex rules that dictated when semicolons were okay or not.
The current scheme has disadvantages but also the significant
advantage that when it disagrees with what programmers expect,
it is utterly trivial to explain what happened.  The complexity of
the changes to lex.c scare me.  They remind me of the other
attempts: when they break they'll be mysterious.  Whatever
proposal is made, it will need a significantly simpler implementation
than what's in this CL.  I haven't thought enough about the spec changes
to tell whether the current code can be simplified, but I urge you to try.

Regarding the spec:

> The semicolon at the end of a statement may be omitted if the statement ends
in
> a new line and the line's final token is

This wording, which does not _require_ semicolon insertion in those cases,
is dangerously close to the JavaScript fiasco, where changing one line
affects whether the previous line has a semicolon inserted.  The equivalent
case in Go is something like

    x = a
    <-b

which is a valid statement list whether or not a semicolon is inserted at the
end of the first line.  The current spec is very clear that semicolons are
always inserted, in part to avoid ambiguity and context sensitive behavior
in examples like this one.

Can you summarize, as concisely as possible, the intended changes
your CL makes to the language?  Is the case above supposed to be
handled differently than in the current implementation?

Russ

[pwil3058, 2010-06-23 05:23:23]

On 23/06/10 12:55, Russ Cox wrote:
> Regarding the code:
>
> We tried various context-sensitive semicolon elision
> before the public release of Go, and they were dissatisfying mainly
> because understanding a failure mode required understanding
> complex rules that dictated when semicolons were okay or not.
> The current scheme has disadvantages but also the significant
> advantage that when it disagrees with what programmers expect,
> it is utterly trivial to explain what happened.

But it will also make Go a laughing stock amongst those who take these 
things seriously.  E.g. allowing:

     for
         i := 0
         i < 10
         i++ {
     }
     for
         i := 0
         i < 10
     {
         i++
     }
     for i := 0; i < 10; i++ {
     }
     for i := 0; i < 10;
     {
         i++
     }
     for ;;
     {
     }

but not:

     for i := 0; i < 10; i++
     {
     }

is pretty hard to explain except as an artefact of a sloppy 
implementation.  Please note instances of new lines before the '{' in 
some of the legal instances.

>  The complexity of
> the changes to lex.c scare me.

You're easily frightened as they're fairly simple.

>  They remind me of the other
> attempts: when they break they'll be mysterious.  Whatever
> proposal is made, it will need a significantly simpler implementation
> than what's in this CL.

It has to be as complex as it has to be.  As a programmer, you should be 
aware of this.

I examined a lot of alternative mechanisms and this was the simplest 
that didn't have unnecessary side effects.

> I haven't thought enough about the spec changes
> to tell whether the current code can be simplified, but I urge you to try.

I think that only places that "surprising" failures due to ';' injection 
with this change will occur are:

1. before a 'func' where it is the return type for a function (easy to 
explain and easy work around using '(' ')') and
2. with new lines in function types and literals headers when not at top 
level (harder but not impossible to explain).  It's not possible to 
safely address these in lex.c but changes to lex.c and go.y may do the job.

In other words, nowhere where they don't already occur.

>
> Regarding the spec:
>
>> The semicolon at the end of a statement may be omitted if the statement ends
in
>> a new line and the line's final token is
>
> This wording, which does not _require_ semicolon insertion in those cases,
> is dangerously close to the JavaScript fiasco, where changing one line
> affects whether the previous line has a semicolon inserted.  The equivalent
> case in Go is something like
>
>      x = a
>      <-b
>
> which is a valid statement list whether or not a semicolon is inserted at the
> end of the first line.

OK.  I see that as a possible problem but my wording does say "at the 
end of a statement".  Does it need a rider to say expressions can only 
be broken by a new line after an operator?

BTW, my change will not effect the interpretation of that code.

>  The current spec is very clear that semicolons are
> always inserted, in part to avoid ambiguity and context sensitive behavior
> in examples like this one.

The current wording does NOT read as a specification.  It reads as a 
statement of what the implementation does.  It's OK to do the 
implementation first and then write the spec but you've got to make it 
sound like the implementation was a consequence of the spec and the 
current wording doesn't pass that test.

>
> Can you summarize, as concisely as possible, the intended changes
> your CL makes to the language?

The changes it makes are:

1. Allow new lines in top level function headers.
2. Allow new lines before '{' in if, for and switch headers.
3. Allow new lines before "else".

The first patch also disallows the use of new lines instead of ';' in 
if, for and switch headers which is different to the current 
implementation which allows it.

>  Is the case above supposed to be
> handled differently than in the current implementation?

If you mean the:

x = a
<- b

then no.

Would it help if I gave a more detailed explanation of how the change works?

Peter
PS another advantage of this patch is that it provides a mechanism to 
get rid of all those annoying invalid "illegal non declaration statement 
at top level" error messages that are triggered by a syntax error 
followed by valid statements inside a function.

[iant2, 2010-06-23 06:38:20]

Peter Williams <pwil3058@gmail.com> writes:

> The current wording does NOT read as a specification.  It reads as a
> statement of what the implementation does.  It's OK to do the
> implementation first and then write the spec but you've got to make it
> sound like the implementation was a consequence of the spec and the
> current wording doesn't pass that test.

You've said things like this before, but I honestly do not understand
this complaint.  The current wording is clear and unambiguous.  It
says precisely how any tool which reads Go is required to behave.
That is exactly what I want from a language spec.


I am concerned by your spec patch

http://codereview.appspot.com/1678042/diff/2001/3001

because your proposed wording does not seem clear and unambiguous to
me.  It relies on the notion of "the end of a statement" which is not
defined.  We all agree that under all proposals statements are
permitted to continue across lines in some cases.  It seems to me that
saying "the end of a statement" in combination with "if the statement
ends in a new line" implies that we already know where the statement
ends.  But the spec is supposed to be telling us where the statement
ends.  It's not supposed to assume that we already know.

Ian

[pwil3058, 2010-06-23 06:52:56]

On 23/06/10 16:37, Ian Lance Taylor wrote:
> Peter Williams<pwil3058@gmail.com>  writes:
>
>> The current wording does NOT read as a specification.  It reads as a
>> statement of what the implementation does.  It's OK to do the
>> implementation first and then write the spec but you've got to make it
>> sound like the implementation was a consequence of the spec and the
>> current wording doesn't pass that test.
>
> You've said things like this before, but I honestly do not understand
> this complaint.

Specifications are supposed to say "this is what we want" not "we did 
this it has strange consequences and you can figure them out for 
yourselves".  But maybe that's just me.

>  The current wording is clear and unambiguous.  It
> says precisely how any tool which reads Go is required to behave.
> That is exactly what I want from a language spec.

It's consequences are far from clear as demonstrated by e-mails to 
golang-nuts.

>
>
> I am concerned by your spec patch
>
> http://codereview.appspot.com/1678042/diff/2001/3001
>
> because your proposed wording does not seem clear and unambiguous to
> me.  It relies on the notion of "the end of a statement" which is not
> defined.

I disagree.

>  We all agree that under all proposals statements are
> permitted to continue across lines in some cases.  It seems to me that
> saying "the end of a statement" in combination with "if the statement
> ends in a new line" implies that we already know where the statement
> ends.  But the spec is supposed to be telling us where the statement
> ends.  It's not supposed to assume that we already know.

I was trying to keep it as simple as possible and make it sound like a 
spec rather than a statement of implementation.  I'm happy for it to be 
reworded or even keep the original.

Peter
PS It worries me that you don't see the problems caused by ';' injection 
as a "show stopper".  You need to eliminate them or abandon ';' 
injection if you want Go to be taken seriously.

[iant2, 2010-06-23 14:06:56]

Peter Williams <pwil3058@gmail.com> writes:

> On 23/06/10 16:37, Ian Lance Taylor wrote:
>> Peter Williams<pwil3058@gmail.com>  writes:
>>
>>> The current wording does NOT read as a specification.  It reads as a
>>> statement of what the implementation does.  It's OK to do the
>>> implementation first and then write the spec but you've got to make it
>>> sound like the implementation was a consequence of the spec and the
>>> current wording doesn't pass that test.
>>
>> You've said things like this before, but I honestly do not understand
>> this complaint.
>
> Specifications are supposed to say "this is what we want" not "we did
> this it has strange consequences and you can figure them out for
> yourselves".  But maybe that's just me.

I disagree.  The place for "this is what we want" is in the
documentation and the tutorial.  The specification is where you
describe precisely how implementations should behave.  The
specification corresponds to the ISO standard for a language; it's not
something everybody is expected to read.  (That said, I agree that the
rest of the documentation for Go is not yet complete; there is no
equivalent to K&R or "The C++ Programming Language".)

That is, for Go, the place to say "this is what we want" regarding
semicolons is: http://golang.org/doc/go_tutorial.html#tmp_35 .


>>  The current wording is clear and unambiguous.  It
>> says precisely how any tool which reads Go is required to behave.
>> That is exactly what I want from a language spec.
>
> It's consequences are far from clear as demonstrated by e-mails to
> golang-nuts.

I'm not sure how to interpret that.  The consequences are perfectly
clear in that they predict exactly the behaviour you have described.
I think you may mean something more like the consequences are
surprising.  When I say clear and unambiguous, I mean that any
reasonable person reading the text will come to the same conclusion.

Of course, it's not ideal for a spec to be surprising, and I do
appreciate your efforts to reduce the surprise.  But as somebody who
writes tools, it's far more important that the spec be clear and
unambiguous than that it be unsurprising.


>> I am concerned by your spec patch
>>
>> http://codereview.appspot.com/1678042/diff/2001/3001
>>
>> because your proposed wording does not seem clear and unambiguous to
>> me.  It relies on the notion of "the end of a statement" which is not
>> defined.
>
> I disagree.

I hope that my writing above has made more clear what my concern is
here.  Go is not a language defined by an implementation--there are
already two independent implementations.  It's important that every
implementor who reads the spec understand it in the same way.  That
means that the spec must not rely on undefined notions like "the end
of a statement."


>>  We all agree that under all proposals statements are
>> permitted to continue across lines in some cases.  It seems to me that
>> saying "the end of a statement" in combination with "if the statement
>> ends in a new line" implies that we already know where the statement
>> ends.  But the spec is supposed to be telling us where the statement
>> ends.  It's not supposed to assume that we already know.
>
> I was trying to keep it as simple as possible and make it sound like a
> spec rather than a statement of implementation.  I'm happy for it to
> be reworded or even keep the original.

Keeping the original language would not make sense under your
proposal, since you are in fact proposing a change in the language.
What I'm trying to get is clear and unambiguous language which
describes your proposed change.


> PS It worries me that you don't see the problems caused by ';'
> injection as a "show stopper".  You need to eliminate them or abandon
> ;' injection if you want Go to be taken seriously.

I don't agree with this argument (this does not imply that I reject
your proposed change).  One of the decisions made early on for the Go
language was that all Go code should be formatted the same way.  You
can agree or disagree with that decision, but it wasn't made
carelessly or accidentally.  One of the consequences of that decision
is that it is not very important what happens if you format your code
in a different way.  It doesn't have zero importance, because we do in
general want to avoid surprises.  But I believe that the issue is not
a show stopper.


Although this is a weak argument in this context, I'll mention that C
and C++, languages for which I am very familiar with the
specifications, have many examples of specifications that are clear,
unambiguous, and surprising.  They are surprising at the semantic
level, which is much worse than the way that Go is surprising at the
syntactic level.  Yet they are successful languages which are taken
seriously.

Ian

[rsc, 2010-06-23 22:28:13]

> The changes it makes are:
>
> 1. Allow new lines in top level function headers.
> 2. Allow new lines before '{' in if, for and switch headers.
> 3. Allow new lines before "else".
>
> The first patch also disallows the use of new lines
> instead of ';' in if, for and switch headers which is
> different to the current implementation which allows it.

It seems to me that there are much simpler ways to solve
those three problems, like defining that there is no ; injected
before a { or the word else.  In fact, that suggestion has been
made and discussed multiple times on the various lists.
But this CL does all sorts of other processing, looking for
tokens such as const, func, import, for, switch.
I'm not comfortable with that.

>> The complexity of
>> the changes to lex.c scare me.
>
> You're easily frightened ...

Most of us are.  That's a large part of why the language is
as simple as it is, and at least for me that's a huge benefit.
There have been many language features that were proposed
and delayed because they were still too complex or still not
right; and when the right thing finally came along, it was worth
the wait, and wouldn't have happened if we'd settled earlier.

> It has to be as complex as it has to be.  As a programmer,
> you should be aware of this.

Ignoring the fact that this is a tautology,
no, it doesn't have to be that complex.
That might as well be Go's slogan.

If the definition you're using requires such complex code,
then maybe the definition should be revised.  Not questioning
the definition, even in the face of overwhelming implementation
complexity, is exactly how we got C++.

> The current wording does NOT read as a specification.
> It reads as a statement of what the implementation does.
> It's OK to do the implementation first and then write the
> spec but you've got to make it sound like the implementation
> was a consequence of the spec and the current wording
> doesn't pass that test.

You've made this opinion very clear.  I disagree.

Also, we're not trying to please everyone.
If someone can't take Go seriously because of one line
in the spec that sounds too much like an implementation
for their taste, I'm okay with them looking elsewhere.

Russ

[pwil3058, 2010-06-24 00:08:17]

On 24/06/10 08:28, Russ Cox wrote:
>> The changes it makes are:
>>
>> 1. Allow new lines in top level function headers.
>> 2. Allow new lines before '{' in if, for and switch headers.
>> 3. Allow new lines before "else".
>>
>> The first patch also disallows the use of new lines
>> instead of ';' in if, for and switch headers which is
>> different to the current implementation which allows it.
>
> It seems to me that there are much simpler ways to solve
> those three problems, like defining that there is no ; injected
> before a {

No. Compound statement rules this out.

> or the word else.

Yes, this would work.  On reflection my attempts to avoid reading ahead 
one token when it might not be needed are a false saving as it has to be 
scanned eventually anyway.  It's always safe to read ahead one token 
when an injection has occurred.

>  In fact, that suggestion has been
> made and discussed multiple times on the various lists.
> But this CL does all sorts of other processing, looking for
> tokens such as const, func, import, for, switch.
> I'm not comfortable with that.

This top level can be simplified as well but may effect some of the 
error reporting stuff in place at the top level.  Basically, Go's 
grammar means the the only place a ';' can legally occur at the top 
level is before an "import", "const", "type", "var" or "func" key word 
and the top level could be simplified to swallowing all new lines except 
immediately before one of those key words.  The effect of this would be 
to weaken the effectiveness of the "non_dcl_stmt" production in "xdcl".

It would also have some effect on the "empty top level declaration" 
error but this could probably be fixed by adding ';' to the list of 
tokens before which it's OK to inject a new line.

>
>>> The complexity of
>>> the changes to lex.c scare me.
>>
>> You're easily frightened ...
>
> Most of us are.  That's a large part of why the language is
> as simple as it is, and at least for me that's a huge benefit.
> There have been many language features that were proposed
> and delayed because they were still too complex or still not
> right; and when the right thing finally came along, it was worth
> the wait, and wouldn't have happened if we'd settled earlier.
>
>> It has to be as complex as it has to be.  As a programmer,
>> you should be aware of this.
>
> Ignoring the fact that this is a tautology,
> no, it doesn't have to be that complex.
> That might as well be Go's slogan.
>
> If the definition you're using requires such complex code,
> then maybe the definition should be revised.  Not questioning
> the definition, even in the face of overwhelming implementation
> complexity, is exactly how we got C++.
>
>> The current wording does NOT read as a specification.
>> It reads as a statement of what the implementation does.
>> It's OK to do the implementation first and then write the
>> spec but you've got to make it sound like the implementation
>> was a consequence of the spec and the current wording
>> doesn't pass that test.
>
> You've made this opinion very clear.  I disagree.
>
> Also, we're not trying to please everyone.
> If someone can't take Go seriously because of one line
> in the spec that sounds too much like an implementation
> for their taste, I'm okay with them looking elsewhere.

The reason it won't be taken seriously are not that statement but its 
consequences: irrational and inconsistent source format restrictions.

No comment yet on whether you really want people to be able to use new 
lines instead of ';' in "if", "for" and "switch" statements?  I'm 
inferring from gofmt's behaviour that you'd rather they didn't.

Peter

[pwil3058, 2010-06-24 00:21:14]

On 24/06/10 00:06, Ian Lance Taylor wrote:
> Peter Williams<pwil3058@gmail.com>  writes:
>> PS It worries me that you don't see the problems caused by ';'
>> injection as a "show stopper".  You need to eliminate them or abandon
>> ;' injection if you want Go to be taken seriously.
>
> I don't agree with this argument (this does not imply that I reject
> your proposed change).  One of the decisions made early on for the Go
> language was that all Go code should be formatted the same way.  You
> can agree or disagree with that decision, but it wasn't made
> carelessly or accidentally.

In that case, you should have specified that format and arranged for the 
compiler to enforce it.  At the moment there are a myriad of ways (some 
of them quite ugly) that a programmer can deviate from "the one true 
way" to format code without having the compiler reject it.

Saying that gofmt takes up the slack doesn't cut it as one only needs to 
use that if one wants to have code accepted into the package library.

>  One of the consequences of that decision
> is that it is not very important what happens if you format your code
> in a different way.  It doesn't have zero importance, because we do in
> general want to avoid surprises.  But I believe that the issue is not
> a show stopper.
>
>
> Although this is a weak argument in this context, I'll mention that C
> and C++, languages for which I am very familiar with the
> specifications, have many examples of specifications that are clear,
> unambiguous, and surprising.  They are surprising at the semantic
> level, which is much worse than the way that Go is surprising at the
> syntactic level.  Yet they are successful languages which are taken
> seriously.

I find that true of C++ but not of C.  I graduated to C from C++ because 
I found C++ to be a confusing mess.

Peter
PS I like a lot of the new things that Go brings with it and hope it 
succeeds but am starting to give up hope.

[iant2, 2010-06-24 04:36:42]

Peter Williams <pwil3058@gmail.com> writes:

> On 24/06/10 00:06, Ian Lance Taylor wrote:
>> Peter Williams<pwil3058@gmail.com>  writes:
>>> PS It worries me that you don't see the problems caused by ';'
>>> injection as a "show stopper".  You need to eliminate them or abandon
>>> ;' injection if you want Go to be taken seriously.
>>
>> I don't agree with this argument (this does not imply that I reject
>> your proposed change).  One of the decisions made early on for the Go
>> language was that all Go code should be formatted the same way.  You
>> can agree or disagree with that decision, but it wasn't made
>> carelessly or accidentally.
>
> In that case, you should have specified that format and arranged for
> the compiler to enforce it.  At the moment there are a myriad of ways
> (some of them quite ugly) that a programmer can deviate from "the one
> true way" to format code without having the compiler reject it.

Having the compiler enforce the formatting would just waste
compilation time which is better used for other things.  In any case
gofmt does not specify the position of every character; it just
applies certain formatting rules.


>> Although this is a weak argument in this context, I'll mention that C
>> and C++, languages for which I am very familiar with the
>> specifications, have many examples of specifications that are clear,
>> unambiguous, and surprising.  They are surprising at the semantic
>> level, which is much worse than the way that Go is surprising at the
>> syntactic level.  Yet they are successful languages which are taken
>> seriously.
>
> I find that true of C++ but not of C.  I graduated to C from C++
> because I found C++ to be a confusing mess.

Specific issues with C which I have found to frequently surprise
people are the fact that signed overflow is undefined, the definition
of volatile, and the definition of restrict.


I hope you will consider my comments about the change to the spec.  I
would have to recommend against your patch if you do not include a
change to the spec which precisely describes the new rule you are
proposing, such that all Go parser implementations will always behave
in precisely the same way.

Ian

[rsc, 2010-06-24 17:24:07]

> The reason it won't be taken seriously are not that statement but its
> consequences: irrational and inconsistent source format restrictions.

You may disagree with them, but I don't think you can argue
that they're either irrational or inconsistent.  We've given you
the rationale, repeatedly, and the rule is nothing if not consistent:
if the line ends with one of the line-ending tokens, you get a
semicolon for free.  Always.

> No comment yet on whether you really want people to be able to use new lines
> instead of ';' in "if", "for" and "switch" statements?  I'm inferring from
> gofmt's behaviour that you'd rather they didn't.

My favorite thing about Go is that it explores the result of applying
simple rules consistently.  The case-based visibility is a great
example of this.  If you look back in the Mercurial history you'll
find a time when top-level declarations were tagged with
"export" or "package" to make them exported or package-local
(as opposed to file-local).  It was a simple rule and seemed to
work fine, but it didn't give us a nice way to handle struct fields.
(Annotating every struct field with export or package was too
heavy on the page.)  After a lot of discussion, we agreed to try
the case-based visibility rule instead.  We gave up file-local
visibility entirely, gave up uppercase local constants and types,
and in return got a simpler, lighter weight rule that provided the
needed functionality (control over fields).  It felt weird at the time,
probably because we were so used to the C++ and Java
approaches, but in retrospect I think it is one of Go's best tiny
features.  When I look at code in C++ or Java now, it frustrates
me that I can't tell from a use of the name whether it is a
public name or private name.  I see a call to DangerousMethod()
and think "wow, I hope external callers can't call that method!".
And then I have to spend time looking around to find out.
We also went through a handful of other proposals that were
rejected as not good enough yet, and in retrospect they weren't.
And if we'd settled for any of those we wouldn't have gotten
to where we are today.

Getting back to semicolons, I don't care whether semicolons are
required in loop headers, nor do I care whether else needs to be
on the same line as }.  What I do care about is that the rule is as
simple and predictable as possible while still giving the basic
result that I can always drop an end-of-statement semicolon from
a valid program without changing the program's meaning.  Not caring
about consequences like whether ; is required in for loop headers
underconstrains the solution space, making it more likely for it to
contain a simple one.

Your question suggests that the approach is "define the exact
behavior you want and then accept any complexity required to
get there."  It's not.  If you can suggest a rule as simple and
predictable as the current rule that produces more useful behavior,
then we'll be happy to consider it.  Multiple people have looked at
this CL and found the spec wording ambiguous and the effects
unsettling.  Instead of continuing to argue about this particular
solution, I suggest looking for others that achieve similar effects
with less mechanism.

Russ

[pwil3058, 2010-06-25 01:46:23]

On 24/06/10 14:36, Ian Lance Taylor wrote:
> Peter Williams<pwil3058@gmail.com>  writes:
>
>> On 24/06/10 00:06, Ian Lance Taylor wrote:
>>> Peter Williams<pwil3058@gmail.com>   writes:
>>>> PS It worries me that you don't see the problems caused by ';'
>>>> injection as a "show stopper".  You need to eliminate them or abandon
>>>> ;' injection if you want Go to be taken seriously.
>>>
>>> I don't agree with this argument (this does not imply that I reject
>>> your proposed change).  One of the decisions made early on for the Go
>>> language was that all Go code should be formatted the same way.  You
>>> can agree or disagree with that decision, but it wasn't made
>>> carelessly or accidentally.
>>
>> In that case, you should have specified that format and arranged for
>> the compiler to enforce it.  At the moment there are a myriad of ways
>> (some of them quite ugly) that a programmer can deviate from "the one
>> true way" to format code without having the compiler reject it.
>
> Having the compiler enforce the formatting would just waste
> compilation time which is better used for other things.  In any case
> gofmt does not specify the position of every character; it just
> applies certain formatting rules.

Then why use the "one true style" argument for supporting the problems 
caused by ';' injection?

>
>
>>> Although this is a weak argument in this context, I'll mention that C
>>> and C++, languages for which I am very familiar with the
>>> specifications, have many examples of specifications that are clear,
>>> unambiguous, and surprising.  They are surprising at the semantic
>>> level, which is much worse than the way that Go is surprising at the
>>> syntactic level.  Yet they are successful languages which are taken
>>> seriously.
>>
>> I find that true of C++ but not of C.  I graduated to C from C++
>> because I found C++ to be a confusing mess.
>
> Specific issues with C which I have found to frequently surprise
> people are the fact that signed overflow is undefined,

Something (overflow that is) that I studiously avoid.

> the definition
> of volatile, and the definition of restrict.

I've never found the need to use either of these (in fact I wasn't even 
aware that "restrict" had been introduced in C99 until you mentioned it 
and I looked it up) which would explain why I don't find them confusing. 
  I had noticed a lot of discussion about "volatile" on LKML a few years 
ago now that you mention it.  I believe the result was a much reduced 
use of "volatile" in the Linux kernel.

>
>
> I hope you will consider my comments about the change to the spec.  I
> would have to recommend against your patch if you do not include a
> change to the spec which precisely describes the new rule you are
> proposing, such that all Go parser implementations will always behave
> in precisely the same way.

Something along the lines of "except in ...." rider?

Peter
PS I just noticed something in go.y (namely the "lbrace" production) 
that makes my assumptions about LBODY invalid so my change will need to 
be changed anyway.

[pwil3058, 2010-06-25 02:05:36]

On 25/06/10 03:24, Russ Cox wrote:
>> The reason it won't be taken seriously are not that statement but its
>> consequences: irrational and inconsistent source format restrictions.
>
> You may disagree with them, but I don't think you can argue
> that they're either irrational or inconsistent.  We've given you
> the rationale,

rationale != rational

> repeatedly, and the rule is nothing if not consistent:

It's the consequences which appear inconsistent to a human.  I draw your 
attention to the list of possible "for" statements that are acceptable 
and those that aren't that I've presented before.

> if the line ends with one of the line-ending tokens, you get a
> semicolon for free.  Always.

Well it's a computer, isn't it.

I look forward to your SIGPLAN article where you justify this situation.

>
>> No comment yet on whether you really want people to be able to use new lines
>> instead of ';' in "if", "for" and "switch" statements?  I'm inferring from
>> gofmt's behaviour that you'd rather they didn't.
>
> My favorite thing about Go is that it explores the result of applying
> simple rules consistently.  The case-based visibility is a great
> example of this.  If you look back in the Mercurial history you'll
> find a time when top-level declarations were tagged with
> "export" or "package" to make them exported or package-local
> (as opposed to file-local).  It was a simple rule and seemed to
> work fine, but it didn't give us a nice way to handle struct fields.
> (Annotating every struct field with export or package was too
> heavy on the page.)  After a lot of discussion, we agreed to try
> the case-based visibility rule instead.  We gave up file-local
> visibility entirely, gave up uppercase local constants and types,
> and in return got a simpler, lighter weight rule that provided the
> needed functionality (control over fields).  It felt weird at the time,
> probably because we were so used to the C++ and Java
> approaches, but in retrospect I think it is one of Go's best tiny
> features.  When I look at code in C++ or Java now, it frustrates
> me that I can't tell from a use of the name whether it is a
> public name or private name.  I see a call to DangerousMethod()
> and think "wow, I hope external callers can't call that method!".
> And then I have to spend time looking around to find out.
> We also went through a handful of other proposals that were
> rejected as not good enough yet, and in retrospect they weren't.
> And if we'd settled for any of those we wouldn't have gotten
> to where we are today.

I mostly agree with your argument here except for your choice of lower 
case characters as the indicator as it reduces the namespace.  Did you 
consider using "_" as the first character for local variables?  Or if 
that's not viable some other character?

In a lot of fields, the capitalization of the first character of a name, 
abbreviation or symbol conveys meaning and you've stifled that a little 
with the current model.

>
> Getting back to semicolons, I don't care whether semicolons are
> required in loop headers,  nor do I care whether else needs to be
> on the same line as }.  What I do care about is that the rule is as
> simple and predictable as possible while still giving the basic
> result that I can always drop an end-of-statement semicolon from
> a valid program without changing the program's meaning.  Not caring
> about consequences like whether ; is required in for loop headers
> underconstrains the solution space, making it more likely for it to
> contain a simple one.
>
> Your question suggests that the approach is "define the exact
> behavior you want and then accept any complexity required to
> get there."  It's not.  If you can suggest a rule as simple and
> predictable as the current rule that produces more useful behavior,
> then we'll be happy to consider it.  Multiple people have looked at
> this CL and found the spec wording ambiguous and the effects
> unsettling.

We can work on that.

>  Instead of continuing to argue about this particular
> solution, I suggest looking for others that achieve similar effects
> with less mechanism.

By the way, if you wanted Go programs to be ';' free why didn't you 
define it that way in the formal grammar i.e. define new line as the 
statement terminator.  All you get from the current mess is the ability 
to put two statements on one line separated by ';' and that could be 
achieved in a formal grammar that uses new line as a statement terminator.

Peter

[iant2, 2010-06-25 05:06:50]

Peter Williams <pwil3058@gmail.com> writes:

> On 24/06/10 14:36, Ian Lance Taylor wrote:
>> Peter Williams<pwil3058@gmail.com>  writes:
>>
>>> On 24/06/10 00:06, Ian Lance Taylor wrote:
>>>> Peter Williams<pwil3058@gmail.com>   writes:
>>>>> PS It worries me that you don't see the problems caused by ';'
>>>>> injection as a "show stopper".  You need to eliminate them or abandon
>>>>> ;' injection if you want Go to be taken seriously.
>>>>
>>>> I don't agree with this argument (this does not imply that I reject
>>>> your proposed change).  One of the decisions made early on for the Go
>>>> language was that all Go code should be formatted the same way.  You
>>>> can agree or disagree with that decision, but it wasn't made
>>>> carelessly or accidentally.
>>>
>>> In that case, you should have specified that format and arranged for
>>> the compiler to enforce it.  At the moment there are a myriad of ways
>>> (some of them quite ugly) that a programmer can deviate from "the one
>>> true way" to format code without having the compiler reject it.
>>
>> Having the compiler enforce the formatting would just waste
>> compilation time which is better used for other things.  In any case
>> gofmt does not specify the position of every character; it just
>> applies certain formatting rules.
>
> Then why use the "one true style" argument for supporting the problems
> caused by ';' injection?

You have changed the sense of my words, which are quoted above.  I
didn't use the "one true style" argument for supporting the problems
caused by ';' injection.  I used it to explain why those problems are
not a "show stopper," and why it is not necessary to eliminate them in
order for Go to be taken seriously.  If all code is formatted the same
way (approximately) then there is no semicolon issue.  The semicolon
issue only arises in code which is not formatted in the recommended
style.


>> I hope you will consider my comments about the change to the spec.  I
>> would have to recommend against your patch if you do not include a
>> change to the spec which precisely describes the new rule you are
>> proposing, such that all Go parser implementations will always behave
>> in precisely the same way.
>
> Something along the lines of "except in ...." rider?

Whatever works.  The point is that somebody new to Go who wants to
write a parser has to be able to read the spec and understand
precisely what should be accepted as a valid Go program and what
should be rejected.

Ian

[pwil3058, 2010-06-25 05:33:27]

On 25/06/10 15:06, Ian Lance Taylor wrote:
> Peter Williams<pwil3058@gmail.com>  writes:
>
>> On 24/06/10 14:36, Ian Lance Taylor wrote:
>>> Peter Williams<pwil3058@gmail.com>   writes:
>>>
>>>> On 24/06/10 00:06, Ian Lance Taylor wrote:
>>>>> Peter Williams<pwil3058@gmail.com>    writes:
>>>>>> PS It worries me that you don't see the problems caused by ';'
>>>>>> injection as a "show stopper".  You need to eliminate them or abandon
>>>>>> ;' injection if you want Go to be taken seriously.
>>>>>
>>>>> I don't agree with this argument (this does not imply that I reject
>>>>> your proposed change).  One of the decisions made early on for the Go
>>>>> language was that all Go code should be formatted the same way.  You
>>>>> can agree or disagree with that decision, but it wasn't made
>>>>> carelessly or accidentally.
>>>>
>>>> In that case, you should have specified that format and arranged for
>>>> the compiler to enforce it.  At the moment there are a myriad of ways
>>>> (some of them quite ugly) that a programmer can deviate from "the one
>>>> true way" to format code without having the compiler reject it.
>>>
>>> Having the compiler enforce the formatting would just waste
>>> compilation time which is better used for other things.  In any case
>>> gofmt does not specify the position of every character; it just
>>> applies certain formatting rules.
>>
>> Then why use the "one true style" argument for supporting the problems
>> caused by ';' injection?
>
> You have changed the sense of my words, which are quoted above.  I
> didn't use the "one true style" argument for supporting the problems
> caused by ';' injection.  I used it to explain why those problems are
> not a "show stopper," and why it is not necessary to eliminate them in
> order for Go to be taken seriously.  If all code is formatted the same
> way (approximately) then there is no semicolon issue.  The semicolon
> issue only arises in code which is not formatted in the recommended
> style.

That's a very big "only".  I can't think of any other languages where 
something like this arises.  As I said to Russ, I look forward to 
reading your SIGPLAN article explaining all of this.

Because you're too lazy to type a few ';'s or too lazy to implement ';' 
elision properly you're ruining what was a very promising new 
programming language.  I think that's a pity.

I wish you good luck with Go but (for me) the frustrations are 
outweighing the benefits.  Back to C and Python for me.

Let me know if you ever get a sensible resolution to this issue?

Thanks
Peter

[rsc, 2010-06-25 14:14:55]

>> You may disagree with them, but I don't think you can argue
>> that they're either irrational or inconsistent.  We've given you
>> the rationale,
>
> rationale != rational

thanks.

>> repeatedly, and the rule is nothing if not consistent:
>
> It's the consequences which appear inconsistent to a human.  I draw your
> attention to the list of possible "for" statements that are acceptable and
> those that aren't that I've presented before.

the consequences are entirely consistent if you know the rule.
if you don't know the rule you don't know the language.
given var x byte,

x == 254 => x+1 == 255
x == 255 => x+1 == 0

that's not very consistent either at some level, but you and i
find it okay because we know the rule.

>> if the line ends with one of the line-ending tokens, you get a
>> semicolon for free.  Always.
>
> Well it's a computer, isn't it.
>
> I look forward to your SIGPLAN article where you justify this situation.

i'm pretty tired of this attitude.

if you come back with another proposal, i'll look at it,
but i'm done with the discussion of this one.

russ

[pwil3058, 2010-06-26 00:20:25]

On 25/06/10 12:05, Peter Williams wrote:
> On 25/06/10 03:24, Russ Cox wrote:
>>> The reason it won't be taken seriously are not that statement but its
>>> consequences: irrational and inconsistent source format restrictions.
>>
>> You may disagree with them, but I don't think you can argue
>> that they're either irrational or inconsistent. We've given you
>> the rationale,
>
> rationale != rational
>
>> repeatedly, and the rule is nothing if not consistent:
>
> It's the consequences which appear inconsistent to a human. I draw your
> attention to the list of possible "for" statements that are acceptable
> and those that aren't that I've presented before.
>
>> if the line ends with one of the line-ending tokens, you get a
>> semicolon for free. Always.
>
> Well it's a computer, isn't it.
>
> I look forward to your SIGPLAN article where you justify this situation.
>
>>
>>> No comment yet on whether you really want people to be able to use
>>> new lines
>>> instead of ';' in "if", "for" and "switch" statements? I'm inferring
>>> from
>>> gofmt's behaviour that you'd rather they didn't.
>>
>> My favorite thing about Go is that it explores the result of applying
>> simple rules consistently. The case-based visibility is a great
>> example of this. If you look back in the Mercurial history you'll
>> find a time when top-level declarations were tagged with
>> "export" or "package" to make them exported or package-local
>> (as opposed to file-local). It was a simple rule and seemed to
>> work fine, but it didn't give us a nice way to handle struct fields.
>> (Annotating every struct field with export or package was too
>> heavy on the page.) After a lot of discussion, we agreed to try
>> the case-based visibility rule instead. We gave up file-local
>> visibility entirely, gave up uppercase local constants and types,
>> and in return got a simpler, lighter weight rule that provided the
>> needed functionality (control over fields). It felt weird at the time,
>> probably because we were so used to the C++ and Java
>> approaches, but in retrospect I think it is one of Go's best tiny
>> features. When I look at code in C++ or Java now, it frustrates
>> me that I can't tell from a use of the name whether it is a
>> public name or private name. I see a call to DangerousMethod()
>> and think "wow, I hope external callers can't call that method!".
>> And then I have to spend time looking around to find out.
>> We also went through a handful of other proposals that were
>> rejected as not good enough yet, and in retrospect they weren't.
>> And if we'd settled for any of those we wouldn't have gotten
>> to where we are today.
>
> I mostly agree with your argument here except for your choice of lower
> case characters as the indicator as it reduces the namespace. Did you
> consider using "_" as the first character for local variables? Or if
> that's not viable some other character?
>
> In a lot of fields, the capitalization of the first character of a name,
> abbreviation or symbol conveys meaning and you've stifled that a little
> with the current model.

By the way, if this failed attempt to demonstrate your brilliance was 
intended to justify your other poor decisions that is an invalid ploy. 
Each decision has to stand on its own.

Peter

[pwil3058, 2010-06-26 00:42:59]

On 26/06/10 00:14, Russ Cox wrote:
>>> You may disagree with them, but I don't think you can argue
>>> that they're either irrational or inconsistent.  We've given you
>>> the rationale,
>>
>> rationale != rational
>
> thanks.
>
>>> repeatedly, and the rule is nothing if not consistent:
>>
>> It's the consequences which appear inconsistent to a human.  I draw your
>> attention to the list of possible "for" statements that are acceptable and
>> those that aren't that I've presented before.
>
> the consequences are entirely consistent if you know the rule.
> if you don't know the rule you don't know the language.
> given var x byte,
>
> x == 254 =>  x+1 == 255
> x == 255 =>  x+1 == 0
>
> that's not very consistent either at some level, but you and i
> find it okay because we know the rule.

That is very consistent with the way 1's compliment arithmetic works.

This is legal:

     for
         i := 0
         i < 10
         i++ {
     }
     for
         i := 0
         i < 10
     {
         i++
     }
     for i := 0; i < 10; i++ {
     }
     for i := 0; i < 10;
     {
         i++
     }
     for ;;
     {
     }

but not:

     for i := 0; i < 10; i++
     {
     }

is not consistent with anything but the refusal of people to admit they 
failed to successfully implement ';' elision and give up on it.

>
>>> if the line ends with one of the line-ending tokens, you get a
>>> semicolon for free.  Always.
>>
>> Well it's a computer, isn't it.
>>
>> I look forward to your SIGPLAN article where you justify this situation.
>
> i'm pretty tired of this attitude.
>
> if you come back with another proposal, i'll look at it,
> but i'm done with the discussion of this one.

Don't worry I'm giving up.  You all obviously have too much invested in 
your bad decisions to look at them rationally and fix the problems they 
cause.  I'm wasting my time trying to convince you otherwise.

Thanks for the constructive/technical feedback on the patch.  Oops, I'm 
sorry there was none.

Good luck
Peter
PS Good to see Qtvali back in golang-nuts.  He's got the right level of 
rationality to fit right in with you guys.

[rsc1, 2010-08-03 20:08:06]

removing reviewers; timed out