Allow quoted identifiers like to be used like \"violin1", not just defined.

Allow quoted identifiers like to be used like \"violin1", not just defined.

Consists of the following four commits:


Add regtest for quoted identifiers


Allow quoted identifiers like to be used like \"violin1", not just defined.

A frequent complaint is the absence of identifiers with numbers in
them, like violin1.

Defining such identifiers has always been possible with

    "violin1" = { c''4 c c c }

This patch lets one actually use them by calling them with

    \"violin1"

lexer.ll: duplicate a few quotes in character sets to help syntax highlighting

This uses regular expressions like [^''] rather than [^'] in order to keep
the confusion of editors like Emacs tolerable.

lexer.ll: lyric_quote was not necessary as separate state.

[lemzwerg, 2012-10-26 13:58:41]

LGTM.

[marc, 2012-10-28 07:46:36]

On 2012/10/26 13:58:41, lemzwerg wrote:
> LGTM.

+1

Nice to have this feature available!

[dak, 2012-10-28 08:25:25]

On 2012/10/28 07:46:36, marc wrote:
> On 2012/10/26 13:58:41, lemzwerg wrote:
> > LGTM.
> 
> +1
> 
> Nice to have this feature available!

It is more a pacifier than a feature since it does not actually add
any functionality.  And I don't want to see anything in our code tree
(except possibly the documentation for it itself) making use of it,
just like I don't want to see durations written as 4 . . .  anywhere.

I am not unconvinced this is a good idea.  It just addresses a pet
peeve in some kind of manner.

[t.daniels_treda.co.uk, 2012-10-28 08:47:50]

David wrote:

>> Nice to have this feature available!
> 
> It is more a pacifier than a feature since it does not actually add
> any functionality.  And I don't want to see anything in our code tree
> (except possibly the documentation for it itself) making use of it,
> just like I don't want to see durations written as 4 . . .  anywhere.

Maybe "anything except durations"?
 
> I am not unconvinced this is a good idea.  It just addresses a pet
> peeve in some kind of manner.

Double negatives are always tricky ;)

But with those changes I agree :)

Trevor

[dak, 2012-10-28 10:02:50]

On 2012/10/28 08:47:50, t.daniels_treda.co.uk wrote:
> David wrote:
> 
> >> Nice to have this feature available!
> > 
> > It is more a pacifier than a feature since it does not actually add
> > any functionality.  And I don't want to see anything in our code tree
> > (except possibly the documentation for it itself) making use of it,
> > just like I don't want to see durations written as 4 . . .  anywhere.
> 
> Maybe "anything except durations"?

No, I don't want to see durations written as 4 . . . instead of 4... anywhere.

[t.daniels_treda.co.uk, 2012-10-28 10:08:54]

> On 2012/10/28 08:47:50, t.daniels_treda.co.uk wrote:
>> David wrote:
> 
>> >> Nice to have this feature available!
>> >
>> > It is more a pacifier than a feature since it does not actually add
>> > any functionality.  And I don't want to see anything in our code
> tree
>> > (except possibly the documentation for it itself) making use of it,
>> > just like I don't want to see durations written as 4 . . .
> anywhere.
> 
>> Maybe "anything except durations"?
> 
> No, I don't want to see durations written as 4 . . . instead of 4...
> anywhere.

Ah, right.  That subtlety escaped me.  I agree with that too.

Trevor

[marc, 2012-10-28 14:38:18]

Am 28.10.2012 09:25, schrieb dak@gnu.org:
> Reviewers: lemzwerg, marc,
>
> Message:
> On 2012/10/28 07:46:36, marc wrote:
>> On 2012/10/26 13:58:41, lemzwerg wrote:
>> > LGTM.
>
>> +1
>
>> Nice to have this feature available!
>
> It is more a pacifier than a feature since it does not actually add
> any functionality.
Well, sometimes variables like \"violin1" come in handy.
> And I don't want to see anything in our code tree
> (except possibly the documentation for it itself) making use of it,
+1
> just like I don't want to see durations written as 4 . . .  anywhere.
+1

Regards,

Marc

[Keith, 2012-10-28 17:38:10]

http://codereview.appspot.com/6778055/diff/1/lily/lexer.ll
File lily/lexer.ll (right):

http://codereview.appspot.com/6778055/diff/1/lily/lexer.ll#newcode95
lily/lexer.ll:95: The inside of \"violin1" is marked by nesting two quote modes
as we
A start-condition named "command_quote" would be less mysterious than having
special meaning for "quote" within "quote".

That is, start-conditions <quote,command_quote> for scanning the innards of
quotes, then
<command_quote> \" {/* pop one state and return the command */}

[dak, 2012-10-28 18:05:20]

On 2012/10/28 17:38:10, Keith wrote:
> http://codereview.appspot.com/6778055/diff/1/lily/lexer.ll
> File lily/lexer.ll (right):
> 
> http://codereview.appspot.com/6778055/diff/1/lily/lexer.ll#newcode95
> lily/lexer.ll:95: The inside of \"violin1" is marked by nesting two quote
modes
> as we
> A start-condition named "command_quote" would be less mysterious than having
> special meaning for "quote" within "quote".
> 
> That is, start-conditions <quote,command_quote> for scanning the innards of
> quotes, then
> <command_quote> \" {/* pop one state and return the command */}

Well, code duplication.  Avoiding code duplication would mean having to have a
"command" mode to return to, just for executing the command and popping and
returning again.  I'll take a closed look at just how complex the duplication
would be.  Maybe it is not that bad.

[Keith, 2012-10-30 04:06:18]

Well, David has pointed out some shortcomings, but I am not sure if they are
practical problems.

The strings in
  "violin1" = c2   { "\violin1" }
must be literal strings, whereas most other places we can
build strings in Scheme.  So the instrumentName setting here works, but the
corresponding variable names do not
  $(format #f "violin~a" 2) = {c' d' e' f'}
  \new Staff \with {
     instrumentName = $(format #f "Violin ~a" (+ 1 1))
  } \$(format #f "violin~a" (+ 1 1))
So the superficial similarity with strings breaks down as soon as you try
putting it to the test.

The valid names for identifiers are no longer a simple lexer pattern, but
anything that can be put between quotes, making syntax highlighting more
difficult.
  "\\" = c''1  { \"\\" }

There is apparently a more powerful array implementation in the works, that will
allow simply \violin1

And finally, we have the choice discussed above between code-duplication and
abuse of the Flex start-condition system.  

By comparison, a simple change to the scan patterns
http://codereview.appspot.com/6493072/diff/19001/lily/lexer.ll
would be much cheaper and better-confined.

[dak, 2012-10-30 05:43:34]

On 2012/10/30 04:06:18, Keith wrote:
> Well, David has pointed out some shortcomings, but I am not sure if they are
> practical problems.

A "practical problem" presumably is one the user can be bothered about.  I never
claimed that problem class here.  Users are not interested in language design or
consistency or programmers having to work overtime fixing problems and choosing
bad compromises, or something being complex to document or explain (one would
notice that only when reading documentation, anyway).

> The strings in
>   "violin1" = c2   { "\violin1" }
> must be literal strings,

You mean \"violin1" here.  Yes, quite so, but if that is a real concern to you
and not just a mock complaint, both are easy to change.  On the left side of the
assignment, rather trivially, on the right side, it requires implementing \$ in
the lexer which is perfectly feasible.

If you feel this is a dealbreaker to you, file the respective issue request and
it can be changed simply enough.

> The valid names for identifiers are no longer a simple lexer pattern, but
> anything that can be put between quotes, making syntax highlighting more
> difficult.
>   "\\" = c''1  { \"\\" }

Yes, I pointed out that syntax highlighting will be affected.  This was
discussed in the mailing list when I first mentioned this possibility in the
course of the GLISS discussion, and people were all "so what?" about it.

Again: I don't consider \"..." a fabulous idea.  But it does not affect existing
syntax.  Making \"..." accept normal string escapes is not necessary and can be
replaced by just admitting an uninterpreted string if you can give a good reason
for it.  I took that choice for symmetry considerations, but not interpreting
string escapes would actually simplify the code.

> There is apparently a more powerful array implementation in the works, that
will
> allow simply \violin1

Sure.

> And finally, we have the choice discussed above between code-duplication and
> abuse of the Flex start-condition system.

Uh no.  As you pointed out, one just needs another start condition (which is not
an "abuse") to change the original code which you considered an abuse, and I was
planning to change this, according to your suggestion, before committing anyway,
just got not around to it yet since I spent too much time on silly discussions.

At any rate, that's about 5 lines of duplication, and if they were so totally
worrisome to you, we also could have wrapped those in a function.  But the
additional start state, as you proposed, seems a cleaner solution.

> By comparison, a simple change to the scan patterns
> http://codereview.appspot.com/6493072/diff/19001/lily/lexer.ll
> would be much cheaper and better-confined.

Oh, it _is_ cheap in code and the _code_ is well-confined (not that the code for
\"..." would be spread out significantly more).  The price is to be paid in
documentation and semantic inconsistency and complexity.  The latter is not
well-confined.

[dak, 2012-10-30 06:05:57]

Actually, I would be perfectly fine with binning both \violin.1 as well as
\"violin1".  I am just afraid it would be a bit like taking candy from a baby.

But if people are fine with the prospect of waiting until \violin1, \violin 1,
\violin $(1+1) will all work, then that's likely the cleanest solution. 
Definitely the one with the least impact on syntax highlighting.

[Keith, 2012-10-30 06:07:10]

On Mon, 29 Oct 2012 22:43:34 -0700, <dak@gnu.org> wrote:

> if that is a real concern to you and not just a mock complaint,

Of course these are mock complaints; you gave me so much to mock.

This should be fine, after adding the start-condition in the scanner.

[Keith, 2012-10-30 06:17:30]

On 2012/10/30 06:05:57, dak wrote:
> Actually, I would be perfectly fine with binning both \violin.1 
> as well as \"violin1".  

That should be fine.  No-one has indicated they would actually use either of
these.  (But I do use "vn1mvt2" with $vn1mvt2 )

[dak, 2012-10-30 07:22:29]

On 2012/10/30 06:17:30, Keith wrote:
> On 2012/10/30 06:05:57, dak wrote:
> > Actually, I would be perfectly fine with binning both \violin.1 
> > as well as \"violin1".  
> 
> That should be fine.  No-one has indicated they would actually use either of
> these.  (But I do use "vn1mvt2" with $vn1mvt2 )

$vn1mvt2 has the disadvantage of running in lexical closure.  That means that it
does not work for something like
#{ \layout { \context { \Voice ... $vn1mvt2 ... } } #}
since it inherits the lexical scope from outside and does not look inside of the
LilyPond scope.

The scoping is a difference between \... and $... that may be relevant at times,
so there is a bit of an incentive to have a \-like operator.

Rethinking this carefully, a native \violin2 (as an actual array/vector/alist)
should not really be away more than a few months hopefully.

This is not really the same as freeform identifiers (you need to declare the
structure of \violin) but if we find that people can be satisfied with that
solution (which is more flexible in some manners), not opening the somewhat
quirky can of worms with weirder identifier syntax forms might prove acceptable.

If we see that the vector/struct solution will not make people happy, there will
still be enough time to resuscitate one of the existing proposals or come up
with yet another one.  We've gotten along without this for a dozen years or so,
so we won't die from waiting a bit more.

This code/patch is not likely to go stale anytime soon.

[t.daniels_treda.co.uk, 2012-10-30 11:54:35]

David wrote Tuesday, October 30, 2012 6:05 AM


> Actually, I would be perfectly fine with binning both \violin.1 as well
> as \"violin1".  I am just afraid it would be a bit like taking candy
> from a baby.
> 
> But if people are fine with the prospect of waiting until \violin1,
> \violin 1, \violin $(1+1) will all work, then that's likely the cleanest
> solution.  Definitely the one with the least impact on syntax
> highlighting.
...
> Rethinking this carefully, a native \violin2 (as an actual
> array/vector/alist) should not really be away more than a few months
> hopefully.

In that case I'd prefer to wait.

Trevor

[marc, 2012-10-30 20:22:59]

Am 30.10.2012 12:54, schrieb Trevor Daniels:
> David wrote Tuesday, October 30, 2012 6:05 AM
>
>
>> Actually, I would be perfectly fine with binning both \violin.1 as well
>> as \"violin1".  I am just afraid it would be a bit like taking candy
>> from a baby.
>>
>> But if people are fine with the prospect of waiting until \violin1,
>> \violin 1, \violin $(1+1) will all work, then that's likely the cleanest
>> solution.  Definitely the one with the least impact on syntax
>> highlighting.
> ...
>> Rethinking this carefully, a native \violin2 (as an actual
>> array/vector/alist) should not really be away more than a few months
>> hopefully.
> In that case I'd prefer to wait.
>
> Trevor
>
+1
Marc

[dak, 2013-09-27 16:51:58]

Rebased.