parser.yy: rearrange to allow more lenient use of music arguments for music functions.

parser.yy: rearrange to allow more lenient use of music arguments for music functions.

This change may be somewhat contentious: it removes a lot of
opportunities for syntax errors by allowing a lot of music functions
to work that did not do so previously.

For one thing, you can use a ly:music? style signature as often as you
want to in all music functions.  There are some cases where it will
only be accepted in "closed form", namely as a music identifier or a
music expression enclosed in { ... } or << ... >>.  Those cases are
before a post-event not belonging to the music argument, or a duration
argument.

Music functions can be used in a number of places with different
semantics, not necessarily the cleanest thing.  When they are used as
post-events and have a music argument last, they will actually try
getting a post-event as that music argument.  What is new (and
probably less than fabulous) that they will instead accept a closed
form music expression.

However, when such a function is used as a primary music event, its
last argument will be a regular music expression anyway.  So it is not
like this argument dichotomy was really all that novel.

In a similar vein, as a chord constituent, a music function will
accept a chord constituent in the place of a prospective last music
argument.

It does not seem overly satisfactory that only the last such component
will be syntactically warped according to the surrounding.  Perhaps
one should use different signatures in the first place.

Suggestions welcome.

[dak, 2011-07-29 05:51:35]

I should point out that I consider this a merge candidate.  AFAICT, it passes
the regtests.

[MikeSol, 2011-07-29 07:17:19]

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
File Documentation/extending/programming-interface.itely (right):

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
Documentation/extending/programming-interface.itely:146: 
Can you give a more lilypond examples as you do above (with s 1*0-\fun).  I
think that is very clear, and more examples like that would help understand this
faster.

http://codereview.appspot.com/4815052/diff/2001/lily/parser.yy
File lily/parser.yy (right):

http://codereview.appspot.com/4815052/diff/2001/lily/parser.yy#newcode1598
lily/parser.yy:1598: }
I think this points to a larger issue of the information that enclosing
characters (brackets, parentheses, etc.) carry in LilyPond.  Is there a way,
like in LaTeX with verb, that people can construct their own begininning and end
delimiters to help the parser?  This would not only be useful here, but in cases
like \musicFunction { a \times 2/3 { a b } c } which in LilyPond would compile
the tuplet on two notes.  However, if we could do \musicFunction ={ a \times 2/3
{ a b =} c }, then the parser could know that ={ and =} are supposed to go into
the music function.

[dak, 2011-07-29 07:51:08]

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
File Documentation/extending/programming-interface.itely (right):

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
Documentation/extending/programming-interface.itely:146: 
On 2011/07/29 07:17:19, MikeSol wrote:
> Can you give a more lilypond examples as you do above (with s 1*0-\fun).  I
> think that is very clear, and more examples like that would help understand
this
> faster.

Don't get me wrong: I don't think that this documentation is of good quality. 
But to a large part, it is not documenting what I have changed, but rather what
has been there for a long time without any documentation.

If you take a look at music-functions.scm, you'll get a few eye-openers about
what was possible previously.

So my main priority with regard to the docs is getting them to be reasonably
complete.  Turning complete documentation into good documentation is then more
of a frog's task.  Or at least it is mostly a separate issue than what this
patch is about.

Personally, I'd like things like #{ -. #} to work, namely let #{ #} be able to
surround more than just serial music, a reasonable subset of what we can do in
assignments or top-level expressions.

http://codereview.appspot.com/4815052/diff/2001/lily/parser.yy
File lily/parser.yy (right):

http://codereview.appspot.com/4815052/diff/2001/lily/parser.yy#newcode1598
lily/parser.yy:1598: }
On 2011/07/29 07:17:19, MikeSol wrote:
> I think this points to a larger issue of the information that enclosing
> characters (brackets, parentheses, etc.) carry in LilyPond.

The problem is rather that music functions can currently accept a music
expression not enclosed with braces or anything else at all.  One reason is that
such music expressions can serve as a poor man's substitute for pitch arguments,
and are used for that.  Perhaps this can be changed in future (we have pitch
arguments now), but currently this would likely break too much existing code.

> Is there a way,
> like in LaTeX with verb, that people can construct their own begininning and
end
> delimiters to help the parser?

Not really.

> This would not only be useful here, but in cases
> like \musicFunction { a \times 2/3 { a b } c } which in LilyPond would compile
> the tuplet on two notes.  However, if we could do \musicFunction ={ a \times
2/3
> { a b =} c }, then the parser could know that ={ and =} are supposed to go
into
> the music function.

The corresponding data structures can't be broken off in the middle like that.

[Reinhold, 2011-07-29 08:55:24]

First, I have to say that I don't really understand the fine details of the
patch...

Can you give some explicit examples that your patch now allows that didn't work
before (preferrably as a regtest, so we notice when a future parser change
breaks it)?

You are talking about the problems with chord constituents. How does this work
with \tweak, which is explicitly also meant to change single chord constituents?
Or should tweak always be handled as a special case and not make use of your
parser changes?

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
File Documentation/extending/programming-interface.itely (right):

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
Documentation/extending/programming-interface.itely:122: @item
Please leave a blank line befor @item.

[dak, 2011-07-29 10:24:42]

On 2011/07/29 08:55:24, Reinhold wrote:
> First, I have to say that I don't really understand the fine details of the
> patch...

Makes music functions more predictable and versatile.  This started with a patch
I already pushed, implementing more argument types.

The documentation would have been quite more contorted if I had properly
documented the state previous to this patch.  I don't really want to write that
up.

> Can you give some explicit examples that your patch now allows that didn't
work
> before (preferrably as a regtest, so we notice when a future parser change
> breaks it)?

Well, one could now move a few things like \relative and \transpose into music
functions (and Scheme-callable functions for the hard work, for efficiency's
sake).

That would be a quite reliable regtest...

> You are talking about the problems with chord constituents. How does this work
> with \tweak, which is explicitly also meant to change single chord
constituents?

It worked previously because \tweak was a special case that matched the previous
semantics reasonably well.  \tweak works just the same as it did previously, but
it is less surprising that it does so.

> Or should tweak always be handled as a special case and not make use of your
> parser changes?

\tweak still works just the same.  But you could, for example, write something
like
\staffposition-at d'
that resulted in a tweak of the next element to have it land in the position of
a d'.

A music function like that was just not possible previously since it would have
been beyond the narrow pattern of things that were implemented.

[dak, 2011-07-29 10:25:17]

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
File Documentation/extending/programming-interface.itely (right):

http://codereview.appspot.com/4815052/diff/2001/Documentation/extending/progr...
Documentation/extending/programming-interface.itely:122: @item
On 2011/07/29 08:55:26, Reinhold wrote:
> Please leave a blank line befor @item.

Done

[dak, 2011-07-29 14:42:08]

I decided to push this since it forms a reasonably capsulated piece of work.  I
agree that the usage documentation is not yet in a state desirable for tutoring.
 However, I am still meddling with stuff like #{ ... #} in a manner that is both
upwards-compatible as well as a definite improvement, and that changes the
situation in a manner that makes the user-level interfaces more straightforward,
and consequently nicer for documentation.  So I would not want to invest too
much work at the current point of time into documenting details and limitations
that I don't want to stay.

The change in its current state is, in my opinion, a strict improvement both in
the consistency as well as the documentation of the user-accessible behavior. 
Neither are at an optimum, but getting there would not seem to require
backtracking.  So I am getting this out of my queue.

[reinhold_kainhofer.com, 2011-07-29 16:40:50]

Am Freitag, 29. Juli 2011, 12:24:43 schrieb dak@gnu.org:
> On 2011/07/29 08:55:24, Reinhold wrote:
> > Can you give some explicit examples that your patch now allows that
> > didn't work before (preferrably as a regtest, so we notice when a future
> > parser change breaks it)?
> 
> Well, one could now move a few things like \relative and \transpose into
> music functions (and Scheme-callable functions for the hard work, for
> efficiency's sake).
> 
> That would be a quite reliable regtest...

Actually, I think we should add a really simple test case to the regtests, 
like your example of \staffposition-at d' below.

> \tweak still works just the same.  But you could, for example, write
> something like
> \staffposition-at d'
> that resulted in a tweak of the next element to have it land in the
> position of a d'.

Can you turn that "could" into real lilypond code in a regtest, so that (1) we 
have a simple test case to check whether future parser changes affect this and 
(2) people like me, who easier understand things by examples, see what this 
patch achieves?

Quite often, I have found that trying to create a very simple regtest, I found 
several problems in my patches, because when thinking theoretically I might 
have missed how people would actually use the features.

Cheers,
Reinhold
-- 
------------------------------------------------------------------
Reinhold Kainhofer, reinhold@kainhofer.com, http://reinhold.kainhofer.com/
 * Financial & Actuarial Math., Vienna Univ. of Technology, Austria
 * http://www.fam.tuwien.ac.at/, DVR: 0005886
 * LilyPond, Music typesetting, http://www.lilypond.org

[dak, 2011-07-29 17:24:29]

Reinhold Kainhofer <reinhold@kainhofer.com> writes:

> Am Freitag, 29. Juli 2011, 12:24:43 schrieb dak@gnu.org:
>> On 2011/07/29 08:55:24, Reinhold wrote:
>> > Can you give some explicit examples that your patch now allows that
>> > didn't work before (preferrably as a regtest, so we notice when a future
>> > parser change breaks it)?
>> 
>> Well, one could now move a few things like \relative and \transpose into
>> music functions (and Scheme-callable functions for the hard work, for
>> efficiency's sake).
>> 
>> That would be a quite reliable regtest...
>
> Actually, I think we should add a really simple test case to the regtests, 
> like your example of \staffposition-at d' below.
>
>> \tweak still works just the same.  But you could, for example, write
>> something like
>> \staffposition-at d'
>> that resulted in a tweak of the next element to have it land in the
>> position of a d'.
>
> Can you turn that "could" into real lilypond code in a regtest, so
> that (1) we have a simple test case to check whether future parser
> changes affect this and (2) people like me, who easier understand
> things by examples, see what this patch achieves?
>
> Quite often, I have found that trying to create a very simple regtest,
> I found several problems in my patches, because when thinking
> theoretically I might have missed how people would actually use the
> features.

Straightforward to do (I did it but did not keep the code): take the
code for \tweak in music-functions.ly and tweak it in the
straightforward way.  Works.

But that is not a readable example.  So I want to modify #{...#} first
so that I can use it for wrapping \tweak insteak of copying its code.

-- 
David Kastrup