Add Staff.midiCC context property, refactor handling of MIDI control changes

Add Staff.midiCC context property, refactor handling of MIDI control changes

Setting the Staff.midiCC context property to a list of (control number,
control value) pairs, where every control number and control value must
be an integer between the inclusive range from 0 to 127, will send the
corresponding raw control changes to the MIDI output on the MIDI channel
associated with the current context.  For example,
\set Staff.midiCC = #'((7 . 100)) will set the MIDI volume control (#7)
to 100.

Most of the logic for handling MIDI control value initialization from
context properties (in Staff_performer::new_audio_staff),
changes (in
Midi_control_function_performer::announce_function_value_change), and
value conversion for output (in
Midi_control_function_value_change::to_string), was moved into the new
Midi_control_change_announcer class.  All MIDI control changes are now
encoded using {Audio,Midi}_control_change items.  This change makes the
old {Audio,Midi}_control_function_value_change classes obsolete.

[ht, 2016-01-16 18:08:08]

Hi,

This patch introduces an interface for adjusting the values of all MIDI
controllers responding to Control Change events from within LilyPond input
using context properties.  Instead of adding a new context property for
every controller, however, this change adds a generic Staff.midiCC context
property, which accepts a list of (control number, value) pairs, where the
control number and the value are integers between 0 and 127 (inclusive).
Setting the context property will generate the corresponding "raw" control
change events in the MIDI output.

In hindsight, the generic context property would have provided (from a
purely technical standpoint) an interface sufficient for making changes to
all of the MIDI controls for whose adjustment I have previously added
support using controller-specific context properties (Staff.midiBalance,
Staff.midiExpression, Staff.midiPanPosition, Staff.midiReverbLevel and
Staff.midiChorusLevel).  I realize that the introduction of another context
property whose value syntax differs from that of the existing ones, but
whose behavior still overlaps with them, will reduce the overall
consistency of the design, but in my opinion there are the following
arguments for making this change:

* The main single benefit of the new context property is that it enables
  making changes to the values of any MIDI controllers adjustable using
  MIDI control change events from within LilyPond input files without
  polluting the LilyPond context property namespace.  (Even after this
  change, there still remain MIDI controls whose values cannot be modified
  from within LilyPond input, such as Channel Pressure or Pitch Wheel -
  however, the current patch concerns only enhancing support for MIDI CC
  events.)

* There are no changes added which would _require_ setting the context
  property in any existing LilyPond files: just as with the other MIDI
  context properties, users need to concern themselves with the new
  property only if they need it.

* This patch simplifies the implementation of handling the existing MIDI
  context properties by reducing their implementation to use the same data
  structures for outputting MIDI control changes.  Adding support for even
  new controller-specific context properties should be as easy (or even
  easier) as before because the definitions of the controller-specific
  context properties are now located in a single file
  (lily/midi-cc-announcer.cc) instead of being scattered over several
  Audio_item and Midi_item subclasses.


In my opinion, the alternative to adding a generic context property for
generating MIDI control changes - that is, adding new context properties
separately each controller - would only lead to a burden of having to write
a lot of new technical documentation (which would have more to do with the
MIDI standard, not LilyPond) because:
  
- the names of MIDI controllers do not appear to be very strictly defined,
  so the context properties could not be named without documenting also
  their associated MIDI controller numbers (unless the numbers were embedded
  in the property names, of course - however, this option would probably
  make the names so generic that having separate context properties for
  every controller would bring little benefit over the Staff.midiCC
  property);

- on the other hand, due to the genericity of many of the names of
  controllers (<http://www.midi.org/techspecs/midimessages.php>), and the
  freedom given by the standard as regards their function, choosing too
  specific names for the context properties would likely result in the names
  not matching the actual function of the corresponding controllers well
  across different MIDI devices; and

- each context property would need a decision on how to represent its value
  in LilyPond input.  However, any value representation scheme that does not
  use "raw" MIDI values (integers between 0 and 127), could make it harder
  for advanced MIDI users who wish to use particular integer values of their
  own choosing for MIDI output: with custom schemes for specifying values
  for the context properties, having a particular integer value outputted in
  MIDI would not be possible without precise knowledge about how LilyPond
  handles the values of the context properties, in order to do reverse
  calculations.  (I do not claim that custom value types would necessarily
  be bad for all controllers: for example, a context property which accepts
  Boolean values could be a natural choice for controller types that only
  have an "on" or "off" setting.)


As this patch will not remove any existing functionality, but only extends
it, I see no reason to remove support for the controller-specific context
properties, even though the behavior of the new context property overlaps
with that of the existing ones.  The controller-specific properties could be
seen just as a convenience interface for adjusting the values of "common"
controllers, and even new controller-specific properties could be defined
later if necessary, whereas the generic Staff.midiCC property is provided to
allow making direct adjustments even to more rarely used (or MIDI device
specific) controllers if necessary.

A few small additional notes about the design of the changes in the patch:

* The current implementation of the Staff.midiCC context property always
  requires a list as the context property value.  It would be possible to
  add support for accepting single (control number, value) pairs as
  themselves, but I think this would require extending scm/c++.scm and
  scm/lily.scm with an "integer-pair-or-integer-pair-list" predicate, which
  to me feels like too specific a predicate to warrant being added to the
  public definitions.  (I guess one cannot use lambda expressions as
  context property predicates in scm/define-context-properties.scm?)

* When updating the Changes document, I did not put the description of the
  new context property to the top of the file, but added it after the
  description of the Staff.midiExpression context property - I think it is
  more logical to describe these context properties together.

[dan_faithful.be, 2016-01-16 18:50:43]

On Jan 16, 2016, at 13:08 , ht.lilypond.development@gmail.com wrote:
> 
> This patch introduces an interface for adjusting the values of all MIDI
> controllers responding to Control Change events from within LilyPond
> input
> using context properties.

Does this allow single instantaneous changes only, or does it support computing
a series of small changes between two user-specified endpoints?
— 
Dan

[ht, 2016-01-16 18:55:29]

On 2016/01/16 18:50:43, dan_faithful.be wrote:
> On Jan 16, 2016, at 13:08 , mailto:ht.lilypond.development@gmail.com wrote:
> > 
> > This patch introduces an interface for adjusting the values of all MIDI
> > controllers responding to Control Change events from within LilyPond
> > input
> > using context properties.
> 
> Does this allow single instantaneous changes only, or does it support
computing
> a series of small changes between two user-specified endpoints?
> — 
> Dan
> 

I'm afraid that this patch supports only instantaneous changes (just like the
other MIDI context properties).  A series of changes could be generated using a
music function, however.

[dak, 2016-01-21 15:21:22]

On 2016/01/16 18:08:08, ht wrote:
> Hi,
> 
> This patch introduces an interface for adjusting the values of all MIDI
> controllers responding to Control Change events from within LilyPond input
> using context properties.  Instead of adding a new context property for
> every controller, however, this change adds a generic Staff.midiCC context
> property, which accepts a list of (control number, value) pairs, where the
> control number and the value are integers between 0 and 127 (inclusive).
> Setting the context property will generate the corresponding "raw" control
> change events in the MIDI output.
> 
> In hindsight, the generic context property would have provided (from a
> purely technical standpoint) an interface sufficient for making changes to
> all of the MIDI controls for whose adjustment I have previously added
> support using controller-specific context properties (Staff.midiBalance,
> Staff.midiExpression, Staff.midiPanPosition, Staff.midiReverbLevel and
> Staff.midiChorusLevel).  I realize that the introduction of another context
> property whose value syntax differs from that of the existing ones, but
> whose behavior still overlaps with them, will reduce the overall
> consistency of the design, but in my opinion there are the following
> arguments for making this change:
> 
> * The main single benefit of the new context property is that it enables
>   making changes to the values of any MIDI controllers adjustable using
>   MIDI control change events from within LilyPond input files without
>   polluting the LilyPond context property namespace.  (Even after this
>   change, there still remain MIDI controls whose values cannot be modified
>   from within LilyPond input, such as Channel Pressure or Pitch Wheel -
>   however, the current patch concerns only enhancing support for MIDI CC
>   events.)
> 
> * There are no changes added which would _require_ setting the context
>   property in any existing LilyPond files: just as with the other MIDI
>   context properties, users need to concern themselves with the new
>   property only if they need it.
> 
> * This patch simplifies the implementation of handling the existing MIDI
>   context properties by reducing their implementation to use the same data
>   structures for outputting MIDI control changes.  Adding support for even
>   new controller-specific context properties should be as easy (or even
>   easier) as before because the definitions of the controller-specific
>   context properties are now located in a single file
>   (lily/midi-cc-announcer.cc) instead of being scattered over several
>   Audio_item and Midi_item subclasses.
> 
> 
> In my opinion, the alternative to adding a generic context property for
> generating MIDI control changes - that is, adding new context properties
> separately each controller - would only lead to a burden of having to write
> a lot of new technical documentation (which would have more to do with the
> MIDI standard, not LilyPond) because:
>   
> - the names of MIDI controllers do not appear to be very strictly defined,
>   so the context properties could not be named without documenting also
>   their associated MIDI controller numbers (unless the numbers were embedded
>   in the property names, of course - however, this option would probably
>   make the names so generic that having separate context properties for
>   every controller would bring little benefit over the Staff.midiCC
>   property);
> 
> - on the other hand, due to the genericity of many of the names of
>   controllers (<http://www.midi.org/techspecs/midimessages.php>), and the
>   freedom given by the standard as regards their function, choosing too
>   specific names for the context properties would likely result in the names
>   not matching the actual function of the corresponding controllers well
>   across different MIDI devices; and
> 
> - each context property would need a decision on how to represent its value
>   in LilyPond input.  However, any value representation scheme that does not
>   use "raw" MIDI values (integers between 0 and 127), could make it harder
>   for advanced MIDI users who wish to use particular integer values of their
>   own choosing for MIDI output: with custom schemes for specifying values
>   for the context properties, having a particular integer value outputted in
>   MIDI would not be possible without precise knowledge about how LilyPond
>   handles the values of the context properties, in order to do reverse
>   calculations.  (I do not claim that custom value types would necessarily
>   be bad for all controllers: for example, a context property which accepts
>   Boolean values could be a natural choice for controller types that only
>   have an "on" or "off" setting.)
> 
> 
> As this patch will not remove any existing functionality, but only extends
> it, I see no reason to remove support for the controller-specific context
> properties, even though the behavior of the new context property overlaps
> with that of the existing ones.  The controller-specific properties could be
> seen just as a convenience interface for adjusting the values of "common"
> controllers, and even new controller-specific properties could be defined
> later if necessary, whereas the generic Staff.midiCC property is provided to
> allow making direct adjustments even to more rarely used (or MIDI device
> specific) controllers if necessary.
> 
> A few small additional notes about the design of the changes in the patch:
> 
> * The current implementation of the Staff.midiCC context property always
>   requires a list as the context property value.  It would be possible to
>   add support for accepting single (control number, value) pairs as
>   themselves, but I think this would require extending scm/c++.scm and
>   scm/lily.scm with an "integer-pair-or-integer-pair-list" predicate, which
>   to me feels like too specific a predicate to warrant being added to the
>   public definitions.  (I guess one cannot use lambda expressions as
>   context property predicates in scm/define-context-properties.scm?)
> 
> * When updating the Changes document, I did not put the description of the
>   new context property to the top of the file, but added it after the
>   description of the Staff.midiExpression context property - I think it is
>   more logical to describe these context properties together.

I am not happy about the increasing abuse of context properties for not actually
setting context properties but rather sending Midi messages.  The previous
interfaces at least actually set a queryable property.  This one takes a
pair/value pair instead and only sets a property in the Midi controller but not
in the context and one cannot actually query this property again.

I think that this is not a good idea.  Suggestions how we could rather do this? 
Possible mechanisms would include Midi-specific music events, or getting the
equivalent of override/revert for Midi (mobs then instead of grobs).  Of course,
if we had mobs, it would make sense to map a _lot_ of the current Midi
generation through them eventually because it would make programming the Midi
generation much more transparent.

[ht, 2016-01-23 17:20:43]

On 2016/01/21 15:21:22, dak wrote:
> I am not happy about the increasing abuse of context properties for not
actually
> setting context properties but rather sending Midi messages.  The previous
> interfaces at least actually set a queryable property.  This one takes a
> pair/value pair instead and only sets a property in the Midi controller but
not
> in the context and one cannot actually query this property again.

I understand that the behavior introduced by the patch is likely
stretching the meaning of a "context property" too far if one considers
all the technical implications.

(As my level of expertise with LilyPond internals is at a level where I
hardly feel like knowing enough about the technical differences between
grob and context properties to consider myself qualified to even speaking
of them in any of the documentation I've written about the MIDI
enhancements :-) [my experiences with these differences are limited to once
trying and failing to access values of context properties in music
functions, only to conclude that context properties "probably just don't
work that way"], I've learned to treat context properties myself mostly
as "write-only" things in everyday use.  I understand that the situation
could be different for advanced users and developers writing new music
iterators [I hope that's the correct term].  I guess these advanced users
are those who will suffer most from breaking the consistency of how
context properties work.)

At a great risk of not exactly understanding what you meant by "This
one [...] only sets a property in the Midi controller but not in the
context and one cannot actually query this property again": isn't it
still the case that the context property will have the list of the most
recent \set as its value at any given point?  In that case, would it
be possible to implement setting the value of the context property as
updating entries in an alist (and generating the corresponding MIDI
control changes as a side effect) without removing any existing
entries, so that querying the value of the context property would
return the list of all settings in effect (as this list of settings
would probably best reflect also the state of the controllers on a
MIDI device)?  Of course, changing the interpretation of what it means
to set the value of a context property, even if it were technically
possible, is not much better for consistency (and there would be no
way to remove entries in the alist without extending the syntax, for
example, with a special controller "value", with the question of
how the removal of an entry should be reflected in MIDI).

Of course, also the conflict in the interaction between midiCC and
the previously introduced MIDI context properties is even more apparent
when one considers setting and reading the values of the properties
which would be settable in MIDI in multiple ways after this change.

> I think that this is not a good idea.  Suggestions how we could rather do
this? 
> Possible mechanisms would include Midi-specific music events, or getting the
> equivalent of override/revert for Midi (mobs then instead of grobs).  Of
course,
> if we had mobs, it would make sense to map a _lot_ of the current Midi
> generation through them eventually because it would make programming the Midi
> generation much more transparent.

All I've wished to have when implementing these MIDI enhancements is
only some way of translating syntactic constructs in LilyPond input 
into MIDI events for MIDI output.  Because the Staff.midiInstrument
property already existed at the time I started, context properties
ended up being "chosen" as the interface also for the enhancements
because this appeared to keep the extensions syntactically consistent
with what was already there (and saved me at that point from having
to learn how to extend LilyPond input syntax...) - when one considers
only the syntax, it's hard to beat the intuitiveness of just saying
"\set midiXXX = ..." in LilyPond input.  In view of the above goal,
the midiCC "property" was then just extending the mechanisms of
handling MIDI control changes to one possible conclusion.

I don't object to defining a new syntax for expressing "arbitrary"
MIDI control changes in LilyPond input if this is necessary to keep
the interpretation of context properties consistent - the only
limitation that I have is just that I don't possess the technical
knowledge on how to start right away with such a task.  (I also
probably have a problem of not ever having fully appreciated the
value that the different syntaxes used for working with grob vs.
context properties bring to the user writing LilyPond input -
although I'm sure there's many technical reasons as to why this
difference exists, as a user I'd sometimes wish I were able to "just"
modify [and even query for] properties of things using a consistent
syntax without having to care about what kind of things they are,
other than that they have the property that I'm interested in
accessing.  With this view, I probably wouldn't like to have yet
another syntax for expressing MIDI changes in LilyPond input,
either...)

Nevertheless, given the suggested choices, I'd probably go for just
having a separate event for handling the control changes instead of
introducing mobs, which sounds like it could need a lot of work 
without much benefit for most users - the number of people asking
questions about tweaking MIDI output in LilyPond is tiny in
comparison with the number of people asking questions about tweaking
the typeset output (and I don't believe that this is because
LilyPond doesn't have mobs).

The only question I have if the syntax is changed is whether it's
then still possible to have values of MIDI controllers initialized in
a \with block of a Staff using that syntax.  (This concern is
based only on the current implementation of Staff_performer, which
just makes get_property calls to query the initial values of
MIDI-related context properties, to pre-initialize each new
Audio_staff with audio items that set the initial values in MIDI
output.)

[dak, 2016-01-30 13:16:29]

Ok, let's get the show back on track.

We have two things to figure out here: do we want this stuff to even run through
properties of any kind?  And if so, how?

One thing that properties are likely not good for is continuous changes of Midi
controllers.  Now the Midi format itself is not good for that either I think
since controllers will need explicit setting to every value they are to assume
(correct me if I'm wrong), so there will be timing impact and granularity to to
such changes, and any continuous control process would need to figure out
whether to change controllers in time for each new note event, or even trigger
timing on its own.

If we are going by properties: how many continuous controllers are there? 128? 
How about biting the bullet and creating properties cc0 to cc127 then?  That's
the simplest way forward.

The next more complicated way forward using properties would be, basically,
mostly a paradigm change: most of the code, I think, is there.  It would
basically declare that selected context properties (which includes the defaults
for all grobs) are managed with a "property stack" (which basically is what
ly:make-grob-properties creates) and afterwards are accessed using
\override/\revert (which can set and reset subproperties as well).  The tricky
detail here would be how we prevent the use of unpure-pure-containers for midi
properties: straightforward callbacks seem fine.

Yet another way forward would be to extend set/unset to deal with subproperties.

And while we are at it: this is one instance where I'm somewhat envious of Lua
which has as its sole data structure the "table", an associative array that can
be indexed with either strings (all of which are interned in Lua and thus are
equivalent to symbols) or numbers, with an array part starting at index 1 being
available for O(1) non-hashed indexing.  If that were the form of our nested
alists, subindexing via a controller number would be trivial.

As it is, I'm afraid that the first variant will likely have to rely on
_symbols_ for indexing.  (string->symbol "42") can actually be entered directly
as #{42}# (a rather weird syntax, and from LilyPond another # would need to be
prepended).  In order to mask the details from the user, I can try to provide an
input syntax for that that will convert numbers in symbol lists into the
respective symbols at first (maybe we'll be able to let them stay numbers in
some later stage of the implementation).

Another possibility is to just allow a property to be a vector, and let
subproperty accessors interpret numbers as indexes then.  I'm somewhat iffy over
how to implement \temporary \override on that basis, though.  But it's probably
doable.

I'm also iffy about indexing from the LilyPond side: it seems to me like indexes
should run starting with 1 so that I can write stuff like

violin = #(make-vector 2)
violin.1 = { ... }
violin.2 = { ... }

With 0-based indexing, this would have be either
violin = #(make-vector 3)
or
violin.0 = { ... }

neither of which feels convincing.  Unfortunately, Midi controllers are numbered
0 to 127 while Midi Channels are numbered 1 to 16 (even while represented by
0..15 in the Midi messages).

So, uh, big can of worms here.  I'd like to see it eventually opened because it
seems useful to have vector subproperties, but for your immediate needs, I think
going via dedicated music events (and performers for them) would likely be the
sanest path for now, also allowing for a strategy for continous changes at some
later point of time.