Add an expert font tree interface

Add an expert font tree interface

See
    http://lists.gnu.org/archive/html/lilypond-user/2014-07/msg00281.html
for the goals;  the proposed font tree interface allows for a very
fine-grained mapping of fonts to LilyPond's default font-families as well
as introducing new font-family symbols.
Furthermore, it implements per-font scaling (e.g., to match the x-heights
of different fonts used in the document).  No attempt has been made for
auto-detection of the x-height of a font; I guess that would require
delving into the C++-Pango parts, which is above my threshold.


expert font selection: first attempt

allows to choose fonts used in a fine-grained manner, bypassing the
default auto-generated pango-font-description strings

add example for make-expert-font-tree as regtest

[pkx166h, 2014-07-15 09:33:30]

Fails make check

--snip--

regression/font-expert-selection.ly

--snip--

Processing `/tmp/build-lilypond-autobuild/out/lybook-testdb/73/lily-16d682d4.ly'
Parsing...
Renaming input to:
`/tmp/lilypond-autobuild/input/regression/font-expert-selection.ly'
/tmp/lilypond-autobuild/input/regression/font-expert-selection.ly:84:31: error:
syntax error, unexpected \lyricmode, 
expecting \sequential or \simultaneous or << or '{'
    \new Lyrics \lyricsto "v" 
                              \lyricmode { Foo -- bar baz xyz -- zy! }
/tmp/lilypond-autobuild/input/regression/font-expert-selection.ly:80:3: error:
errors found, ignoring music expressio
n
  
  <<
/tmp/lilypond-autobuild/input/regression/font-expert-selection.ly:99:1: error:
Unfinished main input

--snip--

[akobel, 2014-07-15 10:07:34]

remove erroneous \lyricmode to pass make check

[akobel, 2014-07-15 10:08:05]

remove erroneous \lyricmode to pass make check

[dak, 2014-07-16 09:20:45]

https://codereview.appspot.com/108700043/diff/20001/scm/font.scm
File scm/font.scm (right):

https://codereview.appspot.com/108700043/diff/20001/scm/font.scm#newcode240
scm/font.scm:240: (define-public (make-expert-font-tree fonts-alist factor)
There is no doc string here or other documentation.  You call the argument
"fonts-alist", but the code later makes obvious that this is, indeed, not at all
an alist.  Instead an alist is later constructed from it.

[akobel, 2014-07-16 12:57:40]

docstring; renamed fonts-alist to font-spec-list

[akobel, 2014-07-16 12:59:02]

docstring; renamed fonts-alist to font-spec-list

[akobel, 2014-07-16 13:04:41]

https://codereview.appspot.com/108700043/diff/20001/scm/font.scm
File scm/font.scm (right):

https://codereview.appspot.com/108700043/diff/20001/scm/font.scm#newcode240
scm/font.scm:240: (define-public (make-expert-font-tree fonts-alist factor)
On 2014/07/16 09:20:45, dak wrote:
> There is no doc string here or other documentation.

I wrote one now; didn't do it because I started with a copy of
make-pango-font-tree, and it does not have one as well (I know, not the best
reason).
Is there some place but the font.scm itself where it can be seen live?  It is
not built into the docs, right?

> You call the argument "fonts-alist", but the code later makes
> obvious that this is, indeed, not at all
> an alist.  Instead an alist is later constructed from it.

Good catch.  I first planned for an alist, but the less verbose input format of
the lists looks more appealing to me.

[pkx166h, 2014-07-16 17:13:27]

On 16/07/14 14:04, perpeduumimmobile@gmail.com wrote:
>
> https://codereview.appspot.com/108700043/diff/20001/scm/font.scm
> File scm/font.scm (right):
>
> https://codereview.appspot.com/108700043/diff/20001/scm/font.scm#newcode240 
>
> scm/font.scm:240: (define-public (make-expert-font-tree fonts-alist
> factor)
> On 2014/07/16 09:20:45, dak wrote:
>> There is no doc string here or other documentation.
>
> I wrote one now; didn't do it because I started with a copy of
> make-pango-font-tree, and it does not have one as well (I know, not the
> best reason).
> Is there some place but the font.scm itself where it can be seen live?
> It is not built into the docs, right?

http://lilypond.org/doc/v2.18/Documentation/internals/

James

[akobel, 2014-07-17 06:27:43]

On 2014/07/16 17:13:27, J_lowe wrote:
> > On 2014/07/16 09:20:45, dak wrote:
> >> There is no doc string here or other documentation.
> >
> > I wrote one now; didn't do it because I started with a copy of
> > make-pango-font-tree, and it does not have one as well (I know, not the
> > best reason).
> > Is there some place but the font.scm itself where it can be seen live?
> > It is not built into the docs, right?
> 
> http://lilypond.org/doc/v2.18/Documentation/internals/

That's why I ask: it does not show up there.  Not in the list of scheme
functions, where I expected it (at least for the publicly defined functions),
nor anywhere else AFAICS.
And it seems that it's not just that I wrote a faulty docstring; e.g. I could
not find any mention of add-music-fonts or any other function from scm/*.scm I
looked for.

Do I need to build the IR with some specific flag?  The Contributor's Guide,
5.8, only says:
"Material in the Internals reference is generated automatically from our source
code.  Any doc work on Internals therefore requires modifying files in
‘scm/*.scm’.  Texinfo is allowed in these docstrings."

Or am I completely mistaken what a docstring is in Scheme?


Best,
your confused Alexander

[dak, 2014-07-17 06:36:20]

perpeduumimmobile@gmail.com writes:

> On 2014/07/16 17:13:27, J_lowe wrote:
>> > On 2014/07/16 09:20:45, dak wrote:
>> >> There is no doc string here or other documentation.
>> >
>> > I wrote one now; didn't do it because I started with a copy of
>>
>> > It is not built into the docs, right?
>
>> http://lilypond.org/doc/v2.18/Documentation/internals/
>
> That's why I ask: it does not show up there.  Not in the list of scheme
> functions, where I expected it (at least for the publicly defined
> functions), nor anywhere else AFAICS.
>
> And it seems that it's not just that I wrote a faulty docstring; e.g. I
> could not find any mention of add-music-fonts or any other function from
> scm/*.scm I looked for.

Correct.  Only the DOC strings of C functions or music functions (and
the syntactically similar "scheme functions" defined with
define-scheme-function and define-void-function).  Functions to be
called from Scheme and defined in Scheme are not automatically mentioned
in the documentation.

But that does not mean that they should not receive a documentation
string.  One will look it up via Scheme (from a command prompt via help)
or by looking at the definition.

> https://codereview.appspot.com/108700043/


-- 
David Kastrup

[akobel, 2014-07-17 07:06:02]

On 2014/07/17 06:36:20, dak wrote:
> mailto:perpeduumimmobile@gmail.com writes:
> 
> > On 2014/07/16 17:13:27, J_lowe wrote:
> >> > On 2014/07/16 09:20:45, dak wrote:
> >> >> There is no doc string here or other documentation.
> >> >
> >> > I wrote one now; didn't do it because I started with a copy of
> >>
> >> > It is not built into the docs, right?
> >
> >> http://lilypond.org/doc/v2.18/Documentation/internals/
> >
> > That's why I ask: it does not show up there.  Not in the list of scheme
> > functions, where I expected it (at least for the publicly defined
> > functions), nor anywhere else AFAICS.
> >
> > And it seems that it's not just that I wrote a faulty docstring; e.g. I
> > could not find any mention of add-music-fonts or any other function from
> > scm/*.scm I looked for.
> 
> Correct.  Only the DOC strings of C functions or music functions (and
> the syntactically similar "scheme functions" defined with
> define-scheme-function and define-void-function).  Functions to be
> called from Scheme and defined in Scheme are not automatically mentioned
> in the documentation.

Aha, I see.

> But that does not mean that they should not receive a documentation
> string.

Sure.

> One will look it up via Scheme (from a command prompt via help)
> or by looking at the definition.

Two follow-up questions:
1.) Is there a way to list all Lily scheme functions and/or all their
docstrings?  If you know what function or variable you are looking for, the
docstrings are obviously great.  But if I look for something which matches your
favourite buzzword, I currently grep in some way through the entire scheme code.

2.) If I run `guile`, `(help)` gives me all sorts of info on how to use, well,
the help.  But if I run `lilypond scheme-sandbox`, `(help)` will only show me a
complaint "ERROR: Unbound variable: help".  Do I have to use a special trick to
access these docs in a guile REPL?


Thanks,
Alexander

[dak, 2014-07-17 08:28:36]

perpeduumimmobile@gmail.com writes:

> Two follow-up questions:
> 1.) Is there a way to list all Lily scheme functions and/or all their
> docstrings?  If you know what function or variable you are looking for,
> the docstrings are obviously great.  But if I look for something which
> matches your favourite buzzword, I currently grep in some way through
> the entire scheme code.

(module-for-each
  (lambda (s v)
    (let ((r (variable-ref v)))
       (if (and (procedure? r) (procedure-documentation r))
           (format #t "~a: ~a\n\n" s (procedure-documentation r)))))
  (resolve-module '(lily)))

But actually

  git grep something scm

*is* pretty great.

> 2.) If I run `guile`, `(help)` gives me all sorts of info on how to use,
> well, the help.  But if I run `lilypond scheme-sandbox`, `(help)` will
> only show me a complaint "ERROR: Unbound variable: help".  Do I have to
> use a special trick to access these docs in a guile REPL?

(use-modules (ice-9 session))

-- 
David Kastrup

[akobel, 2014-07-17 08:55:09]

On 2014/07/17 08:28:36, dak wrote:
> mailto:perpeduumimmobile@gmail.com writes:
> 
> > Two follow-up questions:
> > 1.) Is there a way to list all Lily scheme functions and/or all their
> > docstrings?  If you know what function or variable you are looking for,
> > the docstrings are obviously great.  But if I look for something which
> > matches your favourite buzzword, I currently grep in some way through
> > the entire scheme code.
> 
> (module-for-each
>   (lambda (s v)
>     (let ((r (variable-ref v)))
>        (if (and (procedure? r) (procedure-documentation r))
>            (format #t "~a: ~a\n\n" s (procedure-documentation r)))))
>   (resolve-module '(lily)))
> 
> But actually
> 
>   git grep something scm
> 
> *is* pretty great.

I agree; I was asking more from a user perspective who does not have a clone of
the repo, but still might want to use one or the other of those define-public
thingies in user-level scheme.

> > 2.) If I run `guile`, `(help)` gives me all sorts of info on how to use,
> > well, the help.  But if I run `lilypond scheme-sandbox`, `(help)` will
> > only show me a complaint "ERROR: Unbound variable: help".  Do I have to
> > use a special trick to access these docs in a guile REPL?
> 
> (use-modules (ice-9 session))

Thanks.

That leaves me with only one more question: Is there any reason to use Texinfo
markup in those docstrings, given that all more-or-less obvious ways of
accessing it do not seem to use it? Or should I rather give a well-formatted
uncluttered raw text description for unprocessed human consumption?


Best,
Alexander

[dak, 2014-07-17 09:13:54]

perpeduumimmobile@gmail.com writes:

> That leaves me with only one more question: Is there any reason to use
> Texinfo markup in those docstrings, given that all more-or-less obvious
> ways of accessing it do not seem to use it? Or should I rather give a
> well-formatted uncluttered raw text description for unprocessed human
> consumption?

Obviously, there is nothing precluding us from eventually including the
doc strings in the Internals Guide, most likely stripped down to
actually exported symbols (the rest, if at all, would rather belong to
the Contributor's Guide).

Of course, doing so would require some cleanup and polishing, and come
Guile-2, there might be some separation in different modules.

> https://codereview.appspot.com/108700043/

-- 
David Kastrup

[akobel, 2014-07-17 09:19:10]

On 2014/07/17 09:13:54, dak wrote:
> mailto:perpeduumimmobile@gmail.com writes:
> 
> > That leaves me with only one more question: Is there any reason to use
> > Texinfo markup in those docstrings, given that all more-or-less obvious
> > ways of accessing it do not seem to use it? Or should I rather give a
> > well-formatted uncluttered raw text description for unprocessed human
> > consumption?
> 
> Obviously, there is nothing precluding us from eventually including the
> doc strings in the Internals Guide, most likely stripped down to
> actually exported symbols (the rest, if at all, would rather belong to
> the Contributor's Guide).

Good point. Thanks!

[Mark Polesky, 2014-07-24 20:55:54]

https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...
File input/regression/font-expert-selection.ly (right):

https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...
input/regression/font-expert-selection.ly:33: ;; definition, irregardless of the
name given. (Only before the score?
regardless

https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...
input/regression/font-expert-selection.ly:66: #(define fonts
#(define fonts
   (make-palladio-dejavu-tree
     (/ myStaffSize 20)))

https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...
input/regression/font-expert-selection.ly:77: \markup \huge { \sans "xxx sans
xxx" "xxx serif xxx" \typewriter "xxx monospace xxx" }
\markup \huge {
  \sans "xxx sans xxx"
  "xxx serif xxx"
  \typewriter "xxx monospace xxx"
}

https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...
input/regression/font-expert-selection.ly:81: \new Staff <<
`\new Staff << >>' is redundant, just do `\context Voice'

https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...
input/regression/font-expert-selection.ly:90: \override LyricText . font-family
= #'condensed
LyricText.font-family

https://codereview.appspot.com/108700043/diff/80001/scm/font.scm
File scm/font.scm (right):

https://codereview.appspot.com/108700043/diff/80001/scm/font.scm#newcode285
scm/font.scm:285: (let ((n (make-font-tree-node 'font-encoding 'fetaMusic)))
I'd find this formatting easier to read:
 
  (let ((n (make-font-tree-node 'font-encoding 'fetaMusic)))
    (add-music-fonts n "emmentaler" 'feta feta-design-size-mapping factor)
    (for-each
      (lambda (L) 
        (let* ((lily-family (list-ref L 0)) 
               (shape (list-ref L 1)) 
               (series (list-ref L 2)) 
               (scale (if (= (length L) 5)
                        (list-ref L 4 ) 
                        1.0))
               (desc (string-append (list-ref L 3)
                                    " " 
                                    (number->string (* scale (ly:pt 12))))))
          (add-expert-node n lily-family shape series desc)))
      font-spec-list)
    n))

[news_a-kobel.de, 2014-07-25 11:12:29]

On 07/25/2014 05:55 AM, markpolesky@gmail.com wrote:
>
https://codereview.appspot.com/108700043/diff/80001/input/regression/font-exp...

Mark, thanks for the comments.  James, give this at least one more 
cycle, please.  I'll incorporate the changes as soon as I can find the 
time, and want to double-check the entire patchset again - hopefully 
I'll have more time available next week.


Best,
Alexander

[Mark Polesky, 2014-07-31 07:09:43]

https://codereview.appspot.com/108700043/diff/80001/scm/font.scm
File scm/font.scm (right):

https://codereview.appspot.com/108700043/diff/80001/scm/font.scm#newcode241
scm/font.scm:241: "Construct a font tree consisting of the default Feta music
font and
I think an explanation and clear example is needed at the end of NR 1.8.3 Fonts,
instead of here.  The average user will never find this.

[janek, 2014-07-31 20:36:12]

https://codereview.appspot.com/108700043/diff/80001/scm/font.scm
File scm/font.scm (right):

https://codereview.appspot.com/108700043/diff/80001/scm/font.scm#newcode285
scm/font.scm:285: (let ((n (make-font-tree-node 'font-encoding 'fetaMusic)))
On 2014/07/24 20:55:54, Mark Polesky wrote:
> I'd find this formatting easier to read:
>  
>   (let ((n (make-font-tree-node 'font-encoding 'fetaMusic)))
>     (add-music-fonts n "emmentaler" 'feta feta-design-size-mapping factor)
>     (for-each
>       (lambda (L) 
>         (let* ((lily-family (list-ref L 0)) 
>                (shape (list-ref L 1)) 
>                (series (list-ref L 2)) 
>                (scale (if (= (length L) 5)
>                         (list-ref L 4 ) 
>                         1.0))
>                (desc (string-append (list-ref L 3)
>                                     " " 
>                                     (number->string (* scale (ly:pt 12))))))
>           (add-expert-node n lily-family shape series desc)))
>       font-spec-list)
>     n))

+1

[janek, 2014-07-31 20:40:40]

from description:
> Furthermore, it implements per-font scaling (e.g., to match the x-heights
> of different fonts used in the document).

This is very nice!  Please pardon a stupid question, but i failed to find which
place in the code is responsible for this.  Could you give me a pointer?

> No attempt has been made for
> auto-detection of the x-height of a font; I guess that would require
> delving into the C++-Pango parts, which is above my threshold.

That's something i'm interested in.  Most probably i won't have enough time to
tackle it soon, but someday i'd really like to implement such a feature!

best,
Janek

[akobel, 2014-08-01 10:04:40]

On 2014/07/31 20:40:40, janek wrote:
> from description:
> > Furthermore, it implements per-font scaling (e.g., to match the x-heights
> > of different fonts used in the document).
> 
> This is very nice!  Please pardon a stupid question, but i failed to find
which
> place in the code is responsible for this.  Could you give me a pointer?

The user interface is simply an optional argument (default 1) in the
font-spec-list.  Internally, it works as follows:  Each Pango font description
carries a size value.  Not sure how to call it - design size, preferred size? 
Anyway, the string given by the user as a font specification is appended by some
size which is computed as
  12pt * global-factor [last arg of make-[expert/pango]-font-tree] *
per-font-factor [optional arg in font-spec-list]
The effect is a different output scale for each font.

> > No attempt has been made for
> > auto-detection of the x-height of a font; I guess that would require
> > delving into the C++-Pango parts, which is above my threshold.
> 
> That's something i'm interested in.  Most probably i won't have enough time to
> tackle it soon, but someday i'd really like to implement such a feature!

I have no idea how hard or easy this is.  An interesting side effect is that
probably the list of available (OpenType) font features like expert ligatures,
old style figures etc. can easily be accessed at the same time, and I hope that
it is possible to make them available to the user as well.  But that's much more
involved than just make-expert-font-tree.

[janek, 2014-08-02 13:05:14]

2014-08-01 12:04 GMT+02:00  <perpeduumimmobile@gmail.com>:
> The user interface is simply an optional argument (default 1) in the
> font-spec-list. [...]

Ah, so it's up to the user to provide scaling factors.  I thought that
there is some code measuring height of the x letter and scaling the
font accordingly :)

Ok, now i see that you've already written that in

>> > No attempt has been made for
>> > auto-detection of the x-height of a font; I guess that would require
>> > delving into the C++-Pango parts, which is above my threshold.

(but i've misunderstood it as something else).

best,
Janek