Issue #1204: Document, and add regtest for, external fonts.

Issue #1204: Document, and add regtest for, external fonts.

Add a few details in NR 1.8.3.1 "Fonts explained", and
a regtest with base64-embedded minimal OpenType fonts.
(I didn’t want to use musical symbols since the whole
point is that these glyphs are not to be confused with
whatever’s installed on the OS.)
The regtest has been tested with all major versions
since 2.12, and current master with and without Guile2.

[lemzwerg, 2020-03-30 16:06:08]

Nice idea, thanks.  Some nits and questions.

https://codereview.appspot.com/557640051/diff/571940043/Documentation/notatio...
File Documentation/notation/text.itely (right):

https://codereview.appspot.com/557640051/diff/571940043/Documentation/notatio...
Documentation/notation/text.itely:1552: (and thus available in LilyPond scores),
through the following commands:
no comma

https://codereview.appspot.com/557640051/diff/571940043/Documentation/notatio...
Documentation/notation/text.itely:1558: 
I would remove this empty line in the @example environment.

https://codereview.appspot.com/557640051/diff/571940043/input/regression/font...
File input/regression/font-name-add-files.ly (right):

https://codereview.appspot.com/557640051/diff/571940043/input/regression/font...
input/regression/font-name-add-files.ly:14: dummyfontfile =  #(string-append
(tmpnam) "-dummyfont.otf")
s/  / /

https://codereview.appspot.com/557640051/diff/571940043/input/regression/font...
input/regression/font-name-add-files.ly:210: #(mkdir dummyfontdir)
Is it guaranteed that we can create this directory?  What about srcdir !=
builddir?

[Valentin Villenave, 2020-03-30 18:29:37]

On 2020/03/30 16:06:08, lemzwerg wrote:
> Is it guaranteed that we can create this directory?  What about srcdir !=
> builddir?

Yes.  It’s a (tmpnam) name, located in /tmp or $TMPDIR or whatever:
https://www.gnu.org/software/guile/manual/html_node/File-System.html#index-tm...
That’s the mechanism we already use for intermediate-files.

The alternative would be to create the font files in the cwd, like what
markup-eps.ly does.  I decided against it because creating a file is already
intrusive, creating a subdir is somehow worse.

Now, ideally these should be removed after the PDF is generated.  The problem is
that (like intermediate-files, precisely) they need to be deleted only after
GhostScript’s done with them… and I’m not sure we’re able to overwrite, for
example, postprocess-output or output-framework on-the-fly.  I don’t think the
codebase needs additional hooks only for one regtest to clean after itself, but
it *could* be a problem for systems that run the regtests continuously.

Thanks for the other nits, I’ve addressed them.

V.

[Valentin Villenave, 2020-03-30 18:31:11]

Some nits from Werner

[thomasmorley651, 2020-03-30 18:56:27]

On 2020/03/30 18:29:37, Valentin Villenave wrote:
> On 2020/03/30 16:06:08, lemzwerg wrote:
> > Is it guaranteed that we can create this directory?  What about srcdir !=
> > builddir?
> 
> Yes.  It’s a (tmpnam) name, located in /tmp or $TMPDIR or whatever:
>
https://www.gnu.org/software/guile/manual/html_node/File-System.html#index-tm...
> That’s the mechanism we already use for intermediate-files.

Well, see 
https://lists.gnu.org/archive/html/guile-devel/2020-03/msg00046.html
tmpnam is deprecated in guile-3.0.2
Why introducing it, we would need to move away from it later anyway?
Also, the link you provided speaks about security risks...

[Valentin Villenave, 2020-03-30 19:56:55]

On 2020/03/30 18:56:27, thomasmorley651 wrote:
> tmpnam is deprecated in guile-3.0.2

(sigh) Nice catch!

> Why introducing it, we would need to move away from it later anyway?

Well, I *would* have used mkstemp, but I need to create a directory (rather than
a file), and there’s no mkdtemp in Guile.  Plus, I wanted the code to be
supported by both Guile 1.8 and 2.  Plus, we’re already using tmpnam somewhere
else in the codebase.  Plus…

> Also, the link you provided speaks about security risks...

Yes, so I’ve seen.  I certainly *can* imagine how that could be exploited in a
critical case.  (And by letting users have access to a Turing-complete language
within LilyPond files, our exposure to security faults is already quite wide.) 
Then again, that’s a regtest we’re talking about, not a snippet or a doc
example, not something we’re actively encouraging people to do at home.

Frankly, if anyone can think of a solution to do the exact same thing with
mkstemp (and to remove the temporary files after compilation), I’m all ears.  As
of right now, I know I can’t :-)

V.

[Valentin Villenave, 2020-04-03 09:19:44]

Use mkstemp! instead of tmpnam; don’t leave any temporary file behind.

[Valentin Villenave, 2020-04-03 11:23:14]

Temporarily disable font-export-dir (fixes make test)

[Valentin Villenave, 2020-04-11 09:44:26]

https://codereview.appspot.com/557640051/diff/565870043/input/regression/font...
File input/regression/font-name-add-files.ly (right):

https://codereview.appspot.com/557640051/diff/565870043/input/regression/font...
input/regression/font-name-add-files.ly:244: #(rmdir dummyfontdir)

So, David ran into a "Directory not empty" error with this regtest, but I
haven’t been able to reproduce it.
https://sourceforge.net/p/testlilyissues/issues/1204/?page=1#998e/7f0d

What could be the cause?

V.

[Valentin Villenave, 2020-04-19 15:54:32]

On 2020/04/11 09:44:26, Valentin Villenave wrote:
> What could be the cause?

So, pushed as
https://git.savannah.gnu.org/cgit/lilypond.git/commit/?id=0cfef7069e86f85ad39...

V.

[dak, 2020-04-19 16:53:28]

v.villenave@gmail.com writes:

> On 2020/04/11 09:44:26, Valentin Villenave wrote:
>> What could be the cause?
>
> So, pushed as
>
https://git.savannah.gnu.org/cgit/lilypond.git/commit/?id=0cfef7069e86f85ad39...
>
> V.
>
> https://codereview.appspot.com/557640051/

Breaks my patchy again.  Whose patchy was comfortable moving it from
staging to master?

-- 
David Kastrup

[Valentin Villenave, 2020-04-19 17:18:32]

On 4/19/20, David Kastrup <dak@gnu.org> wrote:
> v.villenave@gmail.com writes:
> Breaks my patchy again.  Whose patchy was comfortable moving it from
> staging to master?

Well, at any rate James’ patchy what fine with it. (As it went through
the review/countdown process not just once but twice.)

The regtest actually produces an additional PDF file that should allow
you to check manually that all temporary files have been removed.

Evidently in your case something else (but what?) gets created inside
the temporary directory, which it then can’t remove because it doesn’t
know about it. (I could issue a system call for `rm -rf`, but that
would miss the point.

I already caught one unintended side-effect which was that during make
doc, 'font-export-dir gets enabled and thus my temporary font file
gets referenced which broke make check. There may very well be another
side effect specific to either your system or your build process, but
I’d need help identifying it as I can’t seem to reproduce it myself
(obviously do feel free to revert in the meantime).

Cheers,
V.