Adapt lilypond to ghostscript 9.22, change command line options

We need to adapt lilypond to ghostscript 9.22 as our old --bigpdf parameter is broken by the new version of gs.

--bigpdf is totally removed.

--pspdfopt=(size/TeX/TeX-GS) is implemented to allow the user to select the appropiate set of -d... parameters for different purposes.

The patch also contains some minor fixes of related bugs, introduces --eps to the command line interface and changes the bug-report message as post.gmane is gone.

Signed-off-by: Knut Petersen <knut_petersen@t-online.de>

[dak, 2017-09-23 16:02:32]

https://codereview.appspot.com/325630043/diff/1/make/lilypond-vars.make
File make/lilypond-vars.make (right):

https://codereview.appspot.com/325630043/diff/1/make/lilypond-vars.make#newco...
make/lilypond-vars.make:54: -dgs-never-embed-fonts -b
Is there a conceivable application for using -b without -dgs-never-embed-fonts ?
 If not, shouldn't -b imply -dgs-never-embed-fonts anyway ?  Otherwise people
are likely to get results they don't expect I think.

[knupero, 2017-09-23 16:22:58]

On 2017/09/23 16:02:32, dak wrote:
> https://codereview.appspot.com/325630043/diff/1/make/lilypond-vars.make
> File make/lilypond-vars.make (right):
> 
>
https://codereview.appspot.com/325630043/diff/1/make/lilypond-vars.make#newco...
> make/lilypond-vars.make:54: -dgs-never-embed-fonts -b
> Is there a conceivable application for using -b without -dgs-never-embed-fonts
?
>  If not, shouldn't -b imply -dgs-never-embed-fonts anyway ?  Otherwise people
> are likely to get results they don't expect I think.

It definitely is usefull to be able to use -dgs-never-embed-fonts without -b.

To force -dgs-never-embed-fonts if -b is used will definitely break existing
scripts. Of course, existing scripts will also be broken by new versions of
ghostscript - but not everyone uses gs versions above 9.16, and it will take a
long time until it is save to assume that everybody uses at least ghostscript
9.22.

I think it's definitely necessary to add some documentation.

I also have to (and I will) test some old versions of ghostscript.

[trueroad, 2017-09-23 16:49:14]

`-dgs-never-embed-fonts` doesn't effect for TrueType fonts.
Therefore, when both `-dgs-never-embed-fonts` and `--bigpdfs`/`-b` are used at
the same time, TrueType fonts are embedded as full set (ie non-subset). (if
there is no bug in gs.)
On the other hand, some intermeiate PDFs use Japanese TrueType font.
Of course the CJK font is a very large file size.

Combining them means that the size of the final PDF may be smaller but the size
of the intermediate PDFs will be larger.
In my experiment, even if there is a bug in gs's full set font embedding, the
disk size that `make doc` eats has increased.

If I understand correctly, `--bigpdfs` / `-b` has two effects.
One is to embed full set (non-subset) font. 
The other is to define and use some encodings for Emmentaler.

In my humble opinion , how about making two options that can specify each of
them individually?
E.g. the former one might be `--dgs-embed-fullset-fonts`.
And ,the latter one might be `--duse-notation-font-encodings`.

Then, `--bigpdfs` / `-b` will be an alias for a combination of
`--dgs-embed-fullset-fonts` and `--duse-notation-font-encodings`.
For `make doc`, use `-dgs-never-embed-fonts --duse-notation-font-encodings`
instead of `-dgs-never-embed-fonts -b`.

[knupero, 2017-09-24 00:25:17]

> Combining them means that the size of the final PDF may be smaller but the
size
> of the intermediate PDFs will be larger.
> In my experiment, even if there is a bug in gs's full set font embedding, the
> disk size that `make doc` eats has increased.

I did not check that, but I would expect an increase in disk usage for the
build.

> If I understand correctly, `--bigpdfs` / `-b` has two effects.
> One is to embed full set (non-subset) font.
> The other is to define and use some encodings for Emmentaler.

It also changes the way emmentaler glyphs are printed. With --bigpdfs we use the
"show" postscript operator, without it we still use "glyphshow". And the three
emmentaler fonts for every size in the intermediate versions of our pdfs are not
3 copies of the full emmentaler-xx font but are three different fonts made of
emmentaler-xx glyphs. Those subsets must not be further subsetted. 

No, it is senseless to split -b into separate options.

[trueroad, 2017-09-24 10:05:43]

On 2017/09/24 00:25:17, knupero wrote:
> > If I understand correctly, `--bigpdfs` / `-b` has two effects.
> > One is to embed full set (non-subset) font.
> > The other is to define and use some encodings for Emmentaler.
> 
> It also changes the way emmentaler glyphs are printed. With --bigpdfs we use
the
> "show" postscript operator, without it we still use "glyphshow". And the three
> emmentaler fonts for every size in the intermediate versions of our pdfs are
not
> 3 copies of the full emmentaler-xx font but are three different fonts made of
> emmentaler-xx glyphs. Those subsets must not be further subsetted. 

I wrote "to define and *use* the some encodings".
If I understand correctly, "glyphshow" doesn't use any encodings but "show" uses
the encoding.
What I wrote as "use" means using "show" instead of "glyphshow".

> No, it is senseless to split -b into separate options.

Again, when both `-dgs-never-embed-fonts` and `--bigpdfs`/`-b` are used at the
same time, the intermediate PDFs become larger.
The disk size that `make doc` eats becomes increased.

If spliting -b, `-dgs-never-embed-fonts -duse-notation-font-encodings` doesn't
embed full set TrueType fonts.
It embeds subset for TrueType fonts, as with only using
`-dgs-never-embed-fonts`.
The intermediate PDFs don't become larger.
The disk size that `make doc` eats doesn't become increased.
Nevertheless, the encodings for notation font is defined and "show" is used.
In my humble opinion, this is better than `-dgs-never-embed-fonts -b`.

Of course, if we do not have to worry about the increase in disk size required,
splitting -b is senseless as you wrote.

[dak, 2017-09-24 11:03:26]

On 2017/09/24 10:05:43, trueroad wrote:
> On 2017/09/24 00:25:17, knupero wrote:
> > > If I understand correctly, `--bigpdfs` / `-b` has two effects.
> > > One is to embed full set (non-subset) font.
> > > The other is to define and use some encodings for Emmentaler.
> > 
> > It also changes the way emmentaler glyphs are printed. With --bigpdfs we use
> the
> > "show" postscript operator, without it we still use "glyphshow". And the
three
> > emmentaler fonts for every size in the intermediate versions of our pdfs are
> not
> > 3 copies of the full emmentaler-xx font but are three different fonts made
of
> > emmentaler-xx glyphs. Those subsets must not be further subsetted. 
> 
> I wrote "to define and *use* the some encodings".
> If I understand correctly, "glyphshow" doesn't use any encodings but "show"
uses
> the encoding.
> What I wrote as "use" means using "show" instead of "glyphshow".
> 
> > No, it is senseless to split -b into separate options.
> 
> Again, when both `-dgs-never-embed-fonts` and `--bigpdfs`/`-b` are used at the
> same time, the intermediate PDFs become larger.
> The disk size that `make doc` eats becomes increased.
> 
> If spliting -b, `-dgs-never-embed-fonts -duse-notation-font-encodings` doesn't
> embed full set TrueType fonts.
> It embeds subset for TrueType fonts, as with only using
> `-dgs-never-embed-fonts`.
> The intermediate PDFs don't become larger.
> The disk size that `make doc` eats doesn't become increased.
> Nevertheless, the encodings for notation font is defined and "show" is used.
> In my humble opinion, this is better than `-dgs-never-embed-fonts -b`.
> 
> Of course, if we do not have to worry about the increase in disk size
required,
> splitting -b is senseless as you wrote.

In my opinion, -b/--bigpdf should be simple to use for lilypond-book-like
applications.  lilypond-book should likely call lilypond using options leading
to good results by default.  For generating our own documentation, I find 3-4GB
of disk space excessive: using special additional options might be suitable if
that helps.  If the additional options work at least as well under all
conceivable circumstances, they should likely be the default.

-dxxx are options appealing more to specialists (well, there are things like
-dpreview not particularly specialist, but when in doubt, people are more likely
to see/consider "normal" options).

I'm not into the whole font mess enough to make a really qualified suggestion
here.

[trueroad, 2017-09-24 14:32:02]

On 2017/09/24 11:03:26, dak wrote:
> In my opinion, -b/--bigpdf should be simple to use for lilypond-book-like
> applications.  lilypond-book should likely call lilypond using options leading
> to good results by default.  For generating our own documentation, I find
3-4GB
> of disk space excessive: using special additional options might be suitable if
> that helps.  If the additional options work at least as well under all
> conceivable circumstances, they should likely be the default.
> 
> -dxxx are options appealing more to specialists (well, there are things like
> -dpreview not particularly specialist, but when in doubt, people are more
likely
> to see/consider "normal" options).
> 
> I'm not into the whole font mess enough to make a really qualified suggestion
> here.

To make the options simple, what about doing as follows?



* --bigpdfs / -b

Keep its behavior

i.e.
- Define and use some encodings for Emmentaler. (Also using "show" instead of
"glyphshow")
- For PDF output, embed full set (non-subset) font (if there is no bugs in
Ghostscript)

This option is for full set (non-subset) embedded font intermediate PDFs, and
later uses MuPDF etc. to remove duplicate the fonts.

This method is simple and easy to understand.
However, the size of the intermediate PDF increases.
Also, the current Ghostscript does not work properly because there seems to be a
bug in which full set of fonts can not be embedded.



* -dgs-neverembed-fonts

Add behavior

Current behavior
- For PDF output, It does not embed fonts except TrueType fonts.

Added behavior
- Define and use some encodings for Emmentaler. (Also using "show" instead of
"glyphshow")
- For PDF output, It does not embed fonts except TrueType fonts.

This option is for non-embedded font intermediate PDFs, and later uses
Ghostscript to embed missing fonts.

This method needs to pass the missing fonts to Ghostscript.
In other words, it is for specialists.
However, if it is used properly, it works even with the current Ghostscript.

[Carl, 2017-09-25 00:44:48]

On 2017/09/24 14:32:02, trueroad wrote:

> * -dgs-neverembed-fonts
> 
> Add behavior
> 
> Current behavior
> - For PDF output, It does not embed fonts except TrueType fonts.
> 
> Added behavior
> - Define and use some encodings for Emmentaler. (Also using "show" instead of
> "glyphshow")
> - For PDF output, It does not embed fonts except TrueType fonts.
> 
> This option is for non-embedded font intermediate PDFs, and later uses
> Ghostscript to embed missing fonts.
> 
> This method needs to pass the missing fonts to Ghostscript.
> In other words, it is for specialists.
> However, if it is used properly, it works even with the current Ghostscript.

So if I understand this correctly, properly using -dgs-neverembed-fonts as part
of our manual build process will give us *both* smaller intermediate files and a
small final file?  That sounds perfect to me!

Carl

[knupero, 2017-09-25 12:13:34]

kill --bigpdfs, introduce --use-encodings, remove code not needed by ghostscript
9.20+

[knupero, 2017-09-25 12:54:24]

A bit more far-reaching.

As newer versions of gs 
  - don't need the helpemmentaler functions and
  - don't need the real inclusion of all font data and
  - don't need some other details and
  - don't work properly with our old code
we might well decide to kill --bigpdfs.

Instead of --bigpdfs this patch introduces --use-encodings

--use-encodings switches emmentaler glyph generation from "glyphshow" to
"show"+encodings and might be combined width -dgs-never-embed-fonts.

With --use-encodings there is an increase in file size, but it is not dramatic.
Nevertheless it should not be the default.

This patch gives a total size of 96MB for all of our documentation pdfs on my
system if used with ghostscript git master or at least ghostscript 9.20. Also
extractpdfmark must be used.

If you use ghostscript versions below 9.20 or if extractpdfmark is not present
on your system, the total size of the documentation is about 306 MB.

du -sh build/ gives 2.6GB after a full build with this patch, 2.5GB without it.
Disk usage during the build probably is higher.

The last stage of building the pdfs does cost a significant amount of processing
time: generating the 306MB set of pdfs takes about 940 seconds on my system,
generating the 96MB set of pdf takes an additional 110 seconds.

Documentation needs to be finished.

Please test on your systems.

[dak, 2017-09-25 13:21:44]

On 2017/09/25 12:13:34, knupero wrote:
> kill --bigpdfs, introduce --use-encodings, remove code not needed by
ghostscript
> 9.20+

On my system (Ubuntu Aarkvard) I have Ghostscript 9.21 which is fine.  Usually
we try to support the last LTS versions of Ubuntu, possibly RedHat (and Debian
unstable ?).  Since we need to get our house in order for the stable 2.20
release, I'd very much like to have a PostScript setup we are comfortable with
in the 2.20 release, and that means that this patch (or some variation of it)
should go in.

I don't really have a clue here: can someone chime in with the current
Ghostscript versions in possibly concerned distributions?  Naturally including
our own Gub-based ones?

Or is 9.20+ just a conservative estimate and older versions will (or might)
likely work as well?

[dak, 2017-09-25 13:41:16]

On 2017/09/25 12:54:24, knupero wrote:
> A bit more far-reaching.
> 
> As newer versions of gs 
>   - don't need the helpemmentaler functions and
>   - don't need the real inclusion of all font data and
>   - don't need some other details and
>   - don't work properly with our old code
> we might well decide to kill --bigpdfs.
> 
> Instead of --bigpdfs this patch introduces --use-encodings
> 
> --use-encodings switches emmentaler glyph generation from "glyphshow" to
> "show"+encodings and might be combined width -dgs-never-embed-fonts.

That sounds good.  It also sounds like -dgs-never-embed-fonts should imply
--use-encodings, right?

> With --use-encodings there is an increase in file size, but it is not
dramatic.
> Nevertheless it should not be the default.

I think it really depends on what "is not dramatic" means.  How is the impact on
really large scores?  If the percentages are comparatively small, I think we are
better off making the option most amicable to further processing of PDF files
the default.

Users or scripts that are sure that they do not want any further processing can
then use --final-pdf (or whatever) if they know that the size/speed wins have no
negative side effects for them.  But the default setting should be the one least
likely to cause followup problems.

> This patch gives a total size of 96MB for all of our documentation pdfs on my
> system if used with ghostscript git master or at least ghostscript 9.20. Also
> extractpdfmark must be used.
> 
> If you use ghostscript versions below 9.20 or if extractpdfmark is not present
> on your system, the total size of the documentation is about 306 MB.

That's sort-of unpleasant but we can likely declare this temporary constellation
"somebody else's problem" as long as our lilydev VM and the gub distributions
are recent enough not to be affected.

> du -sh build/ gives 2.6GB after a full build with this patch, 2.5GB without
it.
> Disk usage during the build probably is higher.

Even if we are using -dgs-never-embed-fonts for the included files?  I mean, the
numbers are smaller than what I remember but still rather large.

> The last stage of building the pdfs does cost a significant amount of
processing
> time: generating the 306MB set of pdfs takes about 940 seconds on my system,
> generating the 96MB set of pdf takes an additional 110 seconds.

I assume that the additional time is in Ghostscript/extractpdfmark so that there
is little in the way of improving LilyPond itself?

[trueroad, 2017-09-25 14:50:09]

On 2017/09/25 13:21:44, dak wrote:
> I don't really have a clue here: can someone chime in with the current
> Ghostscript versions in possibly concerned distributions?  Naturally including
> our own Gub-based ones?

GUB has Ghostscript 9.21 except freebsd-x86.
(Ghostscript 9.2x for freebsd-x86 raises seg fault. So Ghostscript 9.15 is used
for freebsd-x86.)

GUB also has extractpdfmark.

> Or is 9.20+ just a conservative estimate and older versions will (or might)
> likely work as well?

Ghostscript 9.20+ is needed.
Ghostscript 9.19 doesn't work fine.
9.19 can not handle some named destinations like Unicode strings that generated
by Texinfo.
https://bugs.ghostscript.com/show_bug.cgi?id=696974

[knupero, 2017-09-25 16:38:16]

> > --use-encodings switches emmentaler glyph generation from "glyphshow" to
> > "show"+encodings and might be combined width -dgs-never-embed-fonts.
> 
> That sounds good.  It also sounds like -dgs-never-embed-fonts should imply
> --use-encodings, right?

No. There are legitimate uses for both standalone -dgs-never-embed-fonts and
--use-encodings.

> I think it really depends on what "is not dramatic" means.  How is the impact
on
> really large scores?  If the percentages are comparatively small, I think we
are
> better off making the option most amicable to further processing of PDF files
> the default.

All scores tested here (only a few) increased about 10% to 20% in size. 
But the code paths are old and proven, we should keep both.

> Users or scripts that are sure that they do not want any further processing
can
> then use --final-pdf (or whatever) if they know that the size/speed wins have
no
> negative side effects for them. But the default setting should be the one
least
> likely to cause followup problems.

I believe that the ordinary user of lilypond does one score at a time without
further processing. And Fred Foobar probably would complain if his old scores
grow bigger with 2.20.

> > If you use ghostscript versions below 9.20 or if extractpdfmark is not
present
> > on your system, the total size of the documentation is about 306 MB.

At least they are not broken ;-)

> That's sort-of unpleasant but we can likely declare this temporary
constellation
> "somebody else's problem" as long as our lilydev VM and the gub distributions
> are recent enough not to be affected.

Nobody who builds lilypond documentation should use a ghostscript old than 9.20.
As pointed out by Masamichi, GoToR links are not processed correctly by all
older versions of ghostscript. Of course, as not all pdf readers process them
correctly, a lot of people would not see that problem. Some examples: okular
does not process them at all, evince handles them correctly (but fails on link
types not used by our documentation), and mupdf handles them incorrectly (but
correctly implements other links that are broken by evince bugs). 

If I would know that 9.20+ is available on every system supported by lilypond I
would advocate to make that version a build requirement.

> Even if we are using -dgs-never-embed-fonts for the included files?  I mean,
the
> numbers are smaller than what I remember but still rather large.

Yes. Even with -dgs-never-embed-fonts.

> I assume that the additional time is in Ghostscript/extractpdfmark so that
there
> is little in the way of improving LilyPond itself?

I think you are right with that assumption.

[knupero, 2017-09-25 16:45:33]

> I don't really have a clue here: can someone chime in with the current
> Ghostscript versions in possibly concerned distributions?  Naturally including
> our own Gub-based ones?

OpenSuSE Tumbleweed: 9.21
OpenSuSE Leap: 9.15, but also offers 9.21

[dak, 2017-09-25 17:33:47]

Why are we still navigating the bleeding edge?  LilyPond is more than 20 years
old.

On 2017/09/25 16:38:16, knupero wrote:
> > > --use-encodings switches emmentaler glyph generation from "glyphshow" to
> > > "show"+encodings and might be combined width -dgs-never-embed-fonts.
> > 
> > That sounds good.  It also sounds like -dgs-never-embed-fonts should imply
> > --use-encodings, right?
> 
> No. There are legitimate uses for both standalone -dgs-never-embed-fonts and
> --use-encodings.

I see.  Still using -dgs-never-embed-fonts _without_ --use-encodings seems like
the most unusual combination, so I'd prefer if one had to explicitly switch
encodings _off_ rather than _on_.

> > I think it really depends on what "is not dramatic" means.  How is
> > the impact on really large scores?  If the percentages are
> > comparatively small, I think we are better off making the option
> > most amicable to further processing of PDF files the default.

> All scores tested here (only a few) increased about 10% to 20% in
> size.

As long as they were large, that would seem representative.  I was
hoping for less.

> But the code paths are old and proven, we should keep both.

Well, I wasn't suggesting otherwise, at least in the short term.  But
it seemed like putting the glyphshow way behind an explicit option
would have had benefits for making the default PDF files more
versatile.

This is all more murky than I like.

> > Users or scripts that are sure that they do not want any further
> > processing can then use --final-pdf (or whatever) if they know
> > that the size/speed wins have no negative side effects for
> > them. But the default setting should be the one least likely to
> > cause followup problems.

> I believe that the ordinary user of lilypond does one score at a
> time without further processing. And Fred Foobar probably would
> complain if his old scores grow bigger with 2.20.

Depends on how much.  10-20% would make a difference, I'm just not
sure enough for a complaint.  And we would retain the option
--final-pdf (or whatever) for making them with the old size.

> > > If you use ghostscript versions below 9.20 or if extractpdfmark
> > > is not present on your system, the total size of the
> > > documentation is about 306 MB.

> At least they are not broken ;-)

That's the main excuse.  And it wasn't better before we came up with
the --bigpdf scheme.

> > That's sort-of unpleasant but we can likely declare this temporary
> > constellation "somebody else's problem" as long as our lilydev VM
> > and the gub distributions are recent enough not to be affected.

> Nobody who builds lilypond documentation should use a ghostscript
> old than 9.20.  As pointed out by Masamichi, GoToR links are not
> processed correctly by all older versions of ghostscript. Of course,
> as not all pdf readers process them correctly, a lot of people would
> not see that problem. Some examples: okular does not process them at
> all, evince handles them correctly (but fails on link types not used
> by our documentation), and mupdf handles them incorrectly (but
> correctly implements other links that are broken by evince bugs).

> If I would know that 9.20+ is available on every system supported by
> lilypond I would advocate to make that version a build requirement.

I think you could still override such a requirement with

GS=/usr/bin/gs ./configure ...

or similar so I am somewhat sympathetic to that.

Sigh.

Why can't anything be easy?

[knupero, 2017-09-25 22:17:41]

On 2017/09/25 17:33:47, dak wrote:
> Why are we still navigating the bleeding edge?  LilyPond is more than 20 years
> old.

I remember a company that failed to get flash right.

Ghostscript still fails on GoToR without the help of extractpdfmarks ;-)


> 
> On 2017/09/25 16:38:16, knupero wrote:
> > > > --use-encodings switches emmentaler glyph generation from "glyphshow" to
> > > > "show"+encodings and might be combined width -dgs-never-embed-fonts.
> > > 
> > > That sounds good.  It also sounds like -dgs-never-embed-fonts should imply
> > > --use-encodings, right?
> > 
> > No. There are legitimate uses for both standalone -dgs-never-embed-fonts and
> > --use-encodings.
> 
> I see.  Still using -dgs-never-embed-fonts _without_ --use-encodings seems
like
> the most unusual combination, so I'd prefer if one had to explicitly switch
> encodings _off_ rather than _on_.

So you are absolutely sure that enabling "show" and the use of encodings does
not break anything?

What about other notation fonts?

About three weeks ago Malte Meyn complained on lilypond-user that "lilypond -b"
failed with the Haydn font.

Remember: We used the hardcoded name "Emmentaler" at various places for
--bigpdfs, and --use-encodings still needs and uses that portion of the old
code.

I have not checked that in detail, but I believe that we need to adapt
ps/encodingdefs.ps and scm/output-ps to support other notation fonts with
--use-encodings.

Ceterum censeo: We really should stay with glyphshow as our default for lilypond
2.20 (two-two-zero).
 

> This is all more murky than I like.

I totally agree.


> Sigh.
> 
> Why can't anything be easy?

If it would be too easy someone would complain that it is boring ;-))

There is enough to be done for the lilypond 2.21 (two-two-one) low-level
postscript interface:
-  Support other notation fonts with --use-encodings
-  Change the code path that uses print_glyphs to also allow the use of
encodings and show.
-  We should change our eps inclusion code to allow to reuse an image once it is
loaded.
-  We should include a well defined way to set a background color, to use an
background image, and to allow a transparent overlay.
-  We even might discuss if it is wise to keep a separate svg backend as it is
easy to translate postscript to svg.
-  Someone probably will propose to add ghostscript as a git submodule to the
lilypond tree and to add some patches.

Don't complain that this isn't easy, be happy that there are at least two
persons working on this part of the code.

[dak, 2017-09-26 11:40:44]

On 2017/09/25 22:17:41, knupero wrote:
> On 2017/09/25 17:33:47, dak wrote:

> > I see.  Still using -dgs-never-embed-fonts _without_
> > --use-encodings seems like the most unusual combination, so I'd
> > prefer if one had to explicitly switch encodings _off_ rather than
> > _on_.

> So you are absolutely sure that enabling "show" and the use of
> encodings does not break anything?

> What about other notation fonts?

> About three weeks ago Malte Meyn complained on lilypond-user that
> "lilypond -b" failed with the Haydn font.

> Remember: We used the hardcoded name "Emmentaler" at various places
> for --bigpdfs, and --use-encodings still needs and uses that portion
> of the old code.

Before remembering something, I need to know it in the first place.
Sounds like something we should get rid off in the long run.

> I have not checked that in detail, but I believe that we need to
> adapt ps/encodingdefs.ps and scm/output-ps to support other notation
> fonts with --use-encodings.

> Ceterum censeo: We really should stay with glyphshow as our default
> for lilypond 2.20 (two-two-zero).

Well, hard to argue with that.

Nevertheless, for 2.21+ I would want to move to a different default,
so it makes sense to bikeshed the naming of the option(s) such that we
only need to flip the default at some point of time.  Scripts should
already be able to explicitly decide which side of the default they
want to end up with (and will then do so regardless of whether the
default has been flipped).

Regarding the naming of the option, I'd prefer to have something
oriented around the actual use characteristics rather than the
internal details.

Does that sound like something you can agree with?

> - We even might discuss if it is wise to keep a separate svg backend
> as it is easy to translate postscript to svg.

I would not want to touch this before we have Cairo in place.  The
PostScript path is inherently slower than any more direct option, so
if retiring SVG as an explicit backend would be on the slate, a
Cairo-based path would seem to make a lot more sense.

> - Someone probably will propose to add ghostscript as a git
> submodule to the lilypond tree and to add some patches.

I doubt we could reasonably afford the ongoing cost of even a small
fork.  So as long as we can coordinate efforts with upstream, that
seems vastly preferable.

[knupero, 2017-09-26 20:26:36]

> > Remember: We used the hardcoded name "Emmentaler" at various places
> > for --bigpdfs, and --use-encodings still needs and uses that portion
> > of the old code.
> 
> Before remembering something, I need to know it in the first place.
> Sounds like something we should get rid off in the long run.

Yes. I never used an alternative music font, and I don't have the details
how the Authors implement them. This morning I wrote to Abraham Lee.

> Nevertheless, for 2.21+ I would want to move to a different default,
> so it makes sense to bikeshed the naming of the option(s) such that we
> only need to flip the default at some point of time.  Scripts should
> already be able to explicitly decide which side of the default they
> want to end up with (and will then do so regardless of whether the
> default has been flipped).
> 
> Regarding the naming of the option, I'd prefer to have something
> oriented around the actual use characteristics rather than the
> internal details.
> 
> Does that sound like something you can agree with?

Yes. That sounds reasonable.

If you know a better name for the option I'd happy to start sed.

> > - Someone probably will propose to add ghostscript as a git
> > submodule to the lilypond tree and to add some patches.
> 
> I doubt we could reasonably afford the ongoing cost of even a small
> fork.  So as long as we can coordinate efforts with upstream, that
> seems vastly preferable.

We'll talk about that in some months ;-)

[dak, 2017-09-26 22:19:27]

On 2017/09/26 20:26:36, knupero wrote:
>
> > Regarding the naming of the option, I'd prefer to have something
> > oriented around the actual use characteristics rather than the
> > internal details.
> > 
> > Does that sound like something you can agree with?
> 
> Yes. That sounds reasonable.
> 
> If you know a better name for the option I'd happy to start sed.

--final-output ?  --flatten-fonts ?  Not sure about the inversions of those,
however.

[Karlin High, 2017-09-27 02:17:16]

On 2017/09/26 22:19:27, dak wrote:
> > If you know a better name for the option I'd happy to start sed.
> 
> --final-output ?  --flatten-fonts ?  Not sure about the inversions of those,
> however.

Somehow, concisely say "get-rid-of-anything-extra" versus
"keep-everything-that-comes?"

--min-fonts and --max-fonts
--compress-fonts and --expand-fonts
--drop-fonts and --keep-fonts
--trim-fonts and --leave-fonts
--compact-fonts and --full-fonts
--few-fonts and --all-fonts

Bikeshedding done for the day.

[wl_gnu.org, 2017-09-27 04:55:31]

> Somehow, concisely say "get-rid-of-anything-extra" versus
> "keep-everything-that-comes?"
> 
> --min-fonts and --max-fonts
> --compress-fonts and --expand-fonts

I would rather prefer --compress-fonts and --no-compress-fonts.  This
is easier to remember (and rather common in GNU packages).

> --drop-fonts and --keep-fonts
> --trim-fonts and --leave-fonts
> --compact-fonts and --full-fonts
> --few-fonts and --all-fonts

--optimize-fonts and --no-optimize-fonts


    Werner

[pkx166h, 2017-09-27 06:59:25]

On 2017/09/27 04:55:31, wl_gnu.org wrote:
> > Somehow, concisely say "get-rid-of-anything-extra" versus
> > "keep-everything-that-comes?"
> > 
> > --min-fonts and --max-fonts
> > --compress-fonts and --expand-fonts
> 
> I would rather prefer --compress-fonts and --no-compress-fonts.  This
> is easier to remember (and rather common in GNU packages).
> 
> > --drop-fonts and --keep-fonts
> > --trim-fonts and --leave-fonts
> > --compact-fonts and --full-fonts
> > --few-fonts and --all-fonts
> 
> --optimize-fonts and --no-optimize-fonts
> 
> 
>     Werner

Werner beat me to it.

--optimize-fonts for me too please.

Although from a casual user's point of view why would I NOT want to 'optimize'
[anything]?

I.e why wouldn't this be the [silent] default (i.e. no switch required) and we
just have the '--no-optimize-fonts' option available. I am assuming it might be
a time-taken-to-process-the-PDF thing (I do apologise for not really being able
to follow the thread here). Anyway, I don't need any justification or
explanation. I just came by to '+1' Werner's suggestion really.

James

[dak, 2017-09-27 07:24:34]

Werner LEMBERG <wl@gnu.org> writes:

>> Somehow, concisely say "get-rid-of-anything-extra" versus
>> "keep-everything-that-comes?"
>> 
>> --min-fonts and --max-fonts
>> --compress-fonts and --expand-fonts
>
> I would rather prefer --compress-fonts and --no-compress-fonts.  This
> is easier to remember (and rather common in GNU packages).

They aren't compressed but unrecognizably unwrapped and the packaging
thrown away.

>> --drop-fonts and --keep-fonts
>> --trim-fonts and --leave-fonts
>> --compact-fonts and --full-fonts
>> --few-fonts and --all-fonts
>
> --optimize-fonts and --no-optimize-fonts

No, I don't think that captures the gist of the problem.

-- 
David Kastrup

[knupero, 2017-09-27 09:38:21]

> Bikeshedding done for the day.

Let me continue.

"lilypond --help" and "lilypond -dhelp" gives perfect help to all users. I doubt
that there are a lot of people who know which parameters to use for a
non-trivial case even after reading the help texts.

At least the following parameters influence ps / pdf output:

  --pdf
  --ps
  --use-encodings (or --bigpdf without Patch Set #2)
  -daux-files
  -dbackend
  -dgs-load-fonts
  -dgs-load-lily-fonts
  -dgs-never-embed-fonts

All names proposed in this thread (including my --use-encodings) only slightly
varies the high level of obscurity we have reached.

I think we should have for ps/pdf parameters that do not need -d...

  --ps               meaning: generate a ps file / keep the intermediate ps file
if also --pdf is given
  --eps              meaning: generate an eps file
  --pdf              meaning: generate a pdf file
  --pspdf-opt=..     meaning: ps/pdf optimization selection

possible values for the last parameter could be:

  --pdfodf-opt=help            meaning: display some help
  --pspdf-opt=size             meaning: optimize for file size
  --pdfodf-opt=lilypond-book   meaning: optimize for lilypond-book
  --pdfodf-opt=TeX             meaning: optimize for inclusion in *TeX
  --pdfodf-opt=TeX-GS          meaning: optimize for inclusion of multiple files
in *TeX and postprocessing with ghostscript
  --pdfodf-opt=...             meaning: ...

Those simple parameters would be interpreted as given below:

  --pspdf-opt=size would be the default and equivalent to our current defaults
(or something better if we find it in the future)
  --pspdf-opt=TeX would be equivalent to either the default or --use-encodings
(needs test). If combined with --eps it would also set -dauxfiles=#f
  --pspdf-opt=TeX-GS would be equivalent to --use-encodings
-dgs-never-embed-fonts, if combined with --eps it would also set -dauxfiles=#f
  --pdpdf-opt=lilypond-book would be equivalent to whatever users of lilypond
book prefer

Advantages: Significantly drops the level of obscurity, easy to extend.

Knut

[dak, 2017-09-27 11:22:31]

On 2017/09/27 09:38:21, knupero wrote:
> > Bikeshedding done for the day.
> 
> Let me continue.
> 
> "lilypond --help" and "lilypond -dhelp" gives perfect help to all users. I
doubt
> that there are a lot of people who know which parameters to use for a
> non-trivial case even after reading the help texts.
> 
> At least the following parameters influence ps / pdf output:
> 
>   --pdf
>   --ps
>   --use-encodings (or --bigpdf without Patch Set #2)
>   -daux-files
>   -dbackend
>   -dgs-load-fonts
>   -dgs-load-lily-fonts
>   -dgs-never-embed-fonts
> 
> All names proposed in this thread (including my --use-encodings) only slightly
> varies the high level of obscurity we have reached.
> 
> I think we should have for ps/pdf parameters that do not need -d...
> 
>   --ps               meaning: generate a ps file / keep the intermediate ps
file
> if also --pdf is given
>   --eps              meaning: generate an eps file
>   --pdf              meaning: generate a pdf file
>   --pspdf-opt=..     meaning: ps/pdf optimization selection
> 
> possible values for the last parameter could be:
> 
>   --pdfodf-opt=help            meaning: display some help
>   --pspdf-opt=size             meaning: optimize for file size
>   --pdfodf-opt=lilypond-book   meaning: optimize for lilypond-book
>   --pdfodf-opt=TeX             meaning: optimize for inclusion in *TeX
>   --pdfodf-opt=TeX-GS          meaning: optimize for inclusion of multiple
files
> in *TeX and postprocessing with ghostscript
>   --pdfodf-opt=...             meaning: ...

How about just --pdf=<purpose> ?  One option less to remember.  This is going to
be double the fun once we can generate PDF bypassing PS altogether.

> Advantages: Significantly drops the level of obscurity, easy to extend.

Also: can keep using the same purpose-bound options as our ways of dancing
around the current version of Ghostscript change.

Sounds like a good strategy.  Just not in time for 2.20.

[knupero, 2017-09-27 11:49:51]

> How about just --pdf=<purpose> ?  One option less to remember.  This is going
to
> be double the fun once we can generate PDF bypassing PS altogether.

Yes, even better.
 
> Sounds like a good strategy.  Just not in time for 2.20.

I tend to agree.

[dak, 2017-09-27 12:16:44]

On 2017/09/27 11:49:51, knupero wrote:
> > How about just --pdf=<purpose> ?  One option less to remember.  This is
going
> to
> > be double the fun once we can generate PDF bypassing PS altogether.
> 
> Yes, even better.
>  
> > Sounds like a good strategy.  Just not in time for 2.20.
> 
> I tend to agree.

Well, so much for my plan of making the same options work for 2.20 and 2.21. 
Perhaps one can amend the set of -dxxx specialist options and keep them around,
and then in 2.21 implement the more high-level purpose-based system that sets a
bunch of subordinate options then.

Then at least the specialist calls will keep doing the same (even though the
_motivation_ for using them might change, so it might lead to people keeping
behavior that they don't need any more).

[knupero, 2017-09-27 22:49:24]

introduce --pdfsizeopt=size, TeX or TeX-GS, no --bigpdf, no --use-encodings

[knupero, 2017-09-27 22:52:30]

lilypond --help gives the following output:

Usage: lilypond [OPTION]... FILE...

Typeset music and/or produce MIDI from FILE.

LilyPond produces beautiful music notation.
For more information, see http://lilypond.org

Options:
  -f, --formats=FORMATs               dump FORMAT,...  Also as separate options:
      --pdf                           generate PDF files (default)
      --png                           generate PNG files 
      --ps                            generate PostScript files
      --eps                           generate Encapsulated PostScript files
  -O, --pspdfopt=KEY                  set ps/pdf optimization to KEY (default:
size)
                                        KEY is either 'size', 'TeX' or 'TeX-GS'
  -d, --define-default=SYM[=VAL]      set Scheme option SYM to VAL (default:
#t).
                                        Use -dhelp for help.
  -e, --evaluate=EXPR                 evaluate scheme code
  -h, --help                          show this help and exit
  -H, --header=FIELD                  dump header field FIELD to file
                                        named BASENAME.FIELD
  -I, --include=DIR                   add DIR to search path
  -i, --init=FILE                     use FILE as init file
  -j, --jail=USER, GROUP, JAIL, DIR   chroot to JAIL, become USER:GROUP
                                        and cd into DIR
  -l, --loglevel=LOGLEVEL             print log messages according to LOGLEVEL. 
Possible values are:
                                        NONE, ERROR, WARNING, BASIC, PROGRESS,
INFO (default) and DEBUG.
  -o, --output=FILE                   write output to FILE (suffix will be
added)
      --relocate                      relocate using directory of lilypond
program
  -s, --silent                        no progress, only error messages
(equivalent to loglevel=ERROR)
  -v, --version                       show version number and exit
  -V, --verbose                       be verbose (equivalent to loglevel=DEBUG)
  -w, --warranty                      show warranty and copyright

Report bugs via
http://post.gmane.org/post.php?group=gmane.comp.gnu.lilypond.bugs

[wl_gnu.org, 2017-09-28 05:00:41]

> lilypond --help gives the following output: [...]

>   -O, --pspdfopt=KEY                  set ps/pdf optimization to KEY (default:
size)
>                                         KEY is either 'size', 'TeX' or
'TeX-GS'

LGTM, thanks!

BTW, is there any chance to reduce the width of the help screen to
stay within 78 characters?


    Werner

[dak, 2017-09-28 06:16:39]

Werner LEMBERG <wl@gnu.org> writes:

>> lilypond --help gives the following output: [...]
>
>>   -O, --pspdfopt=KEY                  set ps/pdf optimization to KEY
(default: size)
>>                                         KEY is either 'size', 'TeX' or
'TeX-GS'
>
> LGTM, thanks!
>
> BTW, is there any chance to reduce the width of the help screen to
> stay within 78 characters?

While we are at it, it may make sense not to refer to the no longer
existing Gmane page for bug reports.

-- 
David Kastrup

[knupero, 2017-09-28 10:15:26]

fix --eps, change bug report msg, fix documentation

[knupero, 2017-09-28 10:19:13]

> BTW, is there any chance to reduce the width of the help screen to
> stay within 78 characters?

I don't think so

[knupero, 2017-09-28 10:26:05]

> While we are at it, it may make sense not to refer to the no longer
> existing Gmane page for bug reports.

I changed the message, but there are 58 more places where post.gmane is
referenced in the tree.
And there 113 mor place that reference othere places of gmane.

At least http://lilypond.org/bug-reports.html needs to be updated soon.

[dak, 2017-09-28 11:04:03]

On 2017/09/28 10:26:05, knupero wrote:
> > While we are at it, it may make sense not to refer to the no longer
> > existing Gmane page for bug reports.
> 
> I changed the message, but there are 58 more places where post.gmane is
> referenced in the tree.
> And there 113 mor place that reference othere places of gmane.

Ouch.

> At least http://lilypond.org/bug-reports.html needs to be updated soon.

I thought this one was already fixed.

[trueroad, 2017-09-29 12:01:41]

LGTM


For the future, I would like to prefer Identity-H encoding rather than 3 custom
encodings.

If I understand correctly,
All glyphs can be used with Identity-H encoding.
No other encoding is necessary.

For current LilyPond and Ghostscript, when using CID-keyed font, Identity-H
encoding PDF is generated.
If you use the same CID-keyed font in many intermediate PDFs, you can unify them
into one font in the final PDF no matter what glyphs you use.

Unfortunately, non-CID-keyed fonts can not use Identity-H encoding.
XeTeX and LuaTeX seem to convert non-CID-Keyed font to CID-keyed font on the
fly, and embed it to PDF, use Identity-H encoding.

[pkx166h, 2017-10-03 13:54:26]

On 2017/09/28 11:04:03, dak wrote:
> On 2017/09/28 10:26:05, knupero wrote:
> > > While we are at it, it may make sense not to refer to the no longer
> > > existing Gmane page for bug reports.
> > 
> > I changed the message, but there are 58 more places where post.gmane is
> > referenced in the tree.
> > And there 113 mor place that reference othere places of gmane.
> 
> Ouch.
> 
> > At least http://lilypond.org/bug-reports.html needs to be updated soon.
> 
> I thought this one was already fixed.

Not quite.

https://sourceforge.net/p/testlilyissues/issues/4981/

[pkx166h, 2017-10-03 13:55:30]

On 2017/09/29 12:01:41, trueroad wrote:
> LGTM
> 
> 
> For the future, I would like to prefer Identity-H encoding rather than 3
custom
> encodings.
> 
> If I understand correctly,
> All glyphs can be used with Identity-H encoding.
> No other encoding is necessary.
> 
> For current LilyPond and Ghostscript, when using CID-keyed font, Identity-H
> encoding PDF is generated.
> If you use the same CID-keyed font in many intermediate PDFs, you can unify
them
> into one font in the final PDF no matter what glyphs you use.
> 
> Unfortunately, non-CID-keyed fonts can not use Identity-H encoding.
> XeTeX and LuaTeX seem to convert non-CID-Keyed font to CID-keyed font on the
> fly, and embed it to PDF, use Identity-H encoding.

Should we create a tracker for this?

[knupero, 2017-10-04 17:09:52]

> > Unfortunately, non-CID-keyed fonts can not use Identity-H encoding.
> > XeTeX and LuaTeX seem to convert non-CID-Keyed font to CID-keyed font on the
> > fly, and embed it to PDF, use Identity-H encoding.
> 
> Should we create a tracker for this?

I think whoever has an idea and the time to solve one of the remaining
postscript problems will create a tracker.