Allow use of Identity-H CMap and CID versions of Emmentaler

Allow use of Identity-H CMap and CID versions of Emmentaler

When -O TeX or -O TeX-GS is given, lilypond generates
postscript code that instructs ghostscript to use the
Identity-H CMap and CID versions of the emmentaler
fonts. Used e.g. by 'make doc'.

Without '-O TeX' or '-O TeX-GS' we still use the
traditional postscript code.

Documentation is missing.

Only a proof of concept, definitely not ready for
release.

Tested only with current lilypond and ghostscript master.

Signed-off-by: Knut Petersen <knut_petersen@t-online.de>

[knupero, 2018-05-26 13:20:46]

Please review.

[Carl, 2018-05-26 15:17:09]

The patch looks well put together.

However, I believe that the case problem with MacOS needs to be fixed.  I
believe this patch won't work on MacOS -- only one version of the font will be
installed (the last one to be created).

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile#newcode145
mf/GNUmakefile:145: cd $(outdir) && mv $(notdir $@).otf $(notdir $@)
I think there is potential confusion where the CID file has no extension, and 
just has capitalization to indicate it.  Is this a standard for CID-keyed fonts?

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in
File mf/emmentaler-brace.pe.in (right):

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in#new...
mf/emmentaler-brace.pe.in:89:
ConvertByCMap("/usr/share/ghostscript/9.23/Resource/CMap/Identity-H")
This hard-coded 9.23 seems to me to be broken in advance.

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in#new...
mf/emmentaler-brace.pe.in:90: Generate("Emmentaler-Brace.otf")
Will this work on MacOS?  IIRC, the default file system on MacOS is
case-insensitive, although it allows the used of mixed case.  THis means you can
have emmentaler-brace,otf or Emmentaler-Brace.otf, but I don't think you can
have both by default on MacOS.  

For compatibility, I think there needs to be a different name.

Here's an example that shows the problem:

Carls-MBP:junk carlsorensen$ echo "This is a test" > test
Carls-MBP:junk carlsorensen$ cat test
This is a test
Carls-MBP:junk carlsorensen$ echo "This is an UpperCase test " > Test
Carls-MBP:junk carlsorensen$ cat Test
This is an UpperCase test 
Carls-MBP:junk carlsorensen$ cat test
This is an UpperCase test

https://codereview.appspot.com/343970043/diff/1/ps/cidres.in
File ps/cidres.in (right):

https://codereview.appspot.com/343970043/diff/1/ps/cidres.in#newcode1
ps/cidres.in:1: lilypond-datadir (/fonts/otf/Emmentaler-11) concatstrings (r)
file .loadfont
The hardcoding of all the font sizes here seems broken.  If we change the sizes
to be generated from the .mf files; then this won't work any more.  It seems
there should be one variable that lists all of the sizes to be generated, and
that variable should be used throughout the build.

I guess that would meant that rather than having cidres.ps be a
developer-created file, there should be some kind of a template file and a make
command that would cat together cidres.ps using the template file and the font
size variable.

But recognizing that encodingdefs.ps doesn't follow this practice in the current
distribution, this comment may be without merit (or else it would apply equally
to encodingdefs.ps as well.

[lemzwerg, 2018-05-26 16:25:25]

LGTM, but I share Carl's concerns, in particular the issue  with font names that
only differ in case.

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile#newcode145
mf/GNUmakefile:145: cd $(outdir) && mv $(notdir $@).otf $(notdir $@)
On 2018/05/26 15:17:09, Carl wrote:
> I think there is potential confusion where the CID file has no extension,
> and just has capitalization to indicate it.  Is this a standard for
> CID-keyed fonts?

Actually, it is, but I don't see a reason to not use `.cid' as an extension.

[dak, 2018-05-26 16:45:09]

lemzwerg@googlemail.com writes:

> LGTM, but I share Carl's concerns, in particular the issue  with font
> names that only differ in case.

That sounds like a problem not just on MacOSX but also on Windows.

> https://codereview.appspot.com/343970043/

-- 
David Kastrup

[knupero, 2018-05-26 22:24:49]

> However, I believe that the case problem with MacOS needs to be fixed.  I
> believe this patch won't work on MacOS -- only one version of the font will be
> installed (the last one to be created).

I wasn't aware of MacOS case handling, I'll have a look at it.
Thanks for pointing out that problem.

[knupero, 2018-05-26 22:48:39]

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile#newcode145
mf/GNUmakefile:145: cd $(outdir) && mv $(notdir $@).otf $(notdir $@)
On 2018/05/26 15:17:09, Carl wrote:
> I think there is potential confusion where the CID file has no extension, and 
> just has capitalization to indicate it.  Is this a standard for CID-keyed
fonts?

ghostscript uses fontname=filename with no extension for CID replacement fonts.
In an old version of the code I used this mechanism of ghostscript. With this
version of the code the CID font is explicitely loaded in encodingdefs.ps, so
almost any name might me used.

The file emmentaler-20.otf defines the non-cid font Emmentaler-20. It is used by
lilypond during generation of the ps code. We need this non-cid font as it
contains the glyph names we use in lilypond. 

If during processing of the ps code contained in encodingdefs.ps ghostscript
sees that the non-cid Emmentaler-20 font is used, the code instructs gs to load
the cid font. Then it used the same name Emmentaler-20 again for the font
composed from the Emmentaler-20 font and the Identity-H CMap.

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in
File mf/emmentaler-brace.pe.in (right):

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in#new...
mf/emmentaler-brace.pe.in:89:
ConvertByCMap("/usr/share/ghostscript/9.23/Resource/CMap/Identity-H")
On 2018/05/26 15:17:09, Carl wrote:
> This hard-coded 9.23 seems to me to be broken in advance.

Yes, it is broken, I'll fix it. Thanks for spotting.

https://codereview.appspot.com/343970043/diff/1/ps/cidres.in
File ps/cidres.in (right):

https://codereview.appspot.com/343970043/diff/1/ps/cidres.in#newcode1
ps/cidres.in:1: lilypond-datadir (/fonts/otf/Emmentaler-11) concatstrings (r)
file .loadfont
Both cidres.ps and encodingdefs.ps could and should be generated, yes. This is
one of the reasons why I wrote about proof of concept, not ready for release.

[trueroad, 2018-05-27 02:57:22]

Great!
But, I failed to build in my Cygwin (case insensitive) environment.

>> I think there is potential confusion where the CID file has no extension,
>> and just has capitalization to indicate it.  Is this a standard for
>> CID-keyed fonts?
>
> Actually, it is, but I don't see a reason to not use `.cid' as an extension.

If I understand correctly, CID-keyed OpenType fonts have `.otf' extension
instead of `.cid' extension.
e.g. `SourceHanSerif-Regular.otf' is a CID-keyed font.

[knupero, 2018-05-28 07:34:39]

Patchset #2 (support case insensitive filename namespaces, etc)

[knupero, 2018-05-29 22:44:44]

Also use CMap Identity-H for TeX Gyre Schola family, some fixes and
enhancements.

[knupero, 2018-05-29 22:49:50]

On 2018/05/29 22:44:44, knupero wrote:
> Also use CMap Identity-H for TeX Gyre Schola family, some fixes and
> enhancements.

It would be nice to know if this code also works on Windows and MacOS systems.

[lemzwerg, 2018-05-30 04:31:55]

Nice!

https://codereview.appspot.com/343970043/diff/40001/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/40001/mf/GNUmakefile#newcode183
mf/GNUmakefile:183: echo XXXX $(TEXGYRE_DIR)/texgyreschola-regular.otf YYY
This looks like a debug message, probably to be removed (or improved)...

https://codereview.appspot.com/343970043/diff/40001/mf/GNUmakefile#newcode201
mf/GNUmakefile:201: otfinfo -u $(TEXGYRE_DIR)/texgyreschola-regular.otf | \
I suggest to start a line with the pipe symbol:

    foo \
    | bar

[knupero, 2018-05-30 11:50:19]

cleanup / change command line / add otfinfo check / etc / change default ps
output mode

[knupero, 2018-05-30 12:23:58]

Changes:

Command line:
"-O size"  is gone
"-O legacy" is added, uses our old non-encodings code
"-O TeX" new default as the pdf files are smaller.
"-O TeX-GS" use if you want to include lilypond pdfs in TeX document and
postprocess the resulting pdfs with ghostscript. You also need to give
/path/cidres.ps to gs.

If you use -dsafe also use -O legacy. I hope I'll fix that soon. 

BTW: -dsafe on windows system is broken since 2005. Don't rely on it.

Not only fonts, but also most of our ps files are loaded by gs if -O TeX* is
given.

-dmusic-font-encodings is gone as the code has been ripped out.
-duse-cid-fonts is implemented.

-O legacy/TeX/TeX-GS set gs_load_font, gs-never-embed-fonts, use-cid-fonts to
the appropriate values, you never need to use those parameters directly


The minimal lilypond source file t

   \version "2.21.0"
   \score { c''' }

translated with 'lilypond -O legacy --ps --pdf t.ly ; dir t.ps t.pdf; pdffonts
t.pdf' gives:

-rw-r--r-- 1 knut users  26580 30. Mai 14:14 t.pdf
-rw------- 1 knut users 150449 30. Mai 14:14 t.ps

name                                 type              encoding         emb sub
uni object ID
------------------------------------ ----------------- ---------------- --- ---
--- ---------
BIKQCO+Emmentaler-20                 Type 1C           Custom           yes yes
no       8  0
GCWIIH+TeXGyreSchola-Regular         Type 1C           Custom           yes yes
no      12  0


The same file translated with 'lilypond -O TeX --ps --pdf t.ly ; dir t.ps t.pdf;
pdffonts t.pdf' gives:

-rw-r--r-- 1 knut users 22589 30. Mai 14:14 t.pdf
-rw------- 1 knut users  7041 30. Mai 14:14 t.ps

name                                 type              encoding         emb sub
uni object ID
------------------------------------ ----------------- ---------------- --- ---
--- ---------
XZJJSW+TeXGyreSchola-Regular         CID Type 0C       Identity-H       yes yes
no      15  0
LDRNTC+Emmentaler-20                 CID Type 0C       Identity-H       yes yes
no       9  0

TODO:
  documentation
  proper handling of -dsafe, also fixing the old windows security issue
mentioned above
  [...]

[knupero, 2018-05-30 12:32:14]

Answers to comments.

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/1/mf/GNUmakefile#newcode145
mf/GNUmakefile:145: cd $(outdir) && mv $(notdir $@).otf $(notdir $@)
On 2018/05/26 16:25:25, lemzwerg wrote:
> On 2018/05/26 15:17:09, Carl wrote:
> > I think there is potential confusion where the CID file has no extension,
> > and just has capitalization to indicate it.  Is this a standard for
> > CID-keyed fonts?
> 
> Actually, it is, but I don't see a reason to not use `.cid' as an extension.

Done.

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in
File mf/emmentaler-brace.pe.in (right):

https://codereview.appspot.com/343970043/diff/1/mf/emmentaler-brace.pe.in#new...
mf/emmentaler-brace.pe.in:90: Generate("Emmentaler-Brace.otf")
On 2018/05/26 15:17:09, Carl wrote:
> Will this work on MacOS?  IIRC, the default file system on MacOS is
> case-insensitive, although it allows the used of mixed case.  THis means you
can
> have emmentaler-brace,otf or Emmentaler-Brace.otf, but I don't think you can
> have both by default on MacOS.  
> 
> For compatibility, I think there needs to be a different name.
> 
> Here's an example that shows the problem:
> 
> Carls-MBP:junk carlsorensen$ echo "This is a test" > test
> Carls-MBP:junk carlsorensen$ cat test
> This is a test
> Carls-MBP:junk carlsorensen$ echo "This is an UpperCase test " > Test
> Carls-MBP:junk carlsorensen$ cat Test
> This is an UpperCase test 
> Carls-MBP:junk carlsorensen$ cat test
> This is an UpperCase test 

Done.

https://codereview.appspot.com/343970043/diff/1/ps/cidres.in
File ps/cidres.in (right):

https://codereview.appspot.com/343970043/diff/1/ps/cidres.in#newcode1
ps/cidres.in:1: lilypond-datadir (/fonts/otf/Emmentaler-11) concatstrings (r)
file .loadfont
On 2018/05/26 15:17:09, Carl wrote:
> The hardcoding of all the font sizes here seems broken.  If we change the
sizes
> to be generated from the .mf files; then this won't work any more.  It seems
> there should be one variable that lists all of the sizes to be generated, and
> that variable should be used throughout the build.

Yes. But as I doubt that anyone will change the design sizes in the next decade
I think I postpone a fix.

https://codereview.appspot.com/343970043/diff/40001/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/40001/mf/GNUmakefile#newcode183
mf/GNUmakefile:183: echo XXXX $(TEXGYRE_DIR)/texgyreschola-regular.otf YYY
On 2018/05/30 04:31:54, lemzwerg wrote:
> This looks like a debug message, probably to be removed (or improved)...

Done.

https://codereview.appspot.com/343970043/diff/40001/mf/GNUmakefile#newcode201
mf/GNUmakefile:201: otfinfo -u $(TEXGYRE_DIR)/texgyreschola-regular.otf | \
On 2018/05/30 04:31:54, lemzwerg wrote:
> I suggest to start a line with the pipe symbol:
> 
>     foo \
>     | bar

Done.

[trueroad, 2018-06-02 02:48:26]

https://codereview.appspot.com/343970043/diff/60001/mf/GNUmakefile
File mf/GNUmakefile (right):

https://codereview.appspot.com/343970043/diff/60001/mf/GNUmakefile#newcode165
mf/GNUmakefile:165: cd $(outdir) && mv $(notdir $<) $(notdir $@)
Is it necessary to change the file name?
I think that the extension `.otf' is more suitable than the extension `.cid'
since `emmentaler-*-cid.otf' is in Open Type Font format.

If I understand correctly, a file with extension `.cid' means old CID format
instead of OTF format.
The OTF format can contain a CID keyed fonts as well as non-CID keyed fonts.