Rietveld Code Review Tool
Help | Bug tracker | Discussion group | Source code | Sign in
(842)

Issue 206640043: Add support for unicode filenames on Windows (Closed)

Can't Edit
Can't Publish+Mail
Start Review
Created:
9 years ago by pkx166h
Modified:
7 years ago
Reviewers:
CC:
lilypond-devel_gnu.org, trueroad_sea.plala.or.jp
Visibility:
Public.

Description

Add support for unicode filenames on Windows On Windows, LilyPond can't handle unicode filenames. The patch replaces main(), and hooks filename related functions. This converts between UTF-16 unicode (Windows) and UTF-8 unicode (LilyPond, libguile etc.). LilyPond can handle unicode filenames for *.ly, *.mid, *.ps. *.pdf is not supported yet. Ghostscript-8.70 that is included in the binary distribution of LilyPond can't handle unicode filenames. This requires Ghostscript-9.10 or later

Patch Set 1 #

Unified diffs Side-by-side diffs Delta from patch set Stats (+523 lines, -0 lines) Patch
A flower/include/mingw-utf8.hh View 1 chunk +11 lines, -0 lines 0 comments Download
A flower/include/mingw-utf8-conv.hh View 1 chunk +7 lines, -0 lines 0 comments Download
A flower/include/mingw-utf8-func.hh View 1 chunk +16 lines, -0 lines 0 comments Download
A flower/include/mingw-utf8-hook.hh View 1 chunk +6 lines, -0 lines 0 comments Download
A flower/mingw-utf8-conv.cc View 1 chunk +58 lines, -0 lines 0 comments Download
A flower/mingw-utf8-func.cc View 1 chunk +157 lines, -0 lines 0 comments Download
A flower/mingw-utf8-hook.cc View 1 chunk +179 lines, -0 lines 0 comments Download
A flower/mingw-utf8-main.cc View 1 chunk +84 lines, -0 lines 0 comments Download
M lily/GNUmakefile View 1 chunk +1 line, -0 lines 0 comments Download
M lily/main.cc View 1 chunk +4 lines, -0 lines 0 comments Download

Messages

Total messages: 1
pkx166h
9 years ago (2015-03-09 10:50:08 UTC) #1
Pasting update in Tracker by David K for Masamichi's benefit:

--snip--

by dak@gnu.org: Patch: Add support for unicode filenames on Windows
https://code.google.com/p/lilypond/issues/detail?id=4317

Ok, I've been stalling on this patch for about a week because I don't have a
good idea how to address it properly.  So let's start with the worst first: at
the current point of time, I don't see it making sense, and in addition I will
not be able to spend significant amount of time addressing the problems arising
in connection with it for about a month, so I'd like a moratorium on it for
about that time.

The main problem is that the next step needed for GUILEv2 migration (which
nobody is really interested in working on apart from myself) is to get the
coding mess sorted out.  GUILEv1 is basically working with byte streams in its
strings and string ports and files.  GUILEv2 is coding system aware, uses either
Latin-1 or UTF-32 as its string internals, uses UTF-8 for string ports
(necessitating copies and conversions rather than being able to work in-place)
and converts back and forth all the time.

Since LilyPond has a lexer working on an UTF-8 coded byte stream and data is
liberally bounced around between files, strings, and string ports, and the
respective position pointers are not distinguished.

It does not help that GUILEv2 changes its ways to pick particular encodings
basically every 3 or 4 "stable" versions in the 2.0 series, and 2.1 is different
again.  So we need to carefully refactor the code in order to have a chance of
it working in both GUILEv1 and GUILEv2 (and at all in GUILEv2).

This is an ugly and ungrateful task that has been at the front of my
this-is-really-up-next-so-let's-procrastinate-instead queue the last month or
so.  Letting a patch like this in right now would make the situation worse.

Viewed realistically, this patch also needs Ghostscript updates in GUB to make
any sense.  There is no reason _those_ cannot go ahead first: having a buffer of
a few developer releases where we might discover and sort out possible problems
with newer versions of GhostScript on our crosscompiled distributions seems
sensible.

Personally, I don't really like the look of this patch at all as it is very
specifically useful only for one platform, and apart from that I'd very much
want to avoid seeing any wide character functions in LilyPond code: that's
rarely exercised library code of often dubious reliability or availability, and
most programmers are not overly comfortable utilizing it.  I'd very much prefer
that we'd find a better encapsulated solution, probably by utilizing the
encoding support of GUILEv2.  Which again suggests that a moratorium on this
patch makes sense, but that one would not be "until David gets encodings sorted
out so that we can compile and test on GUILEv2" but rather on "until we are able
to ditch GUILEv1 altogether".  And given the current enthusiasm for working on
the GUILEv2 migration and the likelihood of necessary bug fixes in GUILE itself
becoming available as we discover problems, the latter is likely quite longer
than a month away.

The problem this patch tries to address is not a recent problem, and the
workaround ("don't use special Unicode characters in filenames") is obvious.  So
the urgency seems limited.  I very much agree that LilyPond should not balk at
file and directory names containing accented characters on all platforms.  But I
think we will be doing ourselves our favor to postpone addressing this problem
until we have moved LilyPond to GUILEv2, and then look for the best-matching
solution within _this_ framework.  Otherwise we complicate the coding issue work
for GUILEv2 migration and that's something we can afford less than prolonging
the current filename encoding situation.

-- 
You received this message because you are the owner of the issue.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

Reply to this email to add a comment or make updates.
Sign in to reply to this message.

Powered by Google App Engine
RSS Feeds Recent Issues | This issue
This is Rietveld f62528b