Re: why do we need utf8 normalization when compare name?

["Theodore Y. Ts'o" <tytso@mit.edu>]

On Mon, Mar 02, 2020 at 03:28:18PM +0000, Al Viro wrote:
> Why the hell do we need case-insensitive filesystems in the first place?
> I have only heard two explanations:
> 	1) because the layout (including name equivalences) is fixed by
> some OS that happens to be authoritative for that filesystem.  In that
> case we need to match the rules of that OS, whatever they are.  Unicode
> equivalence may be an interesting part of _their_ background reasons
> for setting those rules, but the only thing that really matters is what
> rules have they set.

It significantly helps porting applications that were originally
written for Windows and/or MacOS.  In particular, the work to add
Unicode comparison support to ext4 was funded to enable the ability to
run Windows gaming applications on Linux for Steam.


> 	2) early Android used to include a memory card with VFAT on
> it; the card is long gone, but crapplications came to rely upon having
> that shit.  And rather than giving them a file on the normal filesystem
> with VFAT image on it and /dev/loop set up and mounted, somebody wants
> to use parts of the normal (ext4) filesystem for it.  However, the
> same crapplications have come to rely upon the case-insensitive (sensu
> VFAT) behaviour there, so we must duplicate that vomit-inducing pile
> of hacks on ext4.  Ideally - with that vomit-induc{ing,ed} pile
> reclassified as a generic feature; those look more respectable.

There are a number of reasons why a loop device is not sufficient;
there is a requirement to have selective sharing of application data
between other applications, which is done using user/group ownership.

For Android, previously, this was done using sdcardfs which was based
off of wrapfs.  Wrapfs was the same base as unionfs, and as you may
recall, it had even more horrendous race issues, and more than once I
was asked to help to debug crashes when you fsstress was run on
different views of sdcardfs.  I've been strongly encouraging the push
to something more sane, but a requirement for this is case-folded
directory.

There is a third reason why case folding is necessary, which is for
file serving applications such as Samba which require case-folding,
and without file system level support, trying to simulate this in
userspace requires searching via readdir to do a case-insensitive
lookup (at least in some case).  So finding a more performant way to
support case-folding is a big help for applications like Samba.

Cheers,

						- Ted