Re: vfat: Broken case-insensitive support for UTF-8

From: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
To: "Pali Rohár" <pali.rohar@gmail.com>
Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	"Theodore Y. Ts'o" <tytso@mit.edu>,
	Namjae Jeon <linkinjeon@gmail.com>,
	Gabriel Krisman Bertazi <krisman@collabora.com>
Subject: Re: vfat: Broken case-insensitive support for UTF-8
Date: Mon, 20 Jan 2020 21:07:12 +0900	[thread overview]
Message-ID: <875zh6pc0f.fsf@mail.parknet.co.jp> (raw)
In-Reply-To: <20200120110438.ak7jpyy66clx5v6x@pali> ("Pali =?iso-8859-1?Q?Roh=E1r=22's?= message of "Mon, 20 Jan 2020 12:04:38 +0100")

Pali Rohár <pali.rohar@gmail.com> writes:

>> To be perfect, the table would have to emulate what Windows use. It can
>> be unicode standard, or something other.
>
> Windows FAT32 implementation (fastfat.sys) is opensource. So it should
> be possible to inspect code and figure out how it is working.
>
> I will try to look at it.

I don't think the conversion library is not in fs driver though,
checking implement itself would be good.

>> And other fs can use different what Windows use.
>> 
>> So the table would have to be switchable in perfect world (if there is
>> no consensus to use 1 table).  If we use switchable table, I think it
>> would be better to put in userspace, and loadable like firmware data.
>> 
>> Well, so then it would not be simple work (especially, to be perfect).
>
> Switchable table is not really simple and I think as a first step would
> be enough to have one (hardcoded) table for UTF-8. Like we have for all
> other encodings.

Ignoring if utf8 table is good or not.  If the table is not windows
compatible or doesn't satisfy other fs's requirement, it also is yet
another broken table like now (of course, it would likely be better
off).  Of course, we can define it as linux implementation limitation
though.

So yes, I think this work is not simple.

>> Also, not directly same issue though. There is related issue for
>> case-insensitive. Even if we use some sort of internal wide char
>> (e.g. in nls, 16bits), dcache is holding name in user's encode
>> (e.g. utf8). So inefficient to convert cached name to wide char for each
>> access.
>
> Yes, this is truth. But this conversion is already doing exFAT
> implementation. I think we do not have other choice if we want Windows
> compatible implementation.

For example, we can cache the both of display name, and upper/lower case
name. Anyway, at least, there are some implement options.

Thanks.
-- 
OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>