From: "Pali Rohár" <pali@kernel.org> To: OGAWA Hirofumi <hirofumi@mail.parknet.co.jp> Cc: linux-fsdevel@vger.kernel.org, linux-ntfs-dev@lists.sourceforge.net, linux-cifs@vger.kernel.org, jfs-discussion@lists.sourceforge.net, linux-kernel@vger.kernel.org, "Alexander Viro" <viro@zeniv.linux.org.uk>, "Jan Kara" <jack@suse.cz>, "Theodore Y . Ts'o" <tytso@mit.edu>, "Luis de Bethencourt" <luisbg@kernel.org>, "Salah Triki" <salah.triki@gmail.com>, "Andrew Morton" <akpm@linux-foundation.org>, "Dave Kleikamp" <shaggy@kernel.org>, "Anton Altaparmakov" <anton@tuxera.com>, "Pavel Machek" <pavel@ucw.cz>, "Marek Behún" <marek.behun@nic.cz>, "Christoph Hellwig" <hch@infradead.org> Subject: Re: [RFC PATCH 01/20] fat: Fix iocharset=utf8 mount option Date: Sun, 15 Aug 2021 11:42:24 +0200 [thread overview] Message-ID: <20210815094224.dswbjywnhvajvzjv@pali> (raw) In-Reply-To: <87h7frtlu0.fsf@mail.parknet.co.jp> On Sunday 15 August 2021 12:42:47 OGAWA Hirofumi wrote: > Pali Rohár <pali@kernel.org> writes: > > > Currently iocharset=utf8 mount option is broken and error is printed to > > dmesg when it is used. To use UTF-8 as iocharset, it is required to use > > utf8=1 mount option. > > > > Fix iocharset=utf8 mount option to use be equivalent to the utf8=1 mount > > option and remove printing error from dmesg. > > This change is not equivalent to utf8=1. In the case of utf8=1, vfat > uses iocharset's conversion table and it can handle more than ascii. > > So this patch is incompatible changes, and handles less chars than > utf8=1. So I think this is clean though, but this would be regression > for user of utf8=1. I do not think so... But please correct me, as this code around is mess. Without this change when utf8=1 is set then iocharset= encoding is used for case-insensitivity implementation (toupper / tolower conversion). For all other parts are use correct utf8* conversion functions. But you use touppper / tolower functions from iocharset= encoding on stream of utf8 bytes then you either get identity or some unpredictable garbage in utf8. So when comparing two (different) non-ASCII filenames via this method you in most cases get that filenames are different. Because converting their utf8 bytes via toupper / tolower functions from iocharset= encoding results in two different byte sequences in most cases. Even for two utf8 case-insensitive same strings. But you can play with it and I guess it is possible to find two different utf8 strings which after toupper / tolower conversion from some iocharset= encoding would lead to same byte sequence. This patch uses for utf8 tolower / touppser function simple 7-bit tolower / toupper ascii function. And so for 7-bit ascii file names there is no change. So this patch changes behavior when comparing non 7-bit ascii file names, but only in cases when previously two different file names were marked as same. As now they are marked correctly as different. So this is changed behavior, but I guess it is bug fix which is needed. If you want I can put this change into separate patch. Issue that two case-insensitive same files are marked as different is not changed by this patch and therefore this issue stay here. > Thanks. > -- > OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
next prev parent reply other threads:[~2021-08-15 9:42 UTC|newest] Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-08-08 16:24 [RFC PATCH 00/20] fs: Remove usage of broken nls_utf8 and drop it Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 01/20] fat: Fix iocharset=utf8 mount option Pali Rohár 2021-08-15 3:42 ` OGAWA Hirofumi 2021-08-15 9:42 ` Pali Rohár [this message] 2021-08-15 11:23 ` OGAWA Hirofumi 2021-08-23 3:51 ` Kari Argillander 2021-08-08 16:24 ` [RFC PATCH 02/20] hfsplus: Add iocharset= mount option as alias for nls= Pali Rohár 2021-08-09 17:51 ` Viacheslav Dubeyko 2021-08-09 20:49 ` Kari Argillander 2021-08-09 21:25 ` Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 03/20] udf: Fix iocharset=utf8 mount option Pali Rohár 2021-08-12 14:17 ` Jan Kara 2021-08-12 15:51 ` Pali Rohár 2021-08-13 13:48 ` Jan Kara 2021-08-19 8:34 ` Pali Rohár 2021-08-19 10:41 ` Jan Kara 2021-08-08 16:24 ` [RFC PATCH 04/20] isofs: joliet: " Pali Rohár 2021-08-12 14:18 ` Jan Kara 2021-08-08 16:24 ` [RFC PATCH 05/20] ntfs: Undeprecate iocharset= " Pali Rohár 2021-08-09 20:52 ` Kari Argillander 2021-08-19 1:21 ` Kari Argillander 2021-08-19 8:12 ` Pali Rohár 2021-08-19 10:23 ` Kari Argillander 2021-08-19 22:04 ` Pali Rohár 2021-08-19 23:18 ` Kari Argillander 2021-08-08 16:24 ` [RFC PATCH 06/20] ntfs: Fix error processing when load_nls() fails Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 07/20] befs: Fix printing iocharset= mount option Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 08/20] befs: Rename enum value Opt_charset to Opt_iocharset to match " Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 09/20] befs: Fix error processing when load_nls() fails Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 10/20] befs: Allow to use native UTF-8 mode Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 11/20] hfs: Explicitly set hsb->nls_disk when hsb->nls_io is set Pali Rohár 2021-08-09 17:31 ` Viacheslav Dubeyko 2021-08-09 17:37 ` Matthew Wilcox 2021-08-09 17:47 ` Pali Rohár 2021-08-09 20:43 ` Steve French 2021-08-09 18:00 ` Viacheslav Dubeyko 2021-08-08 16:24 ` [RFC PATCH 12/20] hfs: Do not use broken utf8 NLS table for iocharset=utf8 mount option Pali Rohár 2021-08-09 17:49 ` Viacheslav Dubeyko 2021-08-08 16:24 ` [RFC PATCH 13/20] hfsplus: " Pali Rohár 2021-08-09 17:42 ` Viacheslav Dubeyko 2021-08-08 16:24 ` [RFC PATCH 14/20] jfs: Remove custom iso8859-1 implementation Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 15/20] jfs: Fix buffer overflow in jfs_strfromUCS_le() function Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 16/20] jfs: Do not use broken utf8 NLS table for iocharset=utf8 mount option Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 17/20] ntfs: " Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 18/20] cifs: " Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 19/20] cifs: Remove usage of load_nls_default() calls Pali Rohár 2021-08-08 16:24 ` [RFC PATCH 20/20] nls: Drop broken nls_utf8 module Pali Rohár 2021-09-03 21:26 ` [RFC PATCH 00/20] fs: Remove usage of broken nls_utf8 and drop it Kari Argillander 2021-09-03 21:37 ` Pali Rohár 2021-09-03 22:06 ` Kari Argillander
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20210815094224.dswbjywnhvajvzjv@pali \ --to=pali@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=anton@tuxera.com \ --cc=hch@infradead.org \ --cc=hirofumi@mail.parknet.co.jp \ --cc=jack@suse.cz \ --cc=jfs-discussion@lists.sourceforge.net \ --cc=linux-cifs@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-ntfs-dev@lists.sourceforge.net \ --cc=luisbg@kernel.org \ --cc=marek.behun@nic.cz \ --cc=pavel@ucw.cz \ --cc=salah.triki@gmail.com \ --cc=shaggy@kernel.org \ --cc=tytso@mit.edu \ --cc=viro@zeniv.linux.org.uk \ --subject='Re: [RFC PATCH 01/20] fat: Fix iocharset=utf8 mount option' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).