[PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[Konstantin Komarov]

This adds fs/ntfs3 Kconfig, Makefile and Documentation file

Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
---
 Documentation/filesystems/ntfs3.rst | 93 +++++++++++++++++++++++++++++
 fs/ntfs3/Kconfig                    | 23 +++++++
 fs/ntfs3/Makefile                   | 11 ++++
 3 files changed, 127 insertions(+)
 create mode 100644 Documentation/filesystems/ntfs3.rst
 create mode 100644 fs/ntfs3/Kconfig
 create mode 100644 fs/ntfs3/Makefile

diff --git a/Documentation/filesystems/ntfs3.rst b/Documentation/filesystems/ntfs3.rst
new file mode 100644
index 000000000000..4a510a6cdaee
--- /dev/null
+++ b/Documentation/filesystems/ntfs3.rst
@@ -0,0 +1,93 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=====
+NTFS3
+=====
+
+
+Summary and Features
+====================
+
+NTFS3 is fully functional NTFS Read-Write driver. The driver works with
+NTFS versions up to 3.1, normal/compressed/sparse files
+and journal replaying. File system type to use on mount is 'ntfs3'.
+
+- This driver implements NTFS read/write support for normal, sparsed and
+  compressed files.
+  NOTE: Operations with compressed files require increased memory consumption;
+- Supports native journal replaying;
+- Supports extended attributes;
+- Supports NFS export of mounted NTFS volumes.
+
+Mount Options
+=============
+
+The list below describes mount options supported by NTFS3 driver in addtion to
+generic ones.
+
+===============================================================================
+
+nls=name		These options inform the driver how to interpret path
+			strings and translate them to Unicode and back. In case
+			none of these options are set, or if specified codepage
+			doesn't exist on the system, the default codepage will be
+			used (CONFIG_NLS_DEFAULT).
+			Examples:
+				'nls=utf8'
+
+uid=
+gid=
+umask=			Controls the default permissions for files/directories created
+			after the NTFS volume is mounted.
+
+fmask=
+dmask=			Instead of specifying umask which applies both to
+			files and directories, fmask applies only to files and
+			dmask only to directories.
+
+nohidden		Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
+			attribute will not be shown under Linux.
+
+sys_immutable		Files with the Windows-specific SYSTEM
+			(FILE_ATTRIBUTE_SYSTEM) attribute will be marked as system
+			immutable files.
+
+discard			Enable support of the TRIM command for improved performance
+			on delete operations, which is recommended for use with the
+			solid-state drives (SSD).
+
+force			Forces the driver to mount partitions even if 'dirty' flag
+			(volume dirty) is set. Not recommended for use.
+
+sparse			Create new files as "sparse".
+
+showmeta		Use this parameter to show all meta-files (System Files) on
+			a mounted NTFS partition.
+			By default, all meta-files are hidden.
+
+no_acs_rules		"No access rules" mount option sets access rights for
+			files/folders to 777 and owner/group to root. This mount
+			option absorbs all other permissions:
+			- permissions change for files/folders will be reported
+				as successful, but they will remain 777;
+			- owner/group change will be reported as successful, but
+				they will stay as root
+
+===============================================================================
+
+
+ToDo list
+=========
+
+- Full journaling support (currently journal replaying is supported) over JBD.
+
+
+References
+==========
+https://www.paragon-software.com/home/ntfs-linux-professional/
+	- Commercial version of the NTFS driver for Linux.
+
+almaz.alexandrovich@paragon-software.com
+	- Direct e-mail address for feedback and requests on the NTFS3 implementation.
+
+
diff --git a/fs/ntfs3/Kconfig b/fs/ntfs3/Kconfig
new file mode 100644
index 000000000000..92a9c68008c8
--- /dev/null
+++ b/fs/ntfs3/Kconfig
@@ -0,0 +1,23 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config NTFS3_FS
+	tristate "NTFS Read-Write file system support"
+	select NLS
+	help
+	  Windows OS native file system (NTFS) support up to NTFS version 3.1.
+
+	  Y or M enables the NTFS3 driver with full features enabled (read,
+	  write, journal replaying, sparse/compressed files support).
+	  File system type to use on mount is "ntfs3". Module name (M option)
+	  is also "ntfs3".
+
+	  Documentation: <file:Documentation/filesystems/ntfs3.rst>
+
+config NTFS3_64BIT_CLUSTER
+	bool "64 bits per NTFS clusters"
+	depends on NTFS3_FS && 64BIT
+	help
+	  Windows implementation of ntfs.sys uses 32 bits per clusters.
+	  If activated 64 bits per clusters you will be able to use 4k cluster
+	  for 16T+ volumes. Windows will not be able to mount such volumes.
+
+	  It is recommended to say N here.
diff --git a/fs/ntfs3/Makefile b/fs/ntfs3/Makefile
new file mode 100644
index 000000000000..4d4fe198b8b8
--- /dev/null
+++ b/fs/ntfs3/Makefile
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for the ntfs3 filesystem support.
+#
+
+obj-$(CONFIG_NTFS3_FS) += ntfs3.o
+
+ntfs3-objs := bitfunc.o bitmap.o inode.o fsntfs.o frecord.o \
+	    index.o attrlist.o record.o attrib.o run.o xattr.o\
+	    upcase.o super.o file.o dir.o namei.o lznt.o\
+	    fslog.o
-- 
2.25.2

Re: [PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[Randy Dunlap]

On 8/21/20 9:25 AM, Konstantin Komarov wrote:
> This adds fs/ntfs3 Kconfig, Makefile and Documentation file
> 
> Signed-off-by: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
> ---
>  Documentation/filesystems/ntfs3.rst | 93 +++++++++++++++++++++++++++++
>  fs/ntfs3/Kconfig                    | 23 +++++++
>  fs/ntfs3/Makefile                   | 11 ++++
>  3 files changed, 127 insertions(+)
>  create mode 100644 Documentation/filesystems/ntfs3.rst
>  create mode 100644 fs/ntfs3/Kconfig
>  create mode 100644 fs/ntfs3/Makefile
> 
> diff --git a/Documentation/filesystems/ntfs3.rst b/Documentation/filesystems/ntfs3.rst
> new file mode 100644
> index 000000000000..4a510a6cdaee
> --- /dev/null
> +++ b/Documentation/filesystems/ntfs3.rst
> @@ -0,0 +1,93 @@
> +.. SPDX-License-Identifier: GPL-2.0
> +
> +=====
> +NTFS3
> +=====
> +
> +
> +Summary and Features
> +====================
> +
> +NTFS3 is fully functional NTFS Read-Write driver. The driver works with
> +NTFS versions up to 3.1, normal/compressed/sparse files
> +and journal replaying. File system type to use on mount is 'ntfs3'.
> +
> +- This driver implements NTFS read/write support for normal, sparsed and

                                                                sparse

> +  compressed files.
> +  NOTE: Operations with compressed files require increased memory consumption;
> +- Supports native journal replaying;
> +- Supports extended attributes;
> +- Supports NFS export of mounted NTFS volumes.
> +
> +Mount Options
> +=============
> +
> +The list below describes mount options supported by NTFS3 driver in addtion to

                                                                       addition

> +generic ones.
> +
> +===============================================================================
> +
> +nls=name		These options inform the driver how to interpret path
> +			strings and translate them to Unicode and back. In case
> +			none of these options are set, or if specified codepage
> +			doesn't exist on the system, the default codepage will be
> +			used (CONFIG_NLS_DEFAULT).
> +			Examples:
> +				'nls=utf8'
> +
> +uid=
> +gid=
> +umask=			Controls the default permissions for files/directories created
> +			after the NTFS volume is mounted.
> +
> +fmask=
> +dmask=			Instead of specifying umask which applies both to
> +			files and directories, fmask applies only to files and
> +			dmask only to directories.
> +
> +nohidden		Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
> +			attribute will not be shown under Linux.

Without this mount option, will HIDDEN files be shown by default?

> +
> +sys_immutable		Files with the Windows-specific SYSTEM
> +			(FILE_ATTRIBUTE_SYSTEM) attribute will be marked as system
> +			immutable files.
> +
> +discard			Enable support of the TRIM command for improved performance
> +			on delete operations, which is recommended for use with the
> +			solid-state drives (SSD).
> +
> +force			Forces the driver to mount partitions even if 'dirty' flag
> +			(volume dirty) is set. Not recommended for use.
> +
> +sparse			Create new files as "sparse".
> +
> +showmeta		Use this parameter to show all meta-files (System Files) on
> +			a mounted NTFS partition.
> +			By default, all meta-files are hidden.
> +
> +no_acs_rules		"No access rules" mount option sets access rights for
> +			files/folders to 777 and owner/group to root. This mount
> +			option absorbs all other permissions:
> +			- permissions change for files/folders will be reported
> +				as successful, but they will remain 777;
> +			- owner/group change will be reported as successful, but
> +				they will stay as root
> +
> +===============================================================================
> +
> +
> +ToDo list
> +=========
> +
> +- Full journaling support (currently journal replaying is supported) over JBD.

          journalling
seems to be preferred.

> +
> +
> +References
> +==========
> +https://www.paragon-software.com/home/ntfs-linux-professional/
> +	- Commercial version of the NTFS driver for Linux.
> +
> +almaz.alexandrovich@paragon-software.com
> +	- Direct e-mail address for feedback and requests on the NTFS3 implementation.
> +
> +

thanks.
-- 
~Randy

Re: [PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[Pali Rohár]

On Friday 21 August 2020 16:25:37 Konstantin Komarov wrote:
> +Mount Options
> +=============
> +
> +The list below describes mount options supported by NTFS3 driver in addtion to
> +generic ones.
> +
> +===============================================================================
> +
> +nls=name		These options inform the driver how to interpret path
> +			strings and translate them to Unicode and back. In case
> +			none of these options are set, or if specified codepage
> +			doesn't exist on the system, the default codepage will be
> +			used (CONFIG_NLS_DEFAULT).
> +			Examples:
> +				'nls=utf8'
> +
> +uid=
> +gid=

IIRC ntfs filesystem had concept of storing unix owner/group. Was it
dropped? Or it is incompatible with current Windows implementation? I'm
just curious if we cannot use ntfs-native unix permissions instead of
forcing them from mount options. Maybe as improvement for future.

Normally owner/group on ntfs is stored in that windows SID format.
ntfs-3g fuse driver has some mount option where you can specify mapping
table between SID and unix to make permissions compatible with existing
windows installations.

Such functionality could be a nice feature once somebody would have time
to implement it in future...

> +umask=			Controls the default permissions for files/directories created
> +			after the NTFS volume is mounted.
> +
> +fmask=
> +dmask=			Instead of specifying umask which applies both to
> +			files and directories, fmask applies only to files and
> +			dmask only to directories.
> +
> +nohidden		Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
> +			attribute will not be shown under Linux.

What other people think? It is useful mount option which would disallow
access to hidden files? Hidden attribute is normal attribute which even
normal user without admin rights on Windows can set on its own files.

Also concept of hidden files is already present for fat filesystems and
we do not have such mount option nor for msdosfs, vfat nor for exfat.

Konstantin, what is purpose of this mount option? I would like to know
what usecases have this option.

> +sys_immutable		Files with the Windows-specific SYSTEM
> +			(FILE_ATTRIBUTE_SYSTEM) attribute will be marked as system
> +			immutable files.
> +
> +discard			Enable support of the TRIM command for improved performance
> +			on delete operations, which is recommended for use with the
> +			solid-state drives (SSD).
> +
> +force			Forces the driver to mount partitions even if 'dirty' flag
> +			(volume dirty) is set. Not recommended for use.
> +
> +sparse			Create new files as "sparse".
> +
> +showmeta		Use this parameter to show all meta-files (System Files) on
> +			a mounted NTFS partition.
> +			By default, all meta-files are hidden.
> +
> +no_acs_rules		"No access rules" mount option sets access rights for
> +			files/folders to 777 and owner/group to root. This mount
> +			option absorbs all other permissions:
> +			- permissions change for files/folders will be reported
> +				as successful, but they will remain 777;
> +			- owner/group change will be reported as successful, but
> +				they will stay as root

What about rather adding "mode=" and "dmode=" mount option which would
specify permissions for all files and directories? Other filesystems
have support for "mode=" mount option and I think it is better if
filesystems have some "common" options and not each filesystem its own
mount option for similar features.

> diff --git a/fs/ntfs3/Kconfig b/fs/ntfs3/Kconfig
> new file mode 100644
> index 000000000000..92a9c68008c8
> --- /dev/null
> +++ b/fs/ntfs3/Kconfig
> @@ -0,0 +1,23 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +config NTFS3_FS
> +	tristate "NTFS Read-Write file system support"
> +	select NLS
> +	help
> +	  Windows OS native file system (NTFS) support up to NTFS version 3.1.
> +
> +	  Y or M enables the NTFS3 driver with full features enabled (read,
> +	  write, journal replaying, sparse/compressed files support).
> +	  File system type to use on mount is "ntfs3". Module name (M option)
> +	  is also "ntfs3".
> +
> +	  Documentation: <file:Documentation/filesystems/ntfs3.rst>
> +
> +config NTFS3_64BIT_CLUSTER
> +	bool "64 bits per NTFS clusters"
> +	depends on NTFS3_FS && 64BIT
> +	help
> +	  Windows implementation of ntfs.sys uses 32 bits per clusters.
> +	  If activated 64 bits per clusters you will be able to use 4k cluster
> +	  for 16T+ volumes. Windows will not be able to mount such volumes.

Would it be possible to change this compile time option into mount
option?

Because I do not see any benefit in compile time option which makes
kernel's ntfs driver "fully" incompatible with Windows implementation.

For me it looks like that mount option for such functionality is more
suitable.

RE: [PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[Konstantin Komarov]

From: Randy Dunlap <rdunlap@infradead.org>
Sent: Friday, August 21, 2020 8:23 PM
> 
> On 8/21/20 9:25 AM, Konstantin Komarov wrote:
> > This adds fs/ntfs3 Kconfig, Makefile and Documentation file
[]
> > +
> > +- This driver implements NTFS read/write support for normal, sparsed and
> 
>                                                                 sparse
> 
> > +  compressed files.
> > +  NOTE: Operations with compressed files require increased memory consumption;
> > +- Supports native journal replaying;
> > +- Supports extended attributes;
> > +- Supports NFS export of mounted NTFS volumes.
> > +
> > +Mount Options
> > +=============
> > +
> > +The list below describes mount options supported by NTFS3 driver in addtion to
> 
>                                                                        addition
> 
> > +generic ones.

Thanks! Will be fixed in v3.

> > +===============================================================================
[]
> > +
> > +nohidden		Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
> > +			attribute will not be shown under Linux.
> 
> Without this mount option, will HIDDEN files be shown by default?

Yes, correct, without "nohidden" mount option files with FILE_ATTRIBUTE_HIDDEN set will be shown as regular files.

> > +
> > +sys_immutable		Files with the Windows-specific SYSTEM
[]
> > +
> > +- Full journaling support (currently journal replaying is supported) over JBD.
> 
>           journalling
> seems to be preferred.
> 

Have to disagree on this. According to "journaling" term usage in different sources, the single-L seems to be the standard.

> > +
> > +
> > +References
> > +==========
> > +https://www.paragon-software.com/home/ntfs-linux-professional/
> > +	- Commercial version of the NTFS driver for Linux.
> > +
> > +almaz.alexandrovich@paragon-software.com
> > +	- Direct e-mail address for feedback and requests on the NTFS3 implementation.
> > +
> > +
> 
> thanks.
> --
> ~Randy

Re: [PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[Joe Perches]

On Thu, 2020-08-27 at 16:01 +0000, Konstantin Komarov wrote:
> From: Randy Dunlap <rdunlap@infradead.org>
> Sent: Friday, August 21, 2020 8:23 PM
[]
> > > +- Full journaling support (currently journal replaying is supported) over JBD.
> > 
> >           journalling
> > seems to be preferred.
> > 
> Have to disagree on this. According to "journaling" term usage in
> different sources, the single-L seems to be the standard.

In the kernel it seems to be a tie:

$ git grep -i -w journalling | wc -l
109
$ git grep -i -w journaling | wc -l
111

I think 1 l better.

RE: [PATCH v2 08/10] fs/ntfs3: Add Kconfig, Makefile and doc

[Konstantin Komarov]

From: Pali Rohár <pali@kernel.org>
Sent: Sunday, August 23, 2020 1:17 PM
> 
> On Friday 21 August 2020 16:25:37 Konstantin Komarov wrote:
> > +Mount Options
> > +=============
> > +
> > +The list below describes mount options supported by NTFS3 driver in addtion to
> > +generic ones.
> > +
> > +===============================================================================
> > +
> > +nls=name		These options inform the driver how to interpret path
> > +			strings and translate them to Unicode and back. In case
> > +			none of these options are set, or if specified codepage
> > +			doesn't exist on the system, the default codepage will be
> > +			used (CONFIG_NLS_DEFAULT).
> > +			Examples:
> > +				'nls=utf8'
> > +
> > +uid=
> > +gid=
> 
> IIRC ntfs filesystem had concept of storing unix owner/group. Was it
> dropped? Or it is incompatible with current Windows implementation? I'm
> just curious if we cannot use ntfs-native unix permissions instead of
> forcing them from mount options. Maybe as improvement for future.
> 
> Normally owner/group on ntfs is stored in that windows SID format.
> ntfs-3g fuse driver has some mount option where you can specify mapping
> table between SID and unix to make permissions compatible with existing
> windows installations.
> 
> Such functionality could be a nice feature once somebody would have time
> to implement it in future...
> 

If you mean the way on how WLS implements the unix ownership and permissions (using NTFS extended attributes to store values), then it seems to be quite handy way of operation. Will be done in future versions.

> > +umask=			Controls the default permissions for files/directories created
> > +			after the NTFS volume is mounted.
> > +
> > +fmask=
> > +dmask=			Instead of specifying umask which applies both to
> > +			files and directories, fmask applies only to files and
> > +			dmask only to directories.
> > +
> > +nohidden		Files with the Windows-specific HIDDEN (FILE_ATTRIBUTE_HIDDEN)
> > +			attribute will not be shown under Linux.
> 
> What other people think? It is useful mount option which would disallow
> access to hidden files? Hidden attribute is normal attribute which even
> normal user without admin rights on Windows can set on its own files.
> 
> Also concept of hidden files is already present for fat filesystems and
> we do not have such mount option nor for msdosfs, vfat nor for exfat.
> 
> Konstantin, what is purpose of this mount option? I would like to know
> what usecases have this option.
> 

It is indeed discussional mount option. The purpose of it is to protect users from modifying/deleting Windows system files, which may affect bootability of the OS. Unlikely the case for fat32/exfat nowadays, but quite actual for ntfs (dual-boot win/lin configurations).

> > +sys_immutable		Files with the Windows-specific SYSTEM
> > +			(FILE_ATTRIBUTE_SYSTEM) attribute will be marked as system
> > +			immutable files.
> > +
> > +discard			Enable support of the TRIM command for improved performance
> > +			on delete operations, which is recommended for use with the
> > +			solid-state drives (SSD).
> > +
> > +force			Forces the driver to mount partitions even if 'dirty' flag
> > +			(volume dirty) is set. Not recommended for use.
> > +
> > +sparse			Create new files as "sparse".
> > +
> > +showmeta		Use this parameter to show all meta-files (System Files) on
> > +			a mounted NTFS partition.
> > +			By default, all meta-files are hidden.
> > +
> > +no_acs_rules		"No access rules" mount option sets access rights for
> > +			files/folders to 777 and owner/group to root. This mount
> > +			option absorbs all other permissions:
> > +			- permissions change for files/folders will be reported
> > +				as successful, but they will remain 777;
> > +			- owner/group change will be reported as successful, but
> > +				they will stay as root
> 
> What about rather adding "mode=" and "dmode=" mount option which would
> specify permissions for all files and directories? Other filesystems
> have support for "mode=" mount option and I think it is better if
> filesystems have some "common" options and not each filesystem its own
> mount option for similar features.
> 

According to what we can see: some file systems provide umask/dmask mount options, others provide mode/dmode. It's hard for us to decide which pair should be presented (and providing both may be a bit confusing for usage). But as for implementation - it could be done easily, if needed.

> > diff --git a/fs/ntfs3/Kconfig b/fs/ntfs3/Kconfig
> > new file mode 100644
> > index 000000000000..92a9c68008c8
> > --- /dev/null
> > +++ b/fs/ntfs3/Kconfig
> > @@ -0,0 +1,23 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +config NTFS3_FS
> > +	tristate "NTFS Read-Write file system support"
> > +	select NLS
> > +	help
> > +	  Windows OS native file system (NTFS) support up to NTFS version 3.1.
> > +
> > +	  Y or M enables the NTFS3 driver with full features enabled (read,
> > +	  write, journal replaying, sparse/compressed files support).
> > +	  File system type to use on mount is "ntfs3". Module name (M option)
> > +	  is also "ntfs3".
> > +
> > +	  Documentation: <file:Documentation/filesystems/ntfs3.rst>
> > +
> > +config NTFS3_64BIT_CLUSTER
> > +	bool "64 bits per NTFS clusters"
> > +	depends on NTFS3_FS && 64BIT
> > +	help
> > +	  Windows implementation of ntfs.sys uses 32 bits per clusters.
> > +	  If activated 64 bits per clusters you will be able to use 4k cluster
> > +	  for 16T+ volumes. Windows will not be able to mount such volumes.
> 
> Would it be possible to change this compile time option into mount
> option?
> 
> Because I do not see any benefit in compile time option which makes
> kernel's ntfs driver "fully" incompatible with Windows implementation.
> 
> For me it looks like that mount option for such functionality is more
> suitable.

It would be possible, but I can't find any pros for this. Overall, having the "switch" which will turn this off/on in the runtime won't give any benefits. 
The support for 64bit-sized number of cluster won't break compatibility with Windows, it may just extend ntfs3 capabilities over the windows. Windows in general does not support volumes with more than 2^32 clusters for NTFS. NTFS3 will mount such volumes, Windows won't. But everthing mountable in Windows, will be mountable by ntfs3 and, after this, by Windows again, nevertheless if this option is enabled or not.

Thanks.