linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] ext4: fix direct I/O read error
@ 2020-06-28  6:47 姜迎
  2020-06-29 15:58 ` Ira Weiny
  0 siblings, 1 reply; 4+ messages in thread
From: 姜迎 @ 2020-06-28  6:47 UTC (permalink / raw)
  To: tytso; +Cc: adilger.kernel, linux-ext4, linux-kernel

From: jiangying8582 <jiangying8582@126.com>
Date: Wed, 24 Jun 2020 19:02:34 +0800
Subject: [PATCH] ext4: fix direct I/O read error

This patch is used to fix ext4 direct I/O read error when
the read size is not alignment with block size. Compare the
size between read offset with file size, if read offset is
greater than file size, then return 0.

Then, I will use a test to explain the error.
(1) Make the file that is not alignment wiht block size:
        $dd if=/dev/zero of=./test.jar bs=1000 count=3

(2) I wrote a test script named "direct_io_read_file.c" s following:

        #include <stdio.h>
        #include <stdlib.h>
        #include <unistd.h>
        #include <sys/file.h>
        #include <sys/types.h>
        #include <sys/stat.h>
        #include <string.h>
        #define BUF_SIZE 1024

        int main()
        {
                int fd;
                int ret;

                unsigned char *buf;
                ret = posix_memalign((void **)&buf, 512, BUF_SIZE);
                if (ret) {
                        perror("posix_memalign failed");
                        exit(1);
                }
                fd = open("./test.jar", O_RDONLY | O_DIRECT, 0755);
                if (fd < 0){
                        perror("open ./test.jar failed");
                        exit(1);
                }

                do {
                        ret = read(fd, buf, BUF_SIZE);
                        printf("ret=%d\n",ret);
                        if (ret < 0) {
                                perror("write test.jar failed");
                        }

                } while (ret > 0);

                free(buf);
                close(fd);
        }

(3) Compiling the script:
        $gcc direct_io_read_file.c -D_GNU_SOURCE

(4) Exec the script:
        $./a.out

    The result is as following:
        ret=1024
        ret=1024
        ret=952
        ret=-1
        write rts-segmenter-0.3.7.2.jar failed: Invalid argument

I have tested this script on XFS filesystem, XFS does not have
this problem, because XFS use iomap_dio_rw() to do direct I/O
read. And the comparing between read offset and file size is done
is iomap_dio_rw(), the code is as following:
        if (pos < size) {
                retval = filemap_write_and_wait_range(mapping, pos,
                                        pos + iov_length(iov, nr_segs) - 1);
                if (!retval) {
                        retval = mapping->a_ops->direct_IO(READ, iocb,
                                                iov, pos, nr_segs);
                }
                ...
        }
Only when "pos < size", direct I/O can be done, or 0 will be return.

I have tested my fix patch, it is up to the mustard of EINVAL in
man2(read) as following:
        #include <unistd.h>
        ssize_t read(int fd, void *buf, size_t count);

        EINVAL
                fd is attached to an object which is unsuitable for reading;
                or the file was opened with the O_DIRECT flag, and either the
                address specified in buf, the value specified in count, or the
                current file offset is not suitably aligned.
So I think this patch can be applied to fix ext4 direct I/O problem.

Why this problem can happen? I think
commit <9fe55eea7e4b> ("Fix race when checking i_size on direct i/o read")
caused.

However Ext4 introduces direct I/O read using iomap infrastructure
on kernel 5.5, the patch is commit <b1b4705d54ab>
("ext4: introduce direct I/O read using iomap infrastructure"),
then Ext4 will be the same as XFS, they all use iomap_dio_rw() to do direct
I/O read. So this problem does not exist on kernel 5.5 for Ext4.

From above description, we can see this problem exists on all the kernel
versions between kernel 3.14 and kernel 5.4. Please apply this patch
on these kernel versions, or please use the method on kernel 5.5 to fix
this problem. Thanks.

Signed-off-by: jiangying8582 <jiangying8582@126.com>
---
 fs/ext4/inode.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 516faa2..d514ff5 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3821,6 +3821,12 @@ static ssize_t ext4_direct_IO_read(struct kiocb *iocb, struct iov_iter *iter)
        struct inode *inode = mapping->host;
        size_t count = iov_iter_count(iter);
        ssize_t ret;
+       loff_t offset = iocb->ki_pos;
+       loff_t size;
+
+       size = i_size_read(inode);
+       if (offset >= size)
+               return 0;

        /*
         * Shared inode_lock is enough for us - it protects against concurrent
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 4+ messages in thread
* Re: [PATCH] ext4: fix direct I/O read error
@ 2020-06-28 12:42 Markus Elfring
  0 siblings, 0 replies; 4+ messages in thread
From: Markus Elfring @ 2020-06-28 12:42 UTC (permalink / raw)
  To: jiangying8582, linux-ext4
  Cc: kernel-janitors, linux-kernel, Andreas Dilger, Theodore Ts'o

> Signed-off-by: jiangying8582 <jiangying8582@126.com>

Please use the real name for your contributions.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/process/submitting-patches.rst?id=719fdd32921fb7e3208db8832d32ae1c2d68900f#n431


Would you like to add the tag “Fixes” to the commit message?

Regards,
Markus

^ permalink raw reply	[flat|nested] 4+ messages in thread
* [PATCH] ext4: fix direct I/O read error
@ 2020-08-05  7:37 Jiang Ying
  0 siblings, 0 replies; 4+ messages in thread
From: Jiang Ying @ 2020-08-05  7:37 UTC (permalink / raw)
  To: tytso, adilger.kernel, linux-ext4, linux-kernel, stable
  Cc: wanglong19, heguanjun

This patch is used to fix ext4 direct I/O read error when
the read size is not aligned with block size.

Then, I will use a test to explain the error.

(1) Make a file that is not aligned with block size:
	$dd if=/dev/zero of=./test.jar bs=1000 count=3

(2) I wrote a source file named "direct_io_read_file.c" as following:

	#include <stdio.h>
	#include <stdlib.h>
	#include <unistd.h>
	#include <sys/file.h>
	#include <sys/types.h>
	#include <sys/stat.h>
	#include <string.h>
	#define BUF_SIZE 1024

	int main()
	{
		int fd;
		int ret;

		unsigned char *buf;
		ret = posix_memalign((void **)&buf, 512, BUF_SIZE);
		if (ret) {
			perror("posix_memalign failed");
			exit(1);
		}
		fd = open("./test.jar", O_RDONLY | O_DIRECT, 0755);
		if (fd < 0){
			perror("open ./test.jar failed");
			exit(1);
		}

		do {
			ret = read(fd, buf, BUF_SIZE);
			printf("ret=%d\n",ret);
			if (ret < 0) {
				perror("write test.jar failed");
			}
		} while (ret > 0);

		free(buf);
		close(fd);
	}

(3) Compile the source file:
	$gcc direct_io_read_file.c -D_GNU_SOURCE

(4) Run the test program:
	$./a.out

	The result is as following:
	ret=1024
	ret=1024
	ret=952
	ret=-1
	write test.jar failed: Invalid argument.

I have tested this program on XFS filesystem, XFS does not have
this problem, because XFS use iomap_dio_rw() to do direct I/O
read. And the comparing between read offset and file size is done
in iomap_dio_rw(), the code is as following:

	if (pos < size) {
		retval = filemap_write_and_wait_range(mapping, pos,
				pos + iov_length(iov, nr_segs) - 1);

		if (!retval) {
			retval = mapping->a_ops->direct_IO(READ, iocb,
						iov, pos, nr_segs);
		}
		...
	}

...only when "pos < size", direct I/O can be done, or 0 will be return.

I have tested the fix patch on Ext4, it is up to the mustard of
EINVAL in man2(read) as following:
	#include <unistd.h>
	ssize_t read(int fd, void *buf, size_t count);

	EINVAL
		fd is attached to an object which is unsuitable for reading;
		or the file was opened with the O_DIRECT flag, and either the
		address specified in buf, the value specified in count, or the
		current file offset is not suitably aligned.

So I think this patch can be applied to fix ext4 direct I/O error.

However Ext4 introduces direct I/O read using iomap infrastructure
on kernel 5.5, the patch is commit <b1b4705d54ab>
("ext4: introduce direct I/O read using iomap infrastructure"),
then Ext4 will be the same as XFS, they all use iomap_dio_rw() to do direct
I/O read. So this problem does not exist on kernel 5.5 for Ext4.

From above description, we can see this problem exists on all the kernel
versions between kernel 3.14 and kernel 5.4. It will cause the Applications
to fail to read. For example, when the search service downloads a new full
index file, the search engine is loading the previous index file and is
processing the search request, it can not use buffer io that may squeeze
the previous index file in use from pagecache, so the serch service must
use direct I/O read.

Please apply this patch on these kernel versions, or please use the method
on kernel 5.5 to fix this problem.

Fixes: 9fe55eea7e4b ("Fix race when checking i_size on direct i/o read")
Reviewed-by: Jan Kara <jack@suse.cz>
Co-developed-by: Wang Long <wanglong19@meituan.com>
Signed-off-by: Wang Long <wanglong19@meituan.com>
Signed-off-by: Jiang Ying <jiangying8582@126.com>
---
 fs/ext4/inode.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index 516faa2..a66b0ac 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -3821,6 +3821,11 @@ static ssize_t ext4_direct_IO_read(struct kiocb *iocb, struct iov_iter *iter)
 	struct inode *inode = mapping->host;
 	size_t count = iov_iter_count(iter);
 	ssize_t ret;
+	loff_t offset = iocb->ki_pos;
+	loff_t size = i_size_read(inode);
+
+	if (offset >= size)
+		return 0;
 
 	/*
 	 * Shared inode_lock is enough for us - it protects against concurrent
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-08-05  7:38 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-28  6:47 [PATCH] ext4: fix direct I/O read error 姜迎
2020-06-29 15:58 ` Ira Weiny
2020-06-28 12:42 Markus Elfring
2020-08-05  7:37 Jiang Ying

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).