From: Tao Ma <tm@tao.ma>
To: Andrew Morton <akpm@google.com>
Cc: linux-kernel@vger.kernel.org, "Theodore Ts'o" <tytso@mit.edu>,
Christoph Hellwig <hch@lst.de>, Al Viro <viro@zeniv.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH] fs/direct-io.c: Calcuate fs_count correctly in get_more_blocks.
Date: Fri, 23 Sep 2011 12:49:58 +0800 [thread overview]
Message-ID: <4E7C0FF6.3090502@tao.ma> (raw)
In-Reply-To: <20110919153149.be4ab3ca.akpm@google.com>
[-- Attachment #1: Type: text/plain, Size: 1193 bytes --]
Hi Andrew,
On 09/20/2011 06:31 AM, Andrew Morton wrote:
> On Mon, 19 Sep 2011 16:25:39 +0800
> Tao Ma <tm@tao.ma> wrote:
>
>> In get_more_blocks, we use dio_count to calcuate fs_count and do some
>> tricky things to increase fs_count if dio_count isn't aligned. But
>> actually it still has some cornor case that can't be coverd. See the
>> following example:
>> ./dio_write foo -s 1024 -w 4096(direct write 4096 bytes at offset 1024).
>> The same goes if the offset isn't aligned to fs_blocksize.
>>
>> In this case, the old calculation counts fs_count to be 1, but actually
>> we will write into 2 different blocks(if fs_blocksize=4096). The old code
>> just works, since it will call get_block twice(and may have to allocate
>> and create extent twice for file systems like ext4). So we'd better call
>> get_block just once with the proper fs_count.
>
> Has this been carefully tested with more than just ext4? If so, which?
I have done some more tests on both raw devices and file systems of
ext4, btrfs and xfs.
I have attached the fio test cases and ffsb test cases I used.
Besides this, I also run ffsb -I -s 2G against all these 3 file systems.
By now, no kernel error.
Thanks
Tao
[-- Attachment #2: rndrw-bs-16k-filesize-64G --]
[-- Type: text/plain, Size: 176 bytes --]
[global]
direct=1
ioengine=psync
bs=16k
filename=/mnt/ext4/testfile
size=64G
runtime=600
group_reporting
loops=50
[read]
rw=randread
numjobs=8
[write]
rw=randwrite
numjobs=8
[-- Attachment #3: dio_profile --]
[-- Type: text/plain, Size: 2299 bytes --]
directio = 1
time = 1800
[filesystem]
location = FFSB_DIR
num_dirs = 100
size_weight 4k 33
size_weight 8k 21
size_weight 16k 13
size_weight 32k 10
size_weight 64k 8
size_weight 128k 5
size_weight 256k 4
size_weight 512k 3
size_weight 8m 2
size_weight 32m 1
size_weight 1g 1
# min_filesize = 4k
# max_filesize = 10m
num_files = 1000
init_size = 100m
# init_size = 6GB
# init_size = 1gb
# init_util = 0.002
[end]
[threadgroup0]
num_threads = 16
append_weight = 1
append_fsync_weight = 1
stat_weight = 1
write_weight = 1
write_fsync_weight = 1
read_weight = 1
create_weight = 1
create_fsync_weight = 1
delete_weight = 1
readall_weight = 1
writeall_weight = 1
writeall_fsync_weight = 1
open_close_weight = 1
read_random = 1
write_random = 1
write_size = 4k
write_blocksize = 4k
read_size = 4k
read_blocksize = 4k
op_delay = 0
[stats]
enable_stats = 1
enable_range = 0
# ignore = close
# ignore = open
# ignore = lseek
# ignore = write
# ignore = read
msec_range 0.00 0.01
msec_range 0.01 0.02
msec_range 0.02 0.03
msec_range 0.03 0.04
msec_range 0.04 0.05
msec_range 0.05 0.1
msec_range 0.1 0.2
msec_range 0.2 0.5
msec_range 0.5 1.0
msec_range 1.0 2.0
msec_range 2.0 3.0
msec_range 3.0 4.0
msec_range 4.0 5.0
msec_range 5.0 10.0
msec_range 10.0 10000.0
[end]
[end]
[threadgroup1]
num_threads = 16
append_weight = 1
append_fsync_weight = 1
stat_weight = 1
write_weight = 1
write_fsync_weight = 1
read_weight = 1
create_weight = 1
create_fsync_weight = 1
delete_weight = 1
readall_weight = 1
writeall_weight = 1
writeall_fsync_weight = 1
open_close_weight = 1
read_random = 0
write_random = 0
write_size = 4k
write_blocksize = 4k
read_size = 4k
read_blocksize = 4k
op_delay = 0
[stats]
enable_stats = 1
enable_range = 0
# ignore = close
# ignore = open
# ignore = lseek
# ignore = write
# ignore = read
msec_range 0.00 0.01
msec_range 0.01 0.02
msec_range 0.02 0.03
msec_range 0.03 0.04
msec_range 0.04 0.05
msec_range 0.05 0.1
msec_range 0.1 0.2
msec_range 0.2 0.5
msec_range 0.5 1.0
msec_range 1.0 2.0
msec_range 2.0 3.0
msec_range 3.0 4.0
msec_range 4.0 5.0
msec_range 5.0 10.0
msec_range 10.0 10000.0
[end]
[end]
prev parent reply other threads:[~2011-09-23 4:50 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-19 8:25 [PATCH] fs/direct-io.c: Calcuate fs_count correctly in get_more_blocks Tao Ma
2011-09-19 22:31 ` Andrew Morton
2011-09-20 2:13 ` Tao Ma
2011-09-23 4:49 ` Tao Ma [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4E7C0FF6.3090502@tao.ma \
--to=tm@tao.ma \
--cc=akpm@google.com \
--cc=akpm@linux-foundation.org \
--cc=hch@lst.de \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.