* Re: [PATCH v5 00/51] Add Flexfile Layout Module
@ 2015-01-24 14:37 Peng Tao
2015-01-24 18:49 ` Christoph Hellwig
0 siblings, 1 reply; 5+ messages in thread
From: Peng Tao @ 2015-01-24 14:37 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Trond Myklebust, Linux NFS Mailing list, Tom Haynes
oops, sorry for editing the mail subject... It was supposed to reply
to your email "Re: [PATCH v5 00/51] Add Flexfile Layout Module"
On Sat, Jan 24, 2015 at 10:33 PM, Peng Tao <tao.peng@primarydata.com> wrote:
> Does following patch fix the warning and hang for you?
> I tested it locally on top of your flexfiles+pnfsd branch (tip 4f7961eb6) and xfstest generic/001 passed.
>
> Cheers,
> Tao
>
> From 47bc1a0ffc586ea96c99e453efd7e2435454b486 Mon Sep 17 00:00:00 2001
> From: Peng Tao <tao.peng@primarydata.com>
> Date: Sat, 24 Jan 2015 22:14:52 +0800
> Subject: [PATCH] nfs41: .init_read and .init_write can be called with valid pg_lseg
>
> With pgio refactoring in v3.15, .init_read and .init_write can be
> called with valid pgio->pg_lseg. file layout was fixed at that time
> by commit c6194271f (pnfs: filelayout: support non page aligned
> layouts). But the generic helper still needs to be fixed.
>
> Cc: stable@vger.kernel.org # 3.15+
> Signed-off-by: Peng Tao <tao.peng@primarydata.com>
> ---
> fs/nfs/pnfs.c | 41 ++++++++++++++++++++---------------------
> 1 file changed, 20 insertions(+), 21 deletions(-)
>
> diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
> index 0fb0f19..c7be9b9 100644
> --- a/fs/nfs/pnfs.c
> +++ b/fs/nfs/pnfs.c
> @@ -1711,19 +1711,19 @@ pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *r
> {
> u64 rd_size = req->wb_bytes;
>
> - WARN_ON_ONCE(pgio->pg_lseg != NULL);
> -
> - if (pgio->pg_dreq == NULL)
> - rd_size = i_size_read(pgio->pg_inode) - req_offset(req);
> - else
> - rd_size = nfs_dreq_bytes_left(pgio->pg_dreq);
> -
> - pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> - req->wb_context,
> - req_offset(req),
> - rd_size,
> - IOMODE_READ,
> - GFP_KERNEL);
> + if (pgio->pg_lseg == NULL) {
> + if (pgio->pg_dreq == NULL)
> + rd_size = i_size_read(pgio->pg_inode) - req_offset(req);
> + else
> + rd_size = nfs_dreq_bytes_left(pgio->pg_dreq);
> +
> + pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> + req->wb_context,
> + req_offset(req),
> + rd_size,
> + IOMODE_READ,
> + GFP_KERNEL);
> + }
> /* If no lseg, fall back to read through mds */
> if (pgio->pg_lseg == NULL)
> nfs_pageio_reset_read_mds(pgio);
> @@ -1735,14 +1735,13 @@ void
> pnfs_generic_pg_init_write(struct nfs_pageio_descriptor *pgio,
> struct nfs_page *req, u64 wb_size)
> {
> - WARN_ON_ONCE(pgio->pg_lseg != NULL);
> -
> - pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> - req->wb_context,
> - req_offset(req),
> - wb_size,
> - IOMODE_RW,
> - GFP_NOFS);
> + if (pgio->pg_lseg == NULL)
> + pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> + req->wb_context,
> + req_offset(req),
> + wb_size,
> + IOMODE_RW,
> + GFP_NOFS);
> /* If no lseg, fall back to write through mds */
> if (pgio->pg_lseg == NULL)
> nfs_pageio_reset_write_mds(pgio);
> --
> 1.9.1
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v5 00/51] Add Flexfile Layout Module
2015-01-24 14:37 [PATCH v5 00/51] Add Flexfile Layout Module Peng Tao
@ 2015-01-24 18:49 ` Christoph Hellwig
0 siblings, 0 replies; 5+ messages in thread
From: Christoph Hellwig @ 2015-01-24 18:49 UTC (permalink / raw)
To: Peng Tao; +Cc: Trond Myklebust, Linux NFS Mailing list, Tom Haynes
On Sat, Jan 24, 2015 at 10:37:09PM +0800, Peng Tao wrote:
> oops, sorry for editing the mail subject... It was supposed to reply
> to your email "Re: [PATCH v5 00/51] Add Flexfile Layout Module"
With your patch it surives a little longer, but then this happens:
(you could just try my receipt for the reproducer, it really is that
simple..):
generic/013 26s ...[ 192.023340] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [fsstress:9440]
[ 192.024091] Modules linked in:
[ 192.024489] irq event stamp: 93792
[ 192.024845] hardirqs last enabled at (93791): [<ffffffff81dfd9f7>] restore_args+0x0/0x30
[ 192.025701] hardirqs last disabled at (93792): [<ffffffff81dfdcdd>] apic_timer_interrupt+0x6d/0x80
[ 192.026606] softirqs last enabled at (93790): [<ffffffff810be593>] __do_softirq+0x1e3/0x2d0
[ 192.026651] softirqs last disabled at (93785): [<ffffffff810be96e>] irq_exit+0x9e/0xc0
[ 192.026651] CPU: 0 PID: 9440 Comm: fsstress Not tainted 3.19.0-rc5+ #1555
[ 192.026651] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[ 192.026651] task: ffff88007a4e82d0 ti: ffff88007d7fc000 task.ti: ffff88007d7fc000
[ 192.026651] RIP: 0010:[<ffffffff8178af6c>] [<ffffffff8178af6c>] __radix_tree_lookup+0x5c/0xb0
[ 192.026651] RSP: 0018:ffff88007d7ffa58 EFLAGS: 00000293
[ 192.026651] RAX: 0000000000000002 RBX: ffffffff81dfd9f7 RCX: 0000000000000000
[ 192.026651] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000002
[ 192.026651] RBP: ffff88007d7ffa58 R08: 000000000000002b R09: 0000000000000000
[ 192.026651] R10: ffff880078426d48 R11: 0000000000000000 R12: ffff88007d7ff9c8
[ 192.026651] R13: ffff88007a4e82d0 R14: ffff88007d7fc000 R15: ffff88007a4e82d0
[ 192.026651] FS: 00007ff90c7b1700(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
[ 192.026651] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 192.026651] CR2: 00007ff904020000 CR3: 000000007a12a000 CR4: 00000000000006f0
[ 192.026651] Stack:
[ 192.026651] ffff88007d7ffa68 ffffffff8178afcd ffff88007d7ffa98 ffffffff81178d6a
[ 192.026651] ffff88007adf4140 ffff88007d7ffd48 ffff88007adf4140 ffff88007879b518
[ 192.026651] ffff88007d7ffaf8 ffffffff8139c8b6 ffffffff8139c878 0000000000000000
[ 192.026651] Call Trace:
[ 192.026651] [<ffffffff8178afcd>] radix_tree_lookup+0xd/0x10
[ 192.026651] [<ffffffff81178d6a>] page_cache_next_hole+0x3a/0x50
[ 192.026651] [<ffffffff8139c8b6>] bl_pg_init_write+0xf6/0x140
[ 192.026651] [<ffffffff8139c878>] ? bl_pg_init_write+0xb8/0x140
[ 192.026651] [<ffffffff813559e0>] __nfs_pageio_add_request+0xf0/0x470
[ 192.026651] [<ffffffff81356293>] nfs_pageio_add_request+0x93/0x1c0
[ 192.026651] [<ffffffff811788ff>] ? find_get_pages_tag+0x1f/0x190
[ 192.026651] [<ffffffff81359ed0>] nfs_do_writepage+0x120/0x1e0
[ 192.026651] [<ffffffff8135a048>] nfs_writepages_callback+0x18/0x30
[ 192.026651] [<ffffffff81182616>] write_cache_pages+0x1e6/0x490
[ 192.026651] [<ffffffff8135a030>] ? nfs_writepage_locked+0xa0/0xa0
[ 192.026651] [<ffffffff8135a124>] nfs_writepages+0x94/0x130
[ 192.026651] [<ffffffff8118428c>] do_writepages+0x1c/0x40
[ 192.026651] [<ffffffff8117a281>] __filemap_fdatawrite_range+0x51/0x60
[ 192.026651] [<ffffffff8117a365>] filemap_write_and_wait_range+0x35/0x80
[ 192.026651] [<ffffffff813843df>] nfs4_file_fsync+0x5f/0xb0
[ 192.026651] [<ffffffff81384670>] ? nfs4_file_llseek+0x60/0x60
[ 192.026651] [<ffffffff811f7074>] vfs_fsync+0x24/0x40
[ 192.026651] [<ffffffff8134bd02>] nfs_file_flush+0x82/0xc0
[ 192.026651] [<ffffffff811c3f35>] filp_close+0x35/0x80
[ 192.026651] [<ffffffff811e462a>] __close_fd+0x8a/0xb0
[ 192.026651] [<ffffffff811c3ece>] SyS_close+0x1e/0x50
[ 192.026651] [<ffffffff81dfcd69>] system_call_fastpath+0x12/0x17
[ 192.026651] Code: 07 48 83 c7 08 49 89 3b 5d c3 0f 1f 44 00 00 49 89 c2 49 83 e2 fe 41 8b 3a 81 e7 ff 0f 00 00 89 f8 48 3b 34 c5 80 b3 58 82 77 4c <8d> 04 7f 8d 4c 00 fa eb 06 0f 1f 00 49 89 c2 49 89 f0 49 d3 e8
and never recovers from it, printing that backtrace again and again.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v5 00/51] Add Flexfile Layout Module
2015-01-23 8:07 ` Christoph Hellwig
@ 2015-01-23 8:27 ` Christoph Hellwig
0 siblings, 0 replies; 5+ messages in thread
From: Christoph Hellwig @ 2015-01-23 8:27 UTC (permalink / raw)
To: Tom Haynes; +Cc: Trond Myklebust, Linux NFS Mailing list
On Fri, Jan 23, 2015 at 12:07:32AM -0800, Christoph Hellwig wrote:
> This series breaks xfstests with the first test when used together
> with a local pnfs block export.
And here is the backtrace before it hangs, sorry:
generic/001 14s ...[ 116.095264] pNFS: using block device vdb
[ 116.478161] ------------[ cut here ]------------
[ 116.478773] WARNING: CPU: 3 PID: 4483 at fs/nfs/pnfs.c:1738 pnfs_generic_pg_init_write+0x8f/0xa0()
[ 116.479622] Modules linked in:
[ 116.480029] CPU: 3 PID: 4483 Comm: fill Not tainted 3.19.0-rc5+ #1549
[ 116.480886] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[ 116.481764] ffffffff822ec4aa ffff88007a9c3918 ffffffff81df2678 ffff88007c3e5150
[ 116.483234] 0000000000000000 ffff88007a9c3958 ffffffff810ba9f2 0000000000000000
[ 116.484435] ffff88007a9c3c48 ffff88007a4d3e40 ffff88007b5e3870 0000000000000040
[ 116.485337] Call Trace:
[ 116.485617] [<ffffffff81df2678>] dump_stack+0x45/0x57
[ 116.486118] [<ffffffff810ba9f2>] warn_slowpath_common+0x92/0xd0
[ 116.486820] [<ffffffff810baa45>] warn_slowpath_null+0x15/0x20
[ 116.487490] [<ffffffff81393cdf>] pnfs_generic_pg_init_write+0x8f/0xa0
[ 116.488223] [<ffffffff8139c8a2>] bl_pg_init_write+0x92/0x140
[ 116.488871] [<ffffffff813559e0>] __nfs_pageio_add_request+0xf0/0x470
[ 116.489586] [<ffffffff81356293>] nfs_pageio_add_request+0x93/0x1c0
[ 116.490319] [<ffffffff811788ff>] ? find_get_pages_tag+0x1f/0x190
[ 116.491001] [<ffffffff81359ed0>] nfs_do_writepage+0x120/0x1e0
[ 116.491660] [<ffffffff8135a048>] nfs_writepages_callback+0x18/0x30
[ 116.492362] [<ffffffff81182616>] write_cache_pages+0x1e6/0x490
[ 116.493029] [<ffffffff8135a030>] ? nfs_writepage_locked+0xa0/0xa0
[ 116.493746] [<ffffffff8135a124>] nfs_writepages+0x94/0x130
[ 116.494379] [<ffffffff8118428c>] do_writepages+0x1c/0x40
[ 116.495003] [<ffffffff8117a281>] __filemap_fdatawrite_range+0x51/0x60
[ 116.495726] [<ffffffff8117a365>] filemap_write_and_wait_range+0x35/0x80
[ 116.496468] [<ffffffff813843df>] nfs4_file_fsync+0x5f/0xb0
[ 116.497323] [<ffffffff81384670>] ? nfs4_file_llseek+0x60/0x60
[ 116.498375] [<ffffffff811f7074>] vfs_fsync+0x24/0x40
[ 116.499329] [<ffffffff8134bd02>] nfs_file_flush+0x82/0xc0
[ 116.500418] [<ffffffff811c3f35>] filp_close+0x35/0x80
[ 116.501204] [<ffffffff811e4294>] put_files_struct+0x94/0xe0
[ 116.502007] [<ffffffff811e439d>] exit_files+0x4d/0x60
[ 116.502804] [<ffffffff810bce16>] do_exit+0x2b6/0xa60
[ 116.503491] [<ffffffff811c8ae8>] ? __sb_end_write+0x78/0x80
[ 116.504154] [<ffffffff81dfda1c>] ? retint_swapgs+0x13/0x1b
[ 116.504799] [<ffffffff810bd697>] do_group_exit+0x47/0xc0
[ 116.505425] [<ffffffff810bd722>] SyS_exit_group+0x12/0x20
[ 116.506058] [<ffffffff81dfcda9>] system_call_fastpath+0x12/0x17
[ 116.506782] ---[ end trace f766241646eab9dc ]---
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v5 00/51] Add Flexfile Layout Module
2015-01-22 23:35 Tom Haynes
@ 2015-01-23 8:07 ` Christoph Hellwig
2015-01-23 8:27 ` Christoph Hellwig
0 siblings, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2015-01-23 8:07 UTC (permalink / raw)
To: Tom Haynes; +Cc: Trond Myklebust, Linux NFS Mailing list
This series breaks xfstests with the first test when used together
with a local pnfs block export.
Steps to reproduce:
(1) start a KVM VM with a second virtio-blk disk.
(2) add /mnt/test/ to /etc/exports
(2) configure xfstests with TEST_DIR=/mnt/nfs1 and
TEST_DEV=127.0.0.1:/mnt/test
then run the following commands:
mkfs.xfs -f /dev/vdb
mount /dev/vdb /mnt/test
/etc/init.d/nfs-kernel-server restart
blkmapd
mount -t nfs 127.0.0.1:/mnt/test/ /mnt/nfs1 -o vers=4.1
cd xfstests
./check -nfs -g auto
I've also pushed out the trivially merged flexfiles+pnfsd branch
to git://git.infradead.org/users/hch/pnfs.git that also includes
a hack to force 4.1 and a revert for the commit that causes nfs to
deadlock for me all the time.
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v5 00/51] Add Flexfile Layout Module
@ 2015-01-22 23:35 Tom Haynes
2015-01-23 8:07 ` Christoph Hellwig
0 siblings, 1 reply; 5+ messages in thread
From: Tom Haynes @ 2015-01-22 23:35 UTC (permalink / raw)
To: Trond Myklebust; +Cc: Linux NFS Mailing list
This patchset introduces the Flexfile Layout Module for the
client.
It will correspond to draft 5
(http://tools.ietf.org/id/draft-ietf-nfsv4-flex-files-05.txt)
of the Parallel NFS (pNFS) Flexible File Layout
(https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/).
Note: The major change from version 4 was the XDR change
from a cred to a stringified user and group principal
in the layout.
This version has been rebased to v3.19-rc5 and is ready
for inclusion in 3.20.
The [flexfiles] branch of
git://git.linux-nfs.org/projects/loghyr/linux-nfs.git
also has this code.
Peng Tao (36):
nfs41: pull data server cache from file layout to generic pnfs
nfs41: pull decode_ds_addr from file layout to generic pnfs
nfs41: pull nfs4_ds_connect from file layout to generic pnfs
nfs41: allow LD to choose DS connection auth flavor
nfs41: move file layout macros to generic pnfs
nfsv3: introduce nfs3_set_ds_client
nfs41: allow LD to choose DS connection version/minor_version
nfs41: create NFSv3 DS connection if specified
nfs: allow different protocol in nfs_initiate_commit
nfs4: pass slot table to nfs40_setup_sequence
nfs4: export nfs4_sequence_done
nfs: allow to specify cred in nfs_initiate_pgio
nfs: set hostname when creating nfsv3 ds connection
nfs/flexclient: export pnfs_layoutcommit_inode
nfs41: close a small race window when adding new layout to global list
nfs41: serialize first layoutget of a file
nfs: save server READ/WRITE/COMMIT status
nfs41: pass iomode through layoutreturn args
nfs41: make a helper function to send layoutreturn
nfs41: add a helper to mark layout for return
nfs41: don't use a layout if it is marked for returning
nfs41: send layoutreturn in last put_lseg
nfs41: clear NFS_LAYOUT_RETURN if layoutreturn is sent or failed to
send
nfs/filelayout: use pnfs_error_mark_layout_for_return
nfs41: add a debug warning if we destroy an unempty layout
nfs: only reset desc->pg_mirror_idx when mirroring is supported
nfs: add nfs_pgio_current_mirror helper
pnfs: allow LD to ask to resend read through pnfs
nfs41: add range to layoutreturn args
nfs41: allow async version layoutreturn
nfs41: introduce NFS_LAYOUT_RETURN_BEFORE_CLOSE
nfs/flexfiles: send layoutreturn before freeing lseg
nfs41: add NFS_LAYOUT_RETRY_LAYOUTGET to layout header flags
nfs: add a helper to set NFS_ODIRECT_RESCHED_WRITES to direct writes
nfs41: wait for LAYOUTRETURN before retrying LAYOUTGET
nfs: count DIO good bytes correctly with mirroring
Tom Haynes (5):
pnfs: Prepare for flexfiles by pulling out common code
pnfs: Do not grab the commit_info lock twice when rescheduling writes
pnfs: Add nfs_rpc_ops in calls to nfs_initiate_pgio
pnfs/flexfiles: Add the FlexFile Layout Driver
pnfs: Update documentation on the Layout Drivers
Trond Myklebust (1):
NFSv4.1/NFSv3: Add pNFS callbacks for nfs3_(read|write|commit)_done()
Weston Andros Adamson (9):
sunrpc: add rpc_count_iostats_idx
nfs: introduce pg_cleanup op for pgio descriptors
pnfs: release lseg in pnfs_generic_pg_cleanup
nfs: handle overlapping reqs in lock_and_join
nfs: rename pgio header ds_idx to ds_commit_idx
pnfs: pass ds_commit_idx through the commit path
nfs: add mirroring support to pgio layer
nfs: mirroring support for direct io
pnfs: fail comparison when bucket verifier not set
Documentation/filesystems/nfs/pnfs.txt | 13 +-
fs/nfs/Kconfig | 5 +
fs/nfs/Makefile | 3 +-
fs/nfs/blocklayout/blocklayout.c | 2 +
fs/nfs/direct.c | 112 +-
fs/nfs/filelayout/filelayout.c | 315 +-----
fs/nfs/filelayout/filelayout.h | 40 -
fs/nfs/filelayout/filelayoutdev.c | 469 +--------
fs/nfs/flexfilelayout/Makefile | 5 +
fs/nfs/flexfilelayout/flexfilelayout.c | 1571 +++++++++++++++++++++++++++++
fs/nfs/flexfilelayout/flexfilelayout.h | 155 +++
fs/nfs/flexfilelayout/flexfilelayoutdev.c | 552 ++++++++++
fs/nfs/idmap.c | 3 +-
fs/nfs/internal.h | 31 +-
fs/nfs/nfs2xdr.c | 10 +-
fs/nfs/nfs3_fs.h | 2 +
fs/nfs/nfs3client.c | 41 +
fs/nfs/nfs3proc.c | 9 +
fs/nfs/nfs3super.c | 2 +-
fs/nfs/nfs3xdr.c | 3 +
fs/nfs/nfs4_fs.h | 6 +
fs/nfs/nfs4client.c | 7 +-
fs/nfs/nfs4proc.c | 48 +-
fs/nfs/nfs4xdr.c | 9 +-
fs/nfs/objlayout/objio_osd.c | 5 +-
fs/nfs/pagelist.c | 294 +++++-
fs/nfs/pnfs.c | 419 ++++++--
fs/nfs/pnfs.h | 120 ++-
fs/nfs/pnfs_nfs.c | 808 +++++++++++++++
fs/nfs/read.c | 33 +-
fs/nfs/write.c | 52 +-
include/linux/nfs4.h | 1 +
include/linux/nfs_fs_sb.h | 9 +-
include/linux/nfs_idmap.h | 2 +
include/linux/nfs_page.h | 22 +-
include/linux/nfs_xdr.h | 6 +-
include/linux/sunrpc/metrics.h | 2 +
net/sunrpc/stats.c | 26 +-
38 files changed, 4194 insertions(+), 1018 deletions(-)
create mode 100644 fs/nfs/flexfilelayout/Makefile
create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.c
create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.h
create mode 100644 fs/nfs/flexfilelayout/flexfilelayoutdev.c
create mode 100644 fs/nfs/pnfs_nfs.c
--
1.9.3
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2015-01-24 18:49 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-01-24 14:37 [PATCH v5 00/51] Add Flexfile Layout Module Peng Tao
2015-01-24 18:49 ` Christoph Hellwig
-- strict thread matches above, loose matches on Subject: below --
2015-01-22 23:35 Tom Haynes
2015-01-23 8:07 ` Christoph Hellwig
2015-01-23 8:27 ` Christoph Hellwig
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).