linux-nfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Re: [PATCH v5 00/51] Add Flexfile Layout Module
@ 2015-01-24 14:37 Peng Tao
  2015-01-24 18:49 ` Christoph Hellwig
  0 siblings, 1 reply; 5+ messages in thread
From: Peng Tao @ 2015-01-24 14:37 UTC (permalink / raw)
  To: Christoph Hellwig; +Cc: Trond Myklebust, Linux NFS Mailing list, Tom Haynes

oops, sorry for editing the mail subject... It was supposed to reply
to your email "Re: [PATCH v5 00/51] Add Flexfile Layout Module"

On Sat, Jan 24, 2015 at 10:33 PM, Peng Tao <tao.peng@primarydata.com> wrote:
> Does following patch fix the warning and hang for you?
> I tested it locally on top of your flexfiles+pnfsd branch (tip 4f7961eb6) and xfstest generic/001 passed.
>
> Cheers,
> Tao
>
> From 47bc1a0ffc586ea96c99e453efd7e2435454b486 Mon Sep 17 00:00:00 2001
> From: Peng Tao <tao.peng@primarydata.com>
> Date: Sat, 24 Jan 2015 22:14:52 +0800
> Subject: [PATCH] nfs41: .init_read and .init_write can be called with valid pg_lseg
>
> With pgio refactoring in v3.15, .init_read and .init_write can be
> called with valid pgio->pg_lseg. file layout was fixed at that time
> by commit c6194271f (pnfs: filelayout: support non page aligned
> layouts). But the generic helper still needs to be fixed.
>
> Cc: stable@vger.kernel.org # 3.15+
> Signed-off-by: Peng Tao <tao.peng@primarydata.com>
> ---
>  fs/nfs/pnfs.c | 41 ++++++++++++++++++++---------------------
>  1 file changed, 20 insertions(+), 21 deletions(-)
>
> diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
> index 0fb0f19..c7be9b9 100644
> --- a/fs/nfs/pnfs.c
> +++ b/fs/nfs/pnfs.c
> @@ -1711,19 +1711,19 @@ pnfs_generic_pg_init_read(struct nfs_pageio_descriptor *pgio, struct nfs_page *r
>  {
>         u64 rd_size = req->wb_bytes;
>
> -       WARN_ON_ONCE(pgio->pg_lseg != NULL);
> -
> -       if (pgio->pg_dreq == NULL)
> -               rd_size = i_size_read(pgio->pg_inode) - req_offset(req);
> -       else
> -               rd_size = nfs_dreq_bytes_left(pgio->pg_dreq);
> -
> -       pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> -                                          req->wb_context,
> -                                          req_offset(req),
> -                                          rd_size,
> -                                          IOMODE_READ,
> -                                          GFP_KERNEL);
> +       if (pgio->pg_lseg == NULL) {
> +               if (pgio->pg_dreq == NULL)
> +                       rd_size = i_size_read(pgio->pg_inode) - req_offset(req);
> +               else
> +                       rd_size = nfs_dreq_bytes_left(pgio->pg_dreq);
> +
> +               pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> +                                                  req->wb_context,
> +                                                  req_offset(req),
> +                                                  rd_size,
> +                                                  IOMODE_READ,
> +                                                  GFP_KERNEL);
> +       }
>         /* If no lseg, fall back to read through mds */
>         if (pgio->pg_lseg == NULL)
>                 nfs_pageio_reset_read_mds(pgio);
> @@ -1735,14 +1735,13 @@ void
>  pnfs_generic_pg_init_write(struct nfs_pageio_descriptor *pgio,
>                            struct nfs_page *req, u64 wb_size)
>  {
> -       WARN_ON_ONCE(pgio->pg_lseg != NULL);
> -
> -       pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> -                                          req->wb_context,
> -                                          req_offset(req),
> -                                          wb_size,
> -                                          IOMODE_RW,
> -                                          GFP_NOFS);
> +       if (pgio->pg_lseg == NULL)
> +               pgio->pg_lseg = pnfs_update_layout(pgio->pg_inode,
> +                                                  req->wb_context,
> +                                                  req_offset(req),
> +                                                  wb_size,
> +                                                  IOMODE_RW,
> +                                                  GFP_NOFS);
>         /* If no lseg, fall back to write through mds */
>         if (pgio->pg_lseg == NULL)
>                 nfs_pageio_reset_write_mds(pgio);
> --
> 1.9.1
>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v5 00/51] Add Flexfile Layout Module
  2015-01-24 14:37 [PATCH v5 00/51] Add Flexfile Layout Module Peng Tao
@ 2015-01-24 18:49 ` Christoph Hellwig
  0 siblings, 0 replies; 5+ messages in thread
From: Christoph Hellwig @ 2015-01-24 18:49 UTC (permalink / raw)
  To: Peng Tao; +Cc: Trond Myklebust, Linux NFS Mailing list, Tom Haynes

On Sat, Jan 24, 2015 at 10:37:09PM +0800, Peng Tao wrote:
> oops, sorry for editing the mail subject... It was supposed to reply
> to your email "Re: [PATCH v5 00/51] Add Flexfile Layout Module"

With your patch it surives a little longer, but then this happens:
(you could just try my receipt for the reproducer, it really is that
simple..):

generic/013 26s ...[  192.023340] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [fsstress:9440]
[  192.024091] Modules linked in:
[  192.024489] irq event stamp: 93792
[  192.024845] hardirqs last  enabled at (93791): [<ffffffff81dfd9f7>] restore_args+0x0/0x30
[  192.025701] hardirqs last disabled at (93792): [<ffffffff81dfdcdd>] apic_timer_interrupt+0x6d/0x80
[  192.026606] softirqs last  enabled at (93790): [<ffffffff810be593>] __do_softirq+0x1e3/0x2d0
[  192.026651] softirqs last disabled at (93785): [<ffffffff810be96e>] irq_exit+0x9e/0xc0
[  192.026651] CPU: 0 PID: 9440 Comm: fsstress Not tainted 3.19.0-rc5+ #1555
[  192.026651] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[  192.026651] task: ffff88007a4e82d0 ti: ffff88007d7fc000 task.ti: ffff88007d7fc000
[  192.026651] RIP: 0010:[<ffffffff8178af6c>]  [<ffffffff8178af6c>] __radix_tree_lookup+0x5c/0xb0
[  192.026651] RSP: 0018:ffff88007d7ffa58  EFLAGS: 00000293
[  192.026651] RAX: 0000000000000002 RBX: ffffffff81dfd9f7 RCX: 0000000000000000
[  192.026651] RDX: 0000000000000000 RSI: 0000000000000028 RDI: 0000000000000002
[  192.026651] RBP: ffff88007d7ffa58 R08: 000000000000002b R09: 0000000000000000
[  192.026651] R10: ffff880078426d48 R11: 0000000000000000 R12: ffff88007d7ff9c8
[  192.026651] R13: ffff88007a4e82d0 R14: ffff88007d7fc000 R15: ffff88007a4e82d0
[  192.026651] FS:  00007ff90c7b1700(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
[  192.026651] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  192.026651] CR2: 00007ff904020000 CR3: 000000007a12a000 CR4: 00000000000006f0
[  192.026651] Stack:
[  192.026651]  ffff88007d7ffa68 ffffffff8178afcd ffff88007d7ffa98 ffffffff81178d6a
[  192.026651]  ffff88007adf4140 ffff88007d7ffd48 ffff88007adf4140 ffff88007879b518
[  192.026651]  ffff88007d7ffaf8 ffffffff8139c8b6 ffffffff8139c878 0000000000000000
[  192.026651] Call Trace:
[  192.026651]  [<ffffffff8178afcd>] radix_tree_lookup+0xd/0x10
[  192.026651]  [<ffffffff81178d6a>] page_cache_next_hole+0x3a/0x50
[  192.026651]  [<ffffffff8139c8b6>] bl_pg_init_write+0xf6/0x140
[  192.026651]  [<ffffffff8139c878>] ? bl_pg_init_write+0xb8/0x140
[  192.026651]  [<ffffffff813559e0>] __nfs_pageio_add_request+0xf0/0x470
[  192.026651]  [<ffffffff81356293>] nfs_pageio_add_request+0x93/0x1c0
[  192.026651]  [<ffffffff811788ff>] ? find_get_pages_tag+0x1f/0x190
[  192.026651]  [<ffffffff81359ed0>] nfs_do_writepage+0x120/0x1e0
[  192.026651]  [<ffffffff8135a048>] nfs_writepages_callback+0x18/0x30
[  192.026651]  [<ffffffff81182616>] write_cache_pages+0x1e6/0x490
[  192.026651]  [<ffffffff8135a030>] ? nfs_writepage_locked+0xa0/0xa0
[  192.026651]  [<ffffffff8135a124>] nfs_writepages+0x94/0x130
[  192.026651]  [<ffffffff8118428c>] do_writepages+0x1c/0x40
[  192.026651]  [<ffffffff8117a281>] __filemap_fdatawrite_range+0x51/0x60
[  192.026651]  [<ffffffff8117a365>] filemap_write_and_wait_range+0x35/0x80
[  192.026651]  [<ffffffff813843df>] nfs4_file_fsync+0x5f/0xb0
[  192.026651]  [<ffffffff81384670>] ? nfs4_file_llseek+0x60/0x60
[  192.026651]  [<ffffffff811f7074>] vfs_fsync+0x24/0x40
[  192.026651]  [<ffffffff8134bd02>] nfs_file_flush+0x82/0xc0
[  192.026651]  [<ffffffff811c3f35>] filp_close+0x35/0x80
[  192.026651]  [<ffffffff811e462a>] __close_fd+0x8a/0xb0
[  192.026651]  [<ffffffff811c3ece>] SyS_close+0x1e/0x50
[  192.026651]  [<ffffffff81dfcd69>] system_call_fastpath+0x12/0x17
[  192.026651] Code: 07 48 83 c7 08 49 89 3b 5d c3 0f 1f 44 00 00 49 89 c2 49 83 e2 fe 41 8b 3a 81 e7 ff 0f 00 00 89 f8 48 3b 34 c5 80 b3 58 82 77 4c <8d> 04 7f 8d 4c 00 fa eb 06 0f 1f 00 49 89 c2 49 89 f0 49 d3 e8 

and never recovers from it, printing that backtrace again and again.


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v5 00/51] Add Flexfile Layout Module
  2015-01-23  8:07 ` Christoph Hellwig
@ 2015-01-23  8:27   ` Christoph Hellwig
  0 siblings, 0 replies; 5+ messages in thread
From: Christoph Hellwig @ 2015-01-23  8:27 UTC (permalink / raw)
  To: Tom Haynes; +Cc: Trond Myklebust, Linux NFS Mailing list

On Fri, Jan 23, 2015 at 12:07:32AM -0800, Christoph Hellwig wrote:
> This series breaks xfstests with the first test when used together
> with a local pnfs block export.

And here is the backtrace before it hangs, sorry:

generic/001 14s ...[  116.095264] pNFS: using block device vdb
[  116.478161] ------------[ cut here ]------------
[  116.478773] WARNING: CPU: 3 PID: 4483 at fs/nfs/pnfs.c:1738 pnfs_generic_pg_init_write+0x8f/0xa0()
[  116.479622] Modules linked in:
[  116.480029] CPU: 3 PID: 4483 Comm: fill Not tainted 3.19.0-rc5+ #1549
[  116.480886] Hardware name: Bochs Bochs, BIOS Bochs 01/01/2007
[  116.481764]  ffffffff822ec4aa ffff88007a9c3918 ffffffff81df2678 ffff88007c3e5150
[  116.483234]  0000000000000000 ffff88007a9c3958 ffffffff810ba9f2 0000000000000000
[  116.484435]  ffff88007a9c3c48 ffff88007a4d3e40 ffff88007b5e3870 0000000000000040
[  116.485337] Call Trace:
[  116.485617]  [<ffffffff81df2678>] dump_stack+0x45/0x57
[  116.486118]  [<ffffffff810ba9f2>] warn_slowpath_common+0x92/0xd0
[  116.486820]  [<ffffffff810baa45>] warn_slowpath_null+0x15/0x20
[  116.487490]  [<ffffffff81393cdf>] pnfs_generic_pg_init_write+0x8f/0xa0
[  116.488223]  [<ffffffff8139c8a2>] bl_pg_init_write+0x92/0x140
[  116.488871]  [<ffffffff813559e0>] __nfs_pageio_add_request+0xf0/0x470
[  116.489586]  [<ffffffff81356293>] nfs_pageio_add_request+0x93/0x1c0
[  116.490319]  [<ffffffff811788ff>] ? find_get_pages_tag+0x1f/0x190
[  116.491001]  [<ffffffff81359ed0>] nfs_do_writepage+0x120/0x1e0
[  116.491660]  [<ffffffff8135a048>] nfs_writepages_callback+0x18/0x30
[  116.492362]  [<ffffffff81182616>] write_cache_pages+0x1e6/0x490
[  116.493029]  [<ffffffff8135a030>] ? nfs_writepage_locked+0xa0/0xa0
[  116.493746]  [<ffffffff8135a124>] nfs_writepages+0x94/0x130
[  116.494379]  [<ffffffff8118428c>] do_writepages+0x1c/0x40
[  116.495003]  [<ffffffff8117a281>] __filemap_fdatawrite_range+0x51/0x60
[  116.495726]  [<ffffffff8117a365>] filemap_write_and_wait_range+0x35/0x80
[  116.496468]  [<ffffffff813843df>] nfs4_file_fsync+0x5f/0xb0
[  116.497323]  [<ffffffff81384670>] ? nfs4_file_llseek+0x60/0x60
[  116.498375]  [<ffffffff811f7074>] vfs_fsync+0x24/0x40
[  116.499329]  [<ffffffff8134bd02>] nfs_file_flush+0x82/0xc0
[  116.500418]  [<ffffffff811c3f35>] filp_close+0x35/0x80
[  116.501204]  [<ffffffff811e4294>] put_files_struct+0x94/0xe0
[  116.502007]  [<ffffffff811e439d>] exit_files+0x4d/0x60
[  116.502804]  [<ffffffff810bce16>] do_exit+0x2b6/0xa60
[  116.503491]  [<ffffffff811c8ae8>] ? __sb_end_write+0x78/0x80
[  116.504154]  [<ffffffff81dfda1c>] ? retint_swapgs+0x13/0x1b
[  116.504799]  [<ffffffff810bd697>] do_group_exit+0x47/0xc0
[  116.505425]  [<ffffffff810bd722>] SyS_exit_group+0x12/0x20
[  116.506058]  [<ffffffff81dfcda9>] system_call_fastpath+0x12/0x17
[  116.506782] ---[ end trace f766241646eab9dc ]---


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v5 00/51] Add Flexfile Layout Module
  2015-01-22 23:35 Tom Haynes
@ 2015-01-23  8:07 ` Christoph Hellwig
  2015-01-23  8:27   ` Christoph Hellwig
  0 siblings, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2015-01-23  8:07 UTC (permalink / raw)
  To: Tom Haynes; +Cc: Trond Myklebust, Linux NFS Mailing list

This series breaks xfstests with the first test when used together
with a local pnfs block export.

Steps to reproduce:

 (1) start a KVM VM with a second virtio-blk disk.
 (2) add /mnt/test/ to /etc/exports
 (2) configure xfstests with TEST_DIR=/mnt/nfs1 and
     TEST_DEV=127.0.0.1:/mnt/test

then run the following commands:

mkfs.xfs -f /dev/vdb
mount /dev/vdb /mnt/test

/etc/init.d/nfs-kernel-server restart

blkmapd
mount -t nfs 127.0.0.1:/mnt/test/ /mnt/nfs1 -o vers=4.1

cd xfstests
./check -nfs -g auto

I've also pushed out the trivially merged flexfiles+pnfsd branch
to git://git.infradead.org/users/hch/pnfs.git that also includes
a hack to force 4.1 and a revert for the commit that causes nfs to
deadlock for me all the time.

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v5 00/51] Add Flexfile Layout Module
@ 2015-01-22 23:35 Tom Haynes
  2015-01-23  8:07 ` Christoph Hellwig
  0 siblings, 1 reply; 5+ messages in thread
From: Tom Haynes @ 2015-01-22 23:35 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: Linux NFS Mailing list

This patchset introduces the Flexfile Layout Module for the
client.

It will correspond to draft 5
(http://tools.ietf.org/id/draft-ietf-nfsv4-flex-files-05.txt)
of the Parallel NFS (pNFS) Flexible File Layout
(https://datatracker.ietf.org/doc/draft-ietf-nfsv4-flex-files/).

Note: The major change from version 4 was the XDR change
from a cred to a stringified user and group principal
in the layout.

This version has been rebased to v3.19-rc5 and is ready
for inclusion in 3.20.

The [flexfiles] branch of

git://git.linux-nfs.org/projects/loghyr/linux-nfs.git

also has this code.

Peng Tao (36):
  nfs41: pull data server cache from file layout to generic pnfs
  nfs41: pull decode_ds_addr from file layout to generic pnfs
  nfs41: pull nfs4_ds_connect from file layout to generic pnfs
  nfs41: allow LD to choose DS connection auth flavor
  nfs41: move file layout macros to generic pnfs
  nfsv3: introduce nfs3_set_ds_client
  nfs41: allow LD to choose DS connection version/minor_version
  nfs41: create NFSv3 DS connection if specified
  nfs: allow different protocol in nfs_initiate_commit
  nfs4: pass slot table to nfs40_setup_sequence
  nfs4: export nfs4_sequence_done
  nfs: allow to specify cred in nfs_initiate_pgio
  nfs: set hostname when creating nfsv3 ds connection
  nfs/flexclient: export pnfs_layoutcommit_inode
  nfs41: close a small race window when adding new layout to global list
  nfs41: serialize first layoutget of a file
  nfs: save server READ/WRITE/COMMIT status
  nfs41: pass iomode through layoutreturn args
  nfs41: make a helper function to send layoutreturn
  nfs41: add a helper to mark layout for return
  nfs41: don't use a layout if it is marked for returning
  nfs41: send layoutreturn in last put_lseg
  nfs41: clear NFS_LAYOUT_RETURN if layoutreturn is sent or failed to
    send
  nfs/filelayout: use pnfs_error_mark_layout_for_return
  nfs41: add a debug warning if we destroy an unempty layout
  nfs: only reset desc->pg_mirror_idx when mirroring is supported
  nfs: add nfs_pgio_current_mirror helper
  pnfs: allow LD to ask to resend read through pnfs
  nfs41: add range to layoutreturn args
  nfs41: allow async version layoutreturn
  nfs41: introduce NFS_LAYOUT_RETURN_BEFORE_CLOSE
  nfs/flexfiles: send layoutreturn before freeing lseg
  nfs41: add NFS_LAYOUT_RETRY_LAYOUTGET to layout header flags
  nfs: add a helper to set NFS_ODIRECT_RESCHED_WRITES to direct writes
  nfs41: wait for LAYOUTRETURN before retrying LAYOUTGET
  nfs: count DIO good bytes correctly with mirroring

Tom Haynes (5):
  pnfs: Prepare for flexfiles by pulling out common code
  pnfs: Do not grab the commit_info lock twice when rescheduling writes
  pnfs: Add nfs_rpc_ops in calls to nfs_initiate_pgio
  pnfs/flexfiles: Add the FlexFile Layout Driver
  pnfs: Update documentation on the Layout Drivers

Trond Myklebust (1):
  NFSv4.1/NFSv3: Add pNFS callbacks for nfs3_(read|write|commit)_done()

Weston Andros Adamson (9):
  sunrpc: add rpc_count_iostats_idx
  nfs: introduce pg_cleanup op for pgio descriptors
  pnfs: release lseg in pnfs_generic_pg_cleanup
  nfs: handle overlapping reqs in lock_and_join
  nfs: rename pgio header ds_idx to ds_commit_idx
  pnfs: pass ds_commit_idx through the commit path
  nfs: add mirroring support to pgio layer
  nfs: mirroring support for direct io
  pnfs: fail comparison when bucket verifier not set

 Documentation/filesystems/nfs/pnfs.txt    |   13 +-
 fs/nfs/Kconfig                            |    5 +
 fs/nfs/Makefile                           |    3 +-
 fs/nfs/blocklayout/blocklayout.c          |    2 +
 fs/nfs/direct.c                           |  112 +-
 fs/nfs/filelayout/filelayout.c            |  315 +-----
 fs/nfs/filelayout/filelayout.h            |   40 -
 fs/nfs/filelayout/filelayoutdev.c         |  469 +--------
 fs/nfs/flexfilelayout/Makefile            |    5 +
 fs/nfs/flexfilelayout/flexfilelayout.c    | 1571 +++++++++++++++++++++++++++++
 fs/nfs/flexfilelayout/flexfilelayout.h    |  155 +++
 fs/nfs/flexfilelayout/flexfilelayoutdev.c |  552 ++++++++++
 fs/nfs/idmap.c                            |    3 +-
 fs/nfs/internal.h                         |   31 +-
 fs/nfs/nfs2xdr.c                          |   10 +-
 fs/nfs/nfs3_fs.h                          |    2 +
 fs/nfs/nfs3client.c                       |   41 +
 fs/nfs/nfs3proc.c                         |    9 +
 fs/nfs/nfs3super.c                        |    2 +-
 fs/nfs/nfs3xdr.c                          |    3 +
 fs/nfs/nfs4_fs.h                          |    6 +
 fs/nfs/nfs4client.c                       |    7 +-
 fs/nfs/nfs4proc.c                         |   48 +-
 fs/nfs/nfs4xdr.c                          |    9 +-
 fs/nfs/objlayout/objio_osd.c              |    5 +-
 fs/nfs/pagelist.c                         |  294 +++++-
 fs/nfs/pnfs.c                             |  419 ++++++--
 fs/nfs/pnfs.h                             |  120 ++-
 fs/nfs/pnfs_nfs.c                         |  808 +++++++++++++++
 fs/nfs/read.c                             |   33 +-
 fs/nfs/write.c                            |   52 +-
 include/linux/nfs4.h                      |    1 +
 include/linux/nfs_fs_sb.h                 |    9 +-
 include/linux/nfs_idmap.h                 |    2 +
 include/linux/nfs_page.h                  |   22 +-
 include/linux/nfs_xdr.h                   |    6 +-
 include/linux/sunrpc/metrics.h            |    2 +
 net/sunrpc/stats.c                        |   26 +-
 38 files changed, 4194 insertions(+), 1018 deletions(-)
 create mode 100644 fs/nfs/flexfilelayout/Makefile
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.c
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayout.h
 create mode 100644 fs/nfs/flexfilelayout/flexfilelayoutdev.c
 create mode 100644 fs/nfs/pnfs_nfs.c

-- 
1.9.3


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2015-01-24 18:49 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-01-24 14:37 [PATCH v5 00/51] Add Flexfile Layout Module Peng Tao
2015-01-24 18:49 ` Christoph Hellwig
  -- strict thread matches above, loose matches on Subject: below --
2015-01-22 23:35 Tom Haynes
2015-01-23  8:07 ` Christoph Hellwig
2015-01-23  8:27   ` Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).