From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD049C2D0C2 for ; Fri, 3 Jan 2020 16:47:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AAF8C2072C for ; Fri, 3 Jan 2020 16:47:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727974AbgACQrM (ORCPT ); Fri, 3 Jan 2020 11:47:12 -0500 Received: from fieldses.org ([173.255.197.46]:50554 "EHLO fieldses.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727912AbgACQrM (ORCPT ); Fri, 3 Jan 2020 11:47:12 -0500 Received: by fieldses.org (Postfix, from userid 2815) id B7D8D1CB4; Fri, 3 Jan 2020 11:47:11 -0500 (EST) Date: Fri, 3 Jan 2020 11:47:11 -0500 From: Bruce Fields To: Chuck Lever Cc: Trond Myklebust , Jeff Layton , Linux NFS Mailing List Subject: Re: CPU lockup in or near new filecache code Message-ID: <20200103164711.GB24306@fieldses.org> References: <9977648B-7D14-42EB-BD4A-CBD041A0C21A@oracle.com> <3af633a4016a183a930a44e3287f9da230711629.camel@hammerspace.com> <980CB8E4-0E7F-4F1D-B223-81176BE15A39@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <980CB8E4-0E7F-4F1D-B223-81176BE15A39@oracle.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org On Wed, Dec 18, 2019 at 06:20:56PM -0500, Chuck Lever wrote: > > On Dec 13, 2019, at 3:12 PM, Trond Myklebust wrote: > > Does something like the following help? > > > > 8<--------------------------------------------------- > > From caf515c82ed572e4f92ac8293e5da4818da0c6ce Mon Sep 17 00:00:00 2001 > > From: Trond Myklebust > > Date: Fri, 13 Dec 2019 15:07:33 -0500 > > Subject: [PATCH] nfsd: Fix a soft lockup race in > > nfsd_file_mark_find_or_create() > > > > If nfsd_file_mark_find_or_create() keeps winning the race for the > > nfsd_file_fsnotify_group->mark_mutex against nfsd_file_mark_put() > > then it can soft lock up, since fsnotify_add_inode_mark() ends > > up always finding an existing entry. > > > > Signed-off-by: Trond Myklebust > > --- > > fs/nfsd/filecache.c | 8 ++++++-- > > 1 file changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/fs/nfsd/filecache.c b/fs/nfsd/filecache.c > > index 9c2b29e07975..f275c11c4e28 100644 > > --- a/fs/nfsd/filecache.c > > +++ b/fs/nfsd/filecache.c > > @@ -132,9 +132,13 @@ nfsd_file_mark_find_or_create(struct nfsd_file *nf) > > struct nfsd_file_mark, > > nfm_mark)); > > mutex_unlock(&nfsd_file_fsnotify_group->mark_mutex); > > - fsnotify_put_mark(mark); > > - if (likely(nfm)) > > + if (nfm) { > > + fsnotify_put_mark(mark); > > break; > > + } > > + /* Avoid soft lockup race with nfsd_file_mark_put() */ > > + fsnotify_destroy_mark(mark, nfsd_file_fsnotify_group); > > + fsnotify_put_mark(mark); > > } else > > mutex_unlock(&nfsd_file_fsnotify_group->mark_mutex); > > > > I've tried to reproduce the lockup for three days with this patch > applied to my server. No lockup. > > Tested-by: Chuck Lever I'm applying this for 5.5 with Chuck's tested-by and: Fixes: 65294c1f2c5e "nfsd: add a new struct file caching facility to nfsd" --b.