Re: [PATCH v2] dmaengine: idxd: avoid deadlock in process_misc_interrupts()

From: Jerry Snitselaar <jsnitsel@redhat.com>
To: Dave Jiang <dave.jiang@intel.com>
Cc: linux-kernel@vger.kernel.org, Fenghua Yu <fenghua.yu@intel.com>,
	Vinod Koul <vkoul@kernel.org>,
	dmaengine@vger.kernel.org
Subject: Re: [PATCH v2] dmaengine: idxd: avoid deadlock in process_misc_interrupts()
Date: Wed, 24 Aug 2022 11:42:11 -0700	[thread overview]
Message-ID: <38e416b47bb30fa161e52f24ecbcf95015480fed.camel@redhat.com> (raw)
In-Reply-To: <223e5a43-95a5-da54-0ff7-c2e088a072e3@intel.com>

On Wed, 2022-08-24 at 10:45 -0700, Dave Jiang wrote:
> 
> On 8/23/2022 5:54 PM, Jerry Snitselaar wrote:
> > On Tue, Aug 23, 2022 at 09:46:19AM -0700, Dave Jiang wrote:
> > > On 8/23/2022 9:37 AM, Jerry Snitselaar wrote:
> > > > idxd_device_clear_state() now grabs the idxd->dev_lock
> > > > itself, so don't grab the lock prior to calling it.
> > > > 
> > > > This was seen in testing after dmar fault occurred on system,
> > > > resulting in lockup stack traces.
> > > > 
> > > > Cc: Fenghua Yu <fenghua.yu@intel.com>
> > > > Cc: Dave Jiang <dave.jiang@intel.com>
> > > > Cc: Vinod Koul <vkoul@kernel.org>
> > > > Cc: dmaengine@vger.kernel.org
> > > > Fixes: cf4ac3fef338 ("dmaengine: idxd: fix lockdep warning on
> > > > device driver removal")
> > > > Signed-off-by: Jerry Snitselaar <jsnitsel@redhat.com>
> > > Thanks Jerry!
> > > 
> > > Reviewed-by: Dave Jiang <dave.jiang@intel.com>
> > > 
> > I noticed another problem while looking at this. When the device
> > ends
> > up in the halted state, and needs an flr or system reset, it calls
> > idxd_wqs_unmap_portal(). Then if you do a modprobe -r idxd, you hit
> > the WARN_ON in devm_iounmap(), because the remove code path calls
> > idxd_wq_portal_unmap(), and wq->portal is null. I'm not sure if it
> > just needs a simple sanity check in drv_disable_wq() to avoid the
> > call
> > in the case that it has already been unmapped, or if more cleanup
> > needs to be done, and possibly a state to differentiate between
> > halted + soft reset possible, versus halted + flr or system reset
> > needed.  You get multiple "Device is HALTED" messages during the
> > removal as well.
> 
> Thanks!
> 
> Fenghua, can you please take a look at this when you have a chance? 
> Thank you!
> 
> 

Fenghua,

I see another potential issue. If a software reset is
attempted idxd_device_reinit() will be called which walks
the wqs, and if a wq has the state IDXD_WQ_ENABLED it calls
idxd_wq_enable(), but the first thing idxd_wq_enable() does
is see that the state is IDXD_WQ_ENABLED and returns 0.
Without the wq enable command being sent, it will not be
re-enabled, yes?

Regards,
Jerry

> > 
> > Regards,
> > Jerry
> > 
> 