Re: [PATCH] PCI/DOE: Remove asynchronous task support

From: Jonathan Cameron <Jonathan.Cameron@Huawei.com>
To: "Zhuo, Qiuxu" <qiuxu.zhuo@intel.com>
Cc: "Weiny, Ira" <ira.weiny@intel.com>,
	"Williams, Dan J" <dan.j.williams@intel.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	"Gregory Price" <gregory.price@memverge.com>,
	"Li, Ming4" <ming4.li@intel.com>,
	"Verma, Vishal L" <vishal.l.verma@intel.com>,
	Lukas Wunner <lukas@wunner.de>,
	"Schofield, Alison" <alison.schofield@intel.com>,
	"linux-cxl@vger.kernel.org" <linux-cxl@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH] PCI/DOE: Remove asynchronous task support
Date: Mon, 21 Nov 2022 11:07:14 +0000	[thread overview]
Message-ID: <20221121110714.0000720a@Huawei.com> (raw)
In-Reply-To: <IA1PR11MB6171B27750AB469AFE47CD5E890A9@IA1PR11MB6171.namprd11.prod.outlook.com>

On Mon, 21 Nov 2022 02:01:32 +0000
"Zhuo, Qiuxu" <qiuxu.zhuo@intel.com> wrote:

> > From: Ira Weiny <ira.weiny@intel.com>  
> ＞　...
> > 
> > @@ -423,25 +399,13 @@ struct pci_doe_mb
> > *pcim_doe_create_mb(struct pci_dev *pdev, u16 cap_offset)
> >  	doe_mb->pdev = pdev;
> >  	doe_mb->cap_offset = cap_offset;
> >  	init_waitqueue_head(&doe_mb->wq);
> > +	mutex_init(&doe_mb->exec_lock);  
> 
> In real world, not sure whether there is a case that pcim_doe_create_mb() is invoked by 
> multiple drivers to create multiple DOE mailbox instances? If there is such a case, we may 
> need to ensure there is only one DOE mailbox instance for a physical DOE of pdev @cap_offset.

I think if that happened we'd have a lot of mess.  The main PCI driver for a given
EP, switch port etc needs to handle this part.

Sub drivers can then do similar to
https://elixir.bootlin.com/linux/latest/source/drivers/cxl/core/pci.c#L465
to find a DOE instance that supports what they need and use it.
The DOE code 'should' work fine when doing this - the request/response pairs
will be serialized.

We have discussed moving that 'find' logic and the xarray into the PCI core
and that will need to happen to support CMA etc. For the first submission it
was easier to just do it in the CXL drivers..

Jonathan

> 
> -Qiuxu