From: Ira Weiny <ira.weiny@intel.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Ben Widawsky <bwidawsk@kernel.org>,
"Alison Schofield" <alison.schofield@intel.com>,
Vishal Verma <vishal.l.verma@intel.com>,
Dave Jiang <dave.jiang@intel.com>, <linux-kernel@vger.kernel.org>,
<linux-cxl@vger.kernel.org>, <linux-pci@vger.kernel.org>
Subject: Re: [PATCH V11 4/8] cxl/pci: Create PCI DOE mailbox's for memory devices
Date: Wed, 22 Jun 2022 16:18:27 -0700 [thread overview]
Message-ID: <YrOjQ3IkGZpe1lpN@iweiny-desk3> (raw)
In-Reply-To: <YrIOD9ttvAabfyNu@iweiny-desk3>
On Tue, Jun 21, 2022 at 11:29:35AM -0700, Ira wrote:
> On Fri, Jun 17, 2022 at 04:44:27PM -0700, Dan Williams wrote:
> > ira.weiny@ wrote:
>
> [snip]
>
> > > diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h
> > > index 60d10ee1e7fc..4d2764b865ab 100644
> > > --- a/drivers/cxl/cxlmem.h
> > > +++ b/drivers/cxl/cxlmem.h
> > > @@ -191,6 +191,8 @@ struct cxl_endpoint_dvsec_info {
> > > * @component_reg_phys: register base of component registers
> > > * @info: Cached DVSEC information about the device.
> > > * @serial: PCIe Device Serial Number
> > > + * @doe_mbs: PCI DOE mailbox array
> > > + * @num_mbs: Number of DOE mailboxes
> > > * @mbox_send: @dev specific transport for transmitting mailbox commands
> > > *
> > > * See section 8.2.9.5.2 Capacity Configuration and Label Storage for
> > > @@ -224,6 +226,10 @@ struct cxl_dev_state {
> > > resource_size_t component_reg_phys;
> > > u64 serial;
> > >
> > > + bool doe_use_irq;
> >
> > Don't pass temporary state through a long lived data structure. Just
> > pass flag by reference between the functions that want to coordinate
> > this.
>
> Done.
>
> [snip]
>
> > > +
> > > +static void cxl_alloc_irq_vectors(struct cxl_dev_state *cxlds)
> > > +{
> > > + struct device *dev = cxlds->dev;
> > > + struct pci_dev *pdev = to_pci_dev(dev);
> > > + int max_irqs = 0;
> > > + int off = 0;
> > > + int rc;
> > > +
> > > + /* Account for all the DOE vectors needed */
> > > + pci_doe_for_each_off(pdev, off) {
> > > + int irq = pci_doe_get_irq_num(pdev, off);
> > > +
> > > + if (irq < 0)
> > > + continue;
> > > + max_irqs = max(max_irqs, irq + 1);
> >
> > This seems to assume that different DOEs will get independent vectors.
> > The driver needs to be prepared for DOE instances, Event notifications,
> > and mailbox commands to share a single MSI vector in the worst case.
> > Lets focus on polled mode DOE, or explicitly only support interrupt
> > based operation when no vector sharing is detected.
> >
>
> Ok I see now. I was under the impression they had to be unique.
>
> Do you think it is sufficient to check in this loop for duplicates and bail if
> any are shared?
I'm still removing the irq code from the CXL layer but I had to look a bit
deeper at this for my own knowledge.
I don't think shared interrupt numbers is a problem because the
pci_request_irq() used within pci_doe_create_mb() specifies IRQF_SHARED.
drivers/pci/irq.c:
int pci_request_irq(struct pci_dev *dev, unsigned int nr, irq_handler_t handler,
irq_handler_t thread_fn, void *dev_id, const char *fmt, ...)
{
...
unsigned long irqflags = IRQF_SHARED;
...
So I think this would work even with share vectors, right?
Regardless, setting up the CXL/PCI IRQs is a bit of a mess. So I'm still going
to remove the IRQ code in the CXL layer. But I think it is safe to leave the
IRQ code in the pci/doe.c layer for others to use.
Ira
next prev parent reply other threads:[~2022-06-22 23:18 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-10 20:22 [PATCH V11 0/8] CXL: Read CDAT and DSMAS data ira.weiny
2022-06-10 20:22 ` [PATCH V11 1/8] PCI: Add vendor ID for the PCI SIG ira.weiny
2022-06-10 20:22 ` [PATCH V11 2/8] PCI: Replace magic constant for PCI Sig Vendor ID ira.weiny
2022-06-10 20:22 ` [PATCH V11 3/8] PCI: Create PCI library functions in support of DOE mailboxes ira.weiny
2022-06-14 3:53 ` Li, Ming
2022-06-15 4:18 ` Ira Weiny
2022-06-17 22:40 ` Bjorn Helgaas
2022-06-18 16:39 ` Bjorn Helgaas
2022-06-22 16:46 ` Ira Weiny
2022-06-20 9:24 ` Jonathan Cameron
2022-06-22 23:06 ` Ira Weiny
2022-06-22 16:38 ` Ira Weiny
2022-06-17 22:56 ` Dan Williams
2022-06-20 10:23 ` Jonathan Cameron
2022-06-22 22:57 ` Ira Weiny
2022-06-23 18:03 ` Dan Williams
2022-06-22 22:37 ` Ira Weiny
2022-06-22 22:45 ` Ira Weiny
2022-06-22 22:57 ` Dan Williams
2022-06-23 0:25 ` Ira Weiny
2022-06-23 10:24 ` Jonathan Cameron
2022-06-23 18:14 ` Dan Williams
2022-06-23 18:07 ` Dan Williams
2022-06-10 20:22 ` [PATCH V11 4/8] cxl/pci: Create PCI DOE mailbox's for memory devices ira.weiny
2022-06-17 20:40 ` [PATCH v11 " Davidlohr Bueso
2022-06-17 20:51 ` Davidlohr Bueso
2022-06-21 18:24 ` Ira Weiny
2022-06-17 23:44 ` [PATCH V11 " Dan Williams
2022-06-21 18:29 ` Ira Weiny
2022-06-22 23:18 ` Ira Weiny [this message]
2022-06-21 20:37 ` Bjorn Helgaas
2022-06-10 20:22 ` [PATCH V11 5/8] cxl/port: Read CDAT table ira.weiny
2022-06-18 0:43 ` Dan Williams
2022-06-21 19:10 ` Dan Williams
2022-06-21 19:34 ` Lukas Wunner
2022-06-21 19:41 ` Dan Williams
2022-06-21 20:38 ` Ira Weiny
2022-06-21 21:14 ` Ira Weiny
2022-06-21 21:48 ` Dan Williams
2022-06-28 3:24 ` Ira Weiny
2022-06-10 20:22 ` [PATCH V11 6/8] cxl/port: Introduce cxl_cdat_valid() ira.weiny
2022-06-10 20:22 ` [PATCH V11 7/8] cxl/port: Retry reading CDAT on failure ira.weiny
2022-06-28 3:32 ` Alison Schofield
2022-06-10 20:22 ` [PATCH V11 8/8] cxl/port: Parse out DSMAS data from CDAT table ira.weiny
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YrOjQ3IkGZpe1lpN@iweiny-desk3 \
--to=ira.weiny@intel.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=alison.schofield@intel.com \
--cc=bhelgaas@google.com \
--cc=bwidawsk@kernel.org \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).