From: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
To: Kishon Vijay Abraham I <kishon@ti.com>
Cc: lorenzo.pieralisi@arm.com, Vidya Sagar <vidyas@nvidia.com>,
kw@linux.com, bhelgaas@google.com, linux-pci@vger.kernel.org,
linux-kernel@vger.kernel.org, bjorn.andersson@linaro.org,
dmitry.baryshkov@linaro.org
Subject: Re: [PATCH v2] PCI: endpoint: Use blocking notifier instead of atomic
Date: Wed, 23 Mar 2022 23:39:39 +0530 [thread overview]
Message-ID: <20220323180939.GA81772@thinkpad> (raw)
In-Reply-To: <a66ccea3-b854-75d7-dc3d-6c9bb2057a0d@ti.com>
Hi Kishon,
On Wed, Mar 09, 2022 at 10:07:47AM +0530, Kishon Vijay Abraham I wrote:
> Hi Mani,
>
> On 28/02/22 11:58 am, Manivannan Sadhasivam wrote:
> > Hi,
> >
> > On Mon, Feb 28, 2022 at 11:46:52AM +0530, Kishon Vijay Abraham I wrote:
> >> Hi Manivannan,
> >>
> >> On 28/02/22 11:22 am, Manivannan Sadhasivam wrote:
> >>> The use of atomic notifier causes sleeping in atomic context bug when
> >>> the EPC core functions are used in the notifier chain. This is due to the
> >>> use of epc->lock (mutex) in core functions protecting the concurrent use of
> >>> EPC.
> >>
> >> The notification from the controller to the function driver is used for
> >> propagating interrupts to function driver and should be in interrupt context.
> >> How it should be handled maybe left to the function driver. I don't prefer
> >> moving everything to blocking notifier.
> >>
> >
> > I agree that we need to handle it quick enough but I don't see any other valid
> > options to get rid of the issue. EPF driver may use a non-atomic notifier but
> > that seems to be an overkill workaround for something that could be fixed in the
> > EPC core.
> >
> > And propagating interrupts is not going to work or needed all the time. Do you
> > forsee any issue with blocking notifier?
>
> I think any interrupt to the EP should be delivered to the function driver in
> interrupt context, it could be function level reset interrupt, hot reset
> interrupt, link state interrupt etc., These are right now not supported but it
> will use the same notification mechanism to propagate interrupt from controller
> driver to function driver.
>
In mainline, I can see only 2 users of this notifier:
1. pcie-tegra194
2. pcie-qcom-ep
In both drivers, CORE_INIT is called from a threaded irq handler so it is not
running in interrupt context. And the CORE_INIT of pci-epf-test driver is
calling EPC functions that could potentially sleep.
For LINK_UP, tegra driver is calling it from hard irq handler but the LINK_UP
of pci-epf-test driver is queueing up the delayed work. In the qcom driver,
LINK_UP is called from a threaded irq handler.
In both cases I don't see any necessity to use the atomic notifier chain.
I agree with you that the notification need to be passed from EPC to EPF asap,
but I'm not sure if it really has to be atomic.
Thanks,
Mani
> Thanks,
> Kishon
>
> >
> >> I'm wondering how other users for CORE_INIT didn't see this issue.
> >
> > This can be triggered with EPF test or NTB if CONFIG_DEBUG_ATOMIC_SLEEP is
> > enabled.
> >
> > Thanks,
> > Mani
> >
> >>
> >> Thanks,
> >> Kishon
> >>
> >>>
> >>> So switch to blocking notifier for getting rid of the bug as it runs in
> >>> non-atomic context and allows sleeping in notifier chain.
> >>>
> >>> Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> >>> ---
> >>>
> >>> Changes in v2:
> >>>
> >>> * Removed the changes related to non-upstreamed patches
> >>>
> >>> drivers/pci/endpoint/pci-epc-core.c | 6 +++---
> >>> include/linux/pci-epc.h | 4 ++--
> >>> 2 files changed, 5 insertions(+), 5 deletions(-)
> >>>
> >>> diff --git a/drivers/pci/endpoint/pci-epc-core.c b/drivers/pci/endpoint/pci-epc-core.c
> >>> index 3bc9273d0a08..c4347f472618 100644
> >>> --- a/drivers/pci/endpoint/pci-epc-core.c
> >>> +++ b/drivers/pci/endpoint/pci-epc-core.c
> >>> @@ -693,7 +693,7 @@ void pci_epc_linkup(struct pci_epc *epc)
> >>> if (!epc || IS_ERR(epc))
> >>> return;
> >>>
> >>> - atomic_notifier_call_chain(&epc->notifier, LINK_UP, NULL);
> >>> + blocking_notifier_call_chain(&epc->notifier, LINK_UP, NULL);
> >>> }
> >>> EXPORT_SYMBOL_GPL(pci_epc_linkup);
> >>>
> >>> @@ -710,7 +710,7 @@ void pci_epc_init_notify(struct pci_epc *epc)
> >>> if (!epc || IS_ERR(epc))
> >>> return;
> >>>
> >>> - atomic_notifier_call_chain(&epc->notifier, CORE_INIT, NULL);
> >>> + blocking_notifier_call_chain(&epc->notifier, CORE_INIT, NULL);
> >>> }
> >>> EXPORT_SYMBOL_GPL(pci_epc_init_notify);
> >>>
> >>> @@ -774,7 +774,7 @@ __pci_epc_create(struct device *dev, const struct pci_epc_ops *ops,
> >>>
> >>> mutex_init(&epc->lock);
> >>> INIT_LIST_HEAD(&epc->pci_epf);
> >>> - ATOMIC_INIT_NOTIFIER_HEAD(&epc->notifier);
> >>> + BLOCKING_INIT_NOTIFIER_HEAD(&epc->notifier);
> >>>
> >>> device_initialize(&epc->dev);
> >>> epc->dev.class = pci_epc_class;
> >>> diff --git a/include/linux/pci-epc.h b/include/linux/pci-epc.h
> >>> index a48778e1a4ee..04a2e74aed63 100644
> >>> --- a/include/linux/pci-epc.h
> >>> +++ b/include/linux/pci-epc.h
> >>> @@ -149,7 +149,7 @@ struct pci_epc {
> >>> /* mutex to protect against concurrent access of EP controller */
> >>> struct mutex lock;
> >>> unsigned long function_num_map;
> >>> - struct atomic_notifier_head notifier;
> >>> + struct blocking_notifier_head notifier;
> >>> };
> >>>
> >>> /**
> >>> @@ -195,7 +195,7 @@ static inline void *epc_get_drvdata(struct pci_epc *epc)
> >>> static inline int
> >>> pci_epc_register_notifier(struct pci_epc *epc, struct notifier_block *nb)
> >>> {
> >>> - return atomic_notifier_chain_register(&epc->notifier, nb);
> >>> + return blocking_notifier_chain_register(&epc->notifier, nb);
> >>> }
> >>>
> >>> struct pci_epc *
> >>>
prev parent reply other threads:[~2022-03-23 18:09 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-28 5:52 [PATCH v2] PCI: endpoint: Use blocking notifier instead of atomic Manivannan Sadhasivam
2022-02-28 6:16 ` Kishon Vijay Abraham I
2022-02-28 6:28 ` Manivannan Sadhasivam
2022-03-09 4:37 ` Kishon Vijay Abraham I
2022-03-23 18:09 ` Manivannan Sadhasivam [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220323180939.GA81772@thinkpad \
--to=manivannan.sadhasivam@linaro.org \
--cc=bhelgaas@google.com \
--cc=bjorn.andersson@linaro.org \
--cc=dmitry.baryshkov@linaro.org \
--cc=kishon@ti.com \
--cc=kw@linux.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=vidyas@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).