From: Bjorn Helgaas <helgaas@kernel.org>
To: Johannes Berg <johannes@sipsolutions.net>
Cc: "Heiner Kallweit" <hkallweit1@gmail.com>,
"Pali Rohár" <pali@kernel.org>, "Jonas Dreßler" <verdre@v0yd.nl>,
"Andy Shevchenko" <andy.shevchenko@gmail.com>,
"Amitkumar Karwar" <amitkarwar@gmail.com>,
"Ganapathi Bhat" <ganapathi017@gmail.com>,
"Xinming Hu" <huxinming820@gmail.com>,
"Kalle Valo" <kvalo@codeaurora.org>,
"David S. Miller" <davem@davemloft.net>,
"Jakub Kicinski" <kuba@kernel.org>,
"Tsuchiya Yuto" <kitakar@gmail.com>,
"open list:TI WILINK WIRELES..." <linux-wireless@vger.kernel.org>,
netdev <netdev@vger.kernel.org>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
linux-pci <linux-pci@vger.kernel.org>,
"Maximilian Luz" <luzmaximilian@gmail.com>,
"Andy Shevchenko" <andriy.shevchenko@linux.intel.com>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Keith Busch" <kbusch@kernel.org>
Subject: Re: [PATCH 1/2] mwifiex: Use non-posted PCI register writes
Date: Thu, 2 Sep 2021 09:05:00 -0500 [thread overview]
Message-ID: <20210902140500.GA321043@bjorn-Precision-5520> (raw)
In-Reply-To: <20210901224154.GA230445@bjorn-Precision-5520>
[+cc Keith]
On Wed, Sep 01, 2021 at 05:41:56PM -0500, Bjorn Helgaas wrote:
> On Wed, Sep 01, 2021 at 07:07:58PM +0200, Johannes Berg wrote:
> > On Wed, 2021-09-01 at 18:51 +0200, Heiner Kallweit wrote:
> > > On 01.09.2021 17:51, Pali Rohár wrote:
> > > > On Wednesday 01 September 2021 16:01:54 Jonas Dreßler wrote:
> > > > > On 8/30/21 2:49 PM, Andy Shevchenko wrote:
> > > > > > On Mon, Aug 30, 2021 at 3:38 PM Jonas Dreßler <verdre@v0yd.nl> wrote:
> > > > > > >
> > > > > > > On the 88W8897 card it's very important the TX ring write pointer is
> > > > > > > updated correctly to its new value before setting the TX ready
> > > > > > > interrupt, otherwise the firmware appears to crash (probably because
> > > > > > > it's trying to DMA-read from the wrong place).
> > > > > > >
> > >
> > > This sounds somehow like the typical case where you write DMA descriptors
> > > and then ring the doorbell. This normally requires a dma_wmb().
> > > Maybe something like that is missing here?
> >
> > But it looks like this "TX ring write pointer" is actually the register?
> >
> > However, I would agree that doing it in mwifiex_write_reg() is possibly
> > too big a hammer - could be done only for reg->tx_wrptr, not all the
> > registers?
> >
> > Actually, can two writes actually cross on PCI?
>
> Per PCIe r5.0, sec 2.4.1,
>
> A2a A Posted Request must not pass another Posted Request unless A2b
> applies.
>
> A2b A Posted Request with RO Set is permitted to pass another
> Posted Request. A Posted Request with IDO Set is permitted to
> pass another Posted Request if the two Requester IDs are
> different or if both Requests contain a PASID TLP Prefix and
> the two PASID values are different.
>
> A few drivers enable RO (Relaxed Ordering) for their devices, which
> means the *device* is permitted to set the RO bit in transactions it
> initiates.
>
> BUt IIUC we're talking about MMIO writes initiated by a CPU, and they
> won't have the RO bit set unless the Root Port has Relaxed Ordering
> enabled, and Linux generally does not enable that. So A2a should
> apply, and writes should be ordered on PCI.
>
> There are a few wrinkles that I worry about:
>
> d1e714db8129 ("mtip32xx: Fix ERO and NoSnoop values in PCIe upstream
> on AMD systems") [1] turns off RO for some AMD Root Ports, which
> makes me think BIOS might be enabling RO in these Root Ports.
>
> c56d4450eb68 ("PCI: Turn off Request Attributes to avoid Chelsio T5
> Completion erratum") [2] turns off RO for all Root Ports leading to
> Chelsio T5 devices, which again makes me think there's firmware that
> enables RO in Root Ports. Follow-up [3].
>
> 77ffc1465cec ("tegra: add PCI Express support") [4] (see
> tegra_pcie_relax_enable()) enables RO for Tegra Root Ports due to
> some hardware issue. I don't whether these Root Ports every
> actually *set* RO in the PCIe transactions they generate. Follow-up
> [5].
>
> These concern me because I don't think we have a way for drivers to
> specify whether their writes should use strong ordering or relaxed
> ordering, and I think they depend on strong ordering.
Sorry, I was completely wrong about this part. Keith Busch reminded
me that writel_relaxed() is an obvious way for drivers to specify that
they don't need strong ordering.
I apologize for the distraction.
> If Root Ports have RO enabled, I think we are at risk, so I suspect
> Linux should actively *disable* RO for Root Ports.
>
> [1] https://git.kernel.org/linus/d1e714db8129
> [2] https://git.kernel.org/linus/c56d4450eb68
> [3] https://lore.kernel.org/r/20210901222353.GA251391@bjorn-Precision-5520
> [4] https://git.kernel.org/linus/77ffc1465cec
> [5] https://lore.kernel.org/r/20210901204045.GA236987@bjorn-Precision-5520
next prev parent reply other threads:[~2021-09-02 14:05 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-30 12:37 [PATCH 0/2] mwifiex: Work around firmware bugs on 88W8897 chip Jonas Dreßler
2021-08-30 12:37 ` [PATCH 1/2] mwifiex: Use non-posted PCI register writes Jonas Dreßler
2021-08-30 12:49 ` Andy Shevchenko
2021-09-01 14:01 ` Jonas Dreßler
2021-09-01 15:47 ` Andy Shevchenko
2021-09-01 15:51 ` Pali Rohár
2021-09-01 16:51 ` Heiner Kallweit
2021-09-01 17:07 ` Johannes Berg
2021-09-01 19:07 ` Heiner Kallweit
2021-09-01 22:41 ` Bjorn Helgaas
2021-09-02 14:05 ` Bjorn Helgaas [this message]
2021-09-01 19:40 ` Brian Norris
2021-09-01 20:40 ` Andy Shevchenko
2021-09-01 21:04 ` Brian Norris
2021-09-01 21:07 ` Brian Norris
2021-09-18 7:37 ` Jonas Dreßler
2021-09-20 17:48 ` Brian Norris
2021-09-22 12:50 ` Jonas Dreßler
2021-09-23 15:28 ` Jonas Dreßler
2021-09-23 19:37 ` Andy Shevchenko
2021-09-23 19:41 ` Andy Shevchenko
2021-09-23 20:22 ` Pali Rohár
2021-09-30 15:38 ` Jonas Dreßler
2021-09-30 15:42 ` Pali Rohár
2021-09-30 16:14 ` Jonas Dreßler
2021-09-30 16:19 ` Pali Rohár
2021-09-30 16:22 ` Jonas Dreßler
2021-09-30 16:39 ` Pali Rohár
2021-08-30 12:37 ` [PATCH 2/2] mwifiex: Try waking the firmware until we get an interrupt Jonas Dreßler
2021-08-30 12:51 ` Andy Shevchenko
2021-08-30 12:55 ` Andy Shevchenko
2021-09-25 17:32 ` [PATCH 0/2] mwifiex: Work around firmware bugs on 88W8897 chip Pali Rohár
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210902140500.GA321043@bjorn-Precision-5520 \
--to=helgaas@kernel.org \
--cc=amitkarwar@gmail.com \
--cc=andriy.shevchenko@linux.intel.com \
--cc=andy.shevchenko@gmail.com \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=ganapathi017@gmail.com \
--cc=hkallweit1@gmail.com \
--cc=huxinming820@gmail.com \
--cc=johannes@sipsolutions.net \
--cc=kbusch@kernel.org \
--cc=kitakar@gmail.com \
--cc=kuba@kernel.org \
--cc=kvalo@codeaurora.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-wireless@vger.kernel.org \
--cc=luzmaximilian@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=pali@kernel.org \
--cc=verdre@v0yd.nl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).