From: Bjorn Helgaas <helgaas@kernel.org>
To: Dongdong Liu <liudongdong3@huawei.com>
Cc: hch@infradead.org, kw@linux.com, logang@deltatee.com,
leon@kernel.org, linux-pci@vger.kernel.org, rajur@chelsio.com,
hverkuil-cisco@xs4all.nl, linux-media@vger.kernel.org,
netdev@vger.kernel.org
Subject: Re: [PATCH V7 7/9] PCI/sysfs: Add a 10-Bit Tag sysfs file
Date: Mon, 9 Aug 2021 12:37:33 -0500 [thread overview]
Message-ID: <20210809173733.GA2167121@bjorn-Precision-5520> (raw)
In-Reply-To: <68b85d0a-37d2-f83d-99ba-cd3936af4005@huawei.com>
On Sat, Aug 07, 2021 at 03:01:56PM +0800, Dongdong Liu wrote:
>
> On 2021/8/5 23:31, Bjorn Helgaas wrote:
> > On Thu, Aug 05, 2021 at 04:37:39PM +0800, Dongdong Liu wrote:
> > >
> > >
> > > On 2021/8/5 7:49, Bjorn Helgaas wrote:
> > > > On Wed, Aug 04, 2021 at 09:47:06PM +0800, Dongdong Liu wrote:
> > > > > PCIe spec 5.0 r1.0 section 2.2.6.2 says that if an Endpoint supports
> > > > > sending Requests to other Endpoints (as opposed to host memory), the
> > > > > Endpoint must not send 10-Bit Tag Requests to another given Endpoint
> > > > > unless an implementation-specific mechanism determines that the Endpoint
> > > > > supports 10-Bit Tag Completer capability. Add a 10bit_tag sysfs file,
> > > > > write 0 to disable 10-Bit Tag Requester when the driver does not bind
> > > > > the device if the peer device does not support the 10-Bit Tag Completer.
> > > > > This will make P2P traffic safe. the 10bit_tag file content indicate
> > > > > current 10-Bit Tag Requester Enable status.
> > > > >
> > > > > Signed-off-by: Dongdong Liu <liudongdong3@huawei.com>
> > > > > ---
> > > > > Documentation/ABI/testing/sysfs-bus-pci | 16 +++++++-
> > > > > drivers/pci/pci-sysfs.c | 69 +++++++++++++++++++++++++++++++++
> > > > > 2 files changed, 84 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
> > > > > index 793cbb7..0e0c97d 100644
> > > > > --- a/Documentation/ABI/testing/sysfs-bus-pci
> > > > > +++ b/Documentation/ABI/testing/sysfs-bus-pci
> > > > > @@ -139,7 +139,7 @@ Description:
> > > > > binary file containing the Vital Product Data for the
> > > > > device. It should follow the VPD format defined in
> > > > > PCI Specification 2.1 or 2.2, but users should consider
> > > > > - that some devices may have incorrectly formatted data.
> > > > > + that some devices may have incorrectly formatted data.
> > > > > If the underlying VPD has a writable section then the
> > > > > corresponding section of this file will be writable.
> > > > >
> > > > > @@ -407,3 +407,17 @@ Description:
> > > > >
> > > > > The file is writable if the PF is bound to a driver that
> > > > > implements ->sriov_set_msix_vec_count().
> > > > > +
> > > > > +What: /sys/bus/pci/devices/.../10bit_tag
> > > > > +Date: August 2021
> > > > > +Contact: Dongdong Liu <liudongdong3@huawei.com>
> > > > > +Description:
> > > > > + If a PCI device support 10-Bit Tag Requester, will create the
> > > > > + 10bit_tag sysfs file. The file is readable, the value
> > > > > + indicate current 10-Bit Tag Requester Enable.
> > > > > + 1 - enabled, 0 - disabled.
> > > > > +
> > > > > + The file is also writeable, the value only accept by write 0
> > > > > + to disable 10-Bit Tag Requester when the driver does not bind
> > > > > + the deivce. The typical use case is for p2pdma when the peer
> > > > > + device does not support 10-BIT Tag Completer.
> >
> > > > > +static ssize_t pci_10bit_tag_store(struct device *dev,
> > > > > + struct device_attribute *attr,
> > > > > + const char *buf, size_t count)
> > > > > +{
> > > > > + struct pci_dev *pdev = to_pci_dev(dev);
> > > > > + bool enable;
> > > > > +
> > > > > + if (kstrtobool(buf, &enable) < 0)
> > > > > + return -EINVAL;
> > > > > +
> > > > > + if (enable != false )
> > > > > + return -EINVAL;
> > > >
> > > > Is this the same as "if (enable)"?
> > > Yes, Will fix.
> >
> > I actually don't like the one-way nature of this. When the hierarchy
> > supports 10-bit tags, we automatically enable them during enumeration.
> >
> > Then we provide this sysfs file, but it can only *disable* 10-bit
> > tags. There's no way to re-enable them except by rebooting (or using
> > setpci, I guess).
> >
> > Why can't we allow *enabling* them here if they're supported in this
> > hierarchy?
> Yes, we can also provide this sysfs to enable 10-bit tag for EP devices
> when the hierarchy supports 10-bit tags.
>
> I do not want to provide sysfs to enable/disable 10-bit tag for RP
> devices as I can not tell current if the the Function has outstanding
> Non-Posted Requests, may need to unbind all the EP drivers under the
> RP, and current seems no scenario need to do this. This will make things
> more complex.
You mean "no scenario requires disabling 10-bit tags and then
re-enabling them"? That may be true, but I'm still hesitant to
provide a switch than can only be reversed by rebooting.
This is similar to the issue Leon raised that it's not practical to
reboot machines. Maybe we accept a one-way switch if the sole purpose
is to work around a hardware defect. Or maybe a kernel parameter that
disables 10-bit tags completely is the defect mitigation. I think we
probably need such a parameter in case a defect prevents us from
booting far enough to use the sysfs switch.
next prev parent reply other threads:[~2021-08-09 17:37 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-04 13:46 [PATCH V7 0/9] PCI: Enable 10-Bit tag support for PCIe devices Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 1/9] PCI: Use cached Device Capabilities Register Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 2/9] PCI: Use cached Device Capabilities 2 Register Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 3/9] PCI: Add 10-Bit Tag register definitions Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 4/9] PCI: Enable 10-Bit Tag support for PCIe Endpoint devices Dongdong Liu
2021-08-04 23:17 ` Bjorn Helgaas
2021-08-05 7:47 ` Dongdong Liu
2021-08-05 19:54 ` Bjorn Helgaas
2021-08-07 6:19 ` Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 5/9] PCI/IOV: Enable 10-Bit tag support for PCIe VF devices Dongdong Liu
2021-08-04 23:29 ` Bjorn Helgaas
2021-08-05 8:03 ` Dongdong Liu
2021-08-06 22:59 ` Bjorn Helgaas
2021-08-07 7:46 ` Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 6/9] PCI: Enable 10-Bit Tag support for PCIe RP devices Dongdong Liu
2021-08-04 23:38 ` Bjorn Helgaas
2021-08-05 8:25 ` Dongdong Liu
2021-08-09 17:26 ` Bjorn Helgaas
2021-08-10 11:59 ` Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 7/9] PCI/sysfs: Add a 10-Bit Tag sysfs file Dongdong Liu
2021-08-04 15:51 ` Logan Gunthorpe
2021-08-05 13:14 ` Dongdong Liu
2021-08-05 13:53 ` Leon Romanovsky
2021-08-05 15:36 ` Logan Gunthorpe
2021-08-04 23:49 ` Bjorn Helgaas
2021-08-05 8:37 ` Dongdong Liu
2021-08-05 15:31 ` Bjorn Helgaas
2021-08-07 7:01 ` Dongdong Liu
2021-08-09 17:37 ` Bjorn Helgaas [this message]
2021-08-10 12:16 ` Dongdong Liu
2021-08-04 23:52 ` Bjorn Helgaas
2021-08-05 8:38 ` Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 8/9] PCI/IOV: Add 10-Bit Tag sysfs files for VF devices Dongdong Liu
2021-08-05 0:05 ` Bjorn Helgaas
2021-08-05 8:47 ` Dongdong Liu
2021-08-05 9:39 ` Dongdong Liu
2021-08-04 13:47 ` [PATCH V7 9/9] PCI/P2PDMA: Add a 10-Bit Tag check in P2PDMA Dongdong Liu
2021-08-04 15:56 ` Logan Gunthorpe
2021-08-05 8:49 ` Dongdong Liu
2021-08-05 18:12 ` Bjorn Helgaas
2021-08-07 7:11 ` Dongdong Liu
2021-08-09 17:31 ` Bjorn Helgaas
2021-08-10 12:31 ` Dongdong Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210809173733.GA2167121@bjorn-Precision-5520 \
--to=helgaas@kernel.org \
--cc=hch@infradead.org \
--cc=hverkuil-cisco@xs4all.nl \
--cc=kw@linux.com \
--cc=leon@kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=liudongdong3@huawei.com \
--cc=logang@deltatee.com \
--cc=netdev@vger.kernel.org \
--cc=rajur@chelsio.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).