From: Jason Gunthorpe <jgg@ziepe.ca>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Bjorn Helgaas <helgaas@kernel.org>,
Alexander Duyck <alexander.duyck@gmail.com>,
Keith Busch <kbusch@kernel.org>,
Leon Romanovsky <leon@kernel.org>,
Bjorn Helgaas <bhelgaas@google.com>,
Saeed Mahameed <saeedm@nvidia.com>,
Jakub Kicinski <kuba@kernel.org>,
linux-pci <linux-pci@vger.kernel.org>,
linux-rdma@vger.kernel.org, Netdev <netdev@vger.kernel.org>,
Don Dutile <ddutile@redhat.com>,
Alex Williamson <alex.williamson@redhat.com>,
"David S . Miller" <davem@davemloft.net>
Subject: Re: [PATCH mlx5-next v7 0/4] Dynamically assign MSI-X vectors count
Date: Wed, 31 Mar 2021 14:07:20 -0300 [thread overview]
Message-ID: <20210331170720.GY2710221@ziepe.ca> (raw)
In-Reply-To: <YGSPUewD5J+F7ZRe@kroah.com>
On Wed, Mar 31, 2021 at 05:03:45PM +0200, Greg Kroah-Hartman wrote:
> > It isn't a struct device object at all though, it just organizing
> > attributes.
>
> That's the point, it really is not. You are forced to create a real
> object for that subdirectory, and by doing so, you are "breaking" the
> driver/device model. As is evident by userspace not knowing what is
> going on here.
I'm still not really sure about what this means in practice..
I found an nested attribute in RDMA land so lets see how it behaves.
/sys/class/infiniband/ibp0s9/ <-- This is a struct device/ib_device
Then we have 261 'attribute' files under a ports subdirectory, for
instance:
/sys/class/infiniband/ibp0s9/ports/1/cm_tx_retries/dreq
Open/read works fine, and the specialty userspace that people built on
this has been working for a long time.
Does udev see the deeply nested attributes? Apparently yes:
$ udevadm info -a /sys/class/infiniband/ibp0s9
ATTR{ports/1/cm_rx_duplicates/dreq}=="0"
[..]
Given your remarks, I'm surprised, but it seems to work - I assume if
udevadm shows it then all the rules will work too.
Has udev become confused about what is a struct device? Looks like no:
$ udevadm info -a /sys/class/infiniband/ibp0s9/port
Unknown device "/sys/class/infiniband/ibp0s9/port": No such device
Can you give an example where things go wrong?
(and I inherited this RDMA stuff. In the last two years we moved it
all to netlink and modern userspace largely no longer touches sysfs,
but I can't break in-use uAPI)
> > > Does that help? The rules are:
> > > - Once you use a 'struct device', all subdirs below that device
> > > are either an attribute group for that device or a child
> > > device.
> > > - A struct device can NOT have an attribute group as a parent,
> > > it can ONLY have another struct device as a parent.
> > >
> > > If you break those rules, the kernel has the ability to get really
> > > confused unless you are very careful, and userspace will be totally lost
> > > as you can not do anything special there.
> >
> > The kernel gets confused?
>
> Putting a kobject as a child of a struct device can easily cause
> confusion as that is NOT what you should be doing. Especially if you
> then try to add a device to be a child of that kobject.
That I've never seen. I've only seen people making extra levels of
directories for organizing attributes.
> > How do you fix them? It is uAPI at this point so we can't change the
> > directory names. Can't make them struct devices (userspace would get
> > confused if we add *more* sysfs files)
>
> How would userspace get confused? If anything it would suddenly "wake
> up" and see these attributes properly.
We are talking about specialty userspace that is designed to work with
the sysfs layout as-is. Not udev. In some of these subdirs the
userspace does readdir() on - if you start adding random stuff it will
break it.
> > Since it seems like kind of a big problem can we make this allowed
> > somehow?
>
> No, not at all. Please do not do that. I will look into the existing
> users and try to see if I can fix them up. Maybe start annoying people
> by throwing warnings if you try to register a kobject as a child of a
> device...
How does that mesh with our don't break userspace ideal?? :(
> > Well, from what I understand, it wont be used because udev can't do
> > three level deep attributes, and if that hasn't been a problem in that
> > last 10 years for the existing places, it might not ever be needed in
> > udev at all.
>
> If userspace is not seeing these attributes then WHY CREATE THEM AT
> ALL???
*udev* is not the userspace! People expose sysfs attributes and then
make specialty userspace to consume them! I've seen it many times now.
> Seriously, what is needing to see these in sysfs if not the tools that
> we have today to use sysfs? Are you wanting to create new tools instead
> to handle these new attributes? Maybe just do not create them in the
> first place?
This advice is about 10 years too late :(
Regardless, lets not do deeply nested attributes here in PCI. They are
PITA anyhow.
Jason
next prev parent reply other threads:[~2021-03-31 17:08 UTC|newest]
Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-01 7:55 [PATCH mlx5-next v7 0/4] Dynamically assign MSI-X vectors count Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 1/4] PCI: Add a sysfs file to change the MSI-X table size of SR-IOV VFs Leon Romanovsky
2021-03-01 8:14 ` Greg Kroah-Hartman
2021-03-01 8:32 ` Leon Romanovsky
2021-03-01 8:37 ` Greg Kroah-Hartman
2021-03-01 8:53 ` Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 2/4] net/mlx5: Add dynamic MSI-X capabilities bits Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 3/4] net/mlx5: Dynamically assign MSI-X vectors count Leon Romanovsky
2021-03-01 7:55 ` [PATCH mlx5-next v7 4/4] net/mlx5: Implement sriov_get_vf_total_msix/count() callbacks Leon Romanovsky
2021-03-07 8:11 ` [PATCH mlx5-next v7 0/4] Dynamically assign MSI-X vectors count Leon Romanovsky
2021-03-07 18:55 ` Alexander Duyck
2021-03-07 19:19 ` Leon Romanovsky
2021-03-08 16:33 ` Alexander Duyck
2021-03-08 19:20 ` Leon Romanovsky
2021-03-10 19:09 ` Bjorn Helgaas
2021-03-10 20:10 ` Leon Romanovsky
2021-03-10 20:21 ` Greg Kroah-Hartman
2021-03-11 8:37 ` Leon Romanovsky
2021-03-10 23:34 ` Alexander Duyck
2021-03-11 18:17 ` Bjorn Helgaas
2021-03-11 19:16 ` Keith Busch
2021-03-11 19:21 ` Leon Romanovsky
2021-03-11 20:22 ` Jason Gunthorpe
2021-03-11 20:50 ` Keith Busch
2021-03-11 21:44 ` Jason Gunthorpe
2021-03-25 17:21 ` Bjorn Helgaas
2021-03-25 17:36 ` Jason Gunthorpe
2021-03-25 18:20 ` Bjorn Helgaas
2021-03-25 18:28 ` Jason Gunthorpe
2021-03-26 6:44 ` Leon Romanovsky
2021-03-26 16:00 ` Alexander Duyck
2021-03-26 16:56 ` Jason Gunthorpe
2021-03-26 17:08 ` Bjorn Helgaas
2021-03-26 17:12 ` Jason Gunthorpe
2021-03-27 6:00 ` Leon Romanovsky
2021-03-26 17:29 ` Keith Busch
2021-03-26 17:31 ` Jason Gunthorpe
2021-03-26 18:50 ` Alexander Duyck
2021-03-26 19:01 ` Jason Gunthorpe
2021-03-30 1:29 ` Bjorn Helgaas
2021-03-30 13:57 ` Jason Gunthorpe
2021-03-30 15:00 ` Bjorn Helgaas
2021-03-30 19:47 ` Jason Gunthorpe
2021-03-30 20:41 ` Bjorn Helgaas
2021-03-30 22:43 ` Jason Gunthorpe
2021-03-31 6:38 ` Greg Kroah-Hartman
2021-03-31 12:19 ` Jason Gunthorpe
2021-03-31 15:03 ` Greg Kroah-Hartman
2021-03-31 17:07 ` Jason Gunthorpe [this message]
2021-03-31 4:08 ` Leon Romanovsky
2021-04-01 1:23 ` Bjorn Helgaas
2021-04-01 11:49 ` Leon Romanovsky
2021-03-30 18:10 ` Keith Busch
2021-03-26 19:36 ` Bjorn Helgaas
2021-03-27 12:38 ` Greg Kroah-Hartman
2021-03-25 18:31 ` Keith Busch
2021-03-25 18:36 ` Jason Gunthorpe
2021-03-11 19:17 ` Leon Romanovsky
2021-03-11 19:37 ` Alexander Duyck
2021-03-11 19:51 ` Leon Romanovsky
2021-03-11 20:11 ` Alexander Duyck
2021-03-11 20:19 ` Jason Gunthorpe
2021-03-11 21:49 ` Alexander Duyck
2021-03-11 23:20 ` Jason Gunthorpe
2021-03-12 2:53 ` Alexander Duyck
2021-03-12 6:32 ` Leon Romanovsky
2021-03-12 16:59 ` Alexander Duyck
2021-03-12 17:03 ` Jason Gunthorpe
2021-03-12 18:34 ` Leon Romanovsky
2021-03-12 18:41 ` Leon Romanovsky
2021-03-12 13:00 ` Jason Gunthorpe
2021-03-12 13:36 ` Keith Busch
2021-03-11 20:31 ` Jason Gunthorpe
2021-03-10 5:58 ` Leon Romanovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210331170720.GY2710221@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=alex.williamson@redhat.com \
--cc=alexander.duyck@gmail.com \
--cc=bhelgaas@google.com \
--cc=davem@davemloft.net \
--cc=ddutile@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=helgaas@kernel.org \
--cc=kbusch@kernel.org \
--cc=kuba@kernel.org \
--cc=leon@kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=saeedm@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).