From: Robin Murphy <robin.murphy@arm.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Joerg Roedel <joro@8bytes.org>,
Nicolin Chen <nicolinc@nvidia.com>,
will@kernel.org, alex.williamson@redhat.com,
suravee.suthikulpanit@amd.com, marcan@marcan.st,
sven@svenpeter.dev, alyssa@rosenzweig.io, robdclark@gmail.com,
dwmw2@infradead.org, baolu.lu@linux.intel.com,
mjrosato@linux.ibm.com, gerald.schaefer@linux.ibm.com,
orsonzhai@gmail.com, baolin.wang@linux.alibaba.com,
zhang.lyra@gmail.com, thierry.reding@gmail.com,
vdumpa@nvidia.com, jonathanh@nvidia.com,
jean-philippe@linaro.org, cohuck@redhat.com, tglx@linutronix.de,
shameerali.kolothum.thodi@huawei.com, thunder.leizhen@huawei.com,
christophe.jaillet@wanadoo.fr, yangyingliang@huawei.com,
jon@solid-run.com, iommu@lists.linux.dev,
linux-kernel@vger.kernel.org, asahi@lists.linux.dev,
linux-arm-kernel@lists.infradead.org,
linux-arm-msm@vger.kernel.org, linux-s390@vger.kernel.org,
linux-tegra@vger.kernel.org,
virtualization@lists.linux-foundation.org, kvm@vger.kernel.org,
kevin.tian@intel.com
Subject: Re: [PATCH v6 1/5] iommu: Return -EMEDIUMTYPE for incompatible domain and device/group
Date: Wed, 7 Sep 2022 20:41:13 +0100 [thread overview]
Message-ID: <0b466705-3a17-1bbc-7ef2-5adadc22d1ae@arm.com> (raw)
In-Reply-To: <YxjOPo5FFqu2vE/g@nvidia.com>
On 2022-09-07 18:00, Jason Gunthorpe wrote:
> On Wed, Sep 07, 2022 at 03:23:09PM +0100, Robin Murphy wrote:
>> On 2022-09-07 14:47, Jason Gunthorpe wrote:
>>> On Wed, Sep 07, 2022 at 02:41:54PM +0200, Joerg Roedel wrote:
>>>> On Mon, Aug 15, 2022 at 11:14:33AM -0700, Nicolin Chen wrote:
>>>>> Provide a dedicated errno from the IOMMU driver during attach that the
>>>>> reason attached failed is because of domain incompatability. EMEDIUMTYPE
>>>>> is chosen because it is never used within the iommu subsystem today and
>>>>> evokes a sense that the 'medium' aka the domain is incompatible.
>>>>
>>>> I am not a fan of re-using EMEDIUMTYPE or any other special value. What
>>>> is needed here in EINVAL, but with a way to tell the caller which of the
>>>> function parameters is actually invalid.
>>>
>>> Using errnos to indicate the nature of failure is a well established
>>> unix practice, it is why we have hundreds of error codes and don't
>>> just return -EINVAL for everything.
>>>
>>> What don't you like about it?
>>>
>>> Would you be happier if we wrote it like
>>>
>>> #define IOMMU_EINCOMPATIBLE_DEVICE xx
>>>
>>> Which tells "which of the function parameters is actually invalid" ?
>>
>> FWIW, we're now very close to being able to validate dev->iommu against
>> where the domain came from in core code, and so short-circuit ->attach_dev
>> entirely if they don't match.
>
> I don't think this is a long term direction. We have systems now with
> a number of SMMU blocks and we really are going to see a need that
> they share the iommu_domains so we don't have unncessary overheads
> from duplicated io page table memory.
>
> So ultimately I'd expect to pass the iommu_domain to the driver and
> the driver will decide if the page table memory it represents is
> compatible or not. Restricting to only the same iommu instance isn't
> good..
Who said IOMMU instance? As a reminder, the patch I currently have[1] is
matching the driver (via the device ops), which happens to be entirely
compatible with drivers supporting cross-instance domains. Mostly
because we already have drivers that support cross-instance domains and
callers that use them.
>> At that point -EINVAL at the driver callback level could be assumed
>> to refer to the domain argument, while anything else could be taken
>> as something going unexpectedly wrong when the attach may otherwise
>> have worked. I've forgotten if we actually had a valid case anywhere
>> for "this is my device but even if you retry with a different domain
>> it's still never going to work", but I think we wouldn't actually
>> need that anyway - it should be clear enough to a caller that if
>> attaching to an existing domain fails, then allocating a fresh
>> domain and attaching also fails, that's the point to give up.
>
> The point was to have clear error handling, we either have permenent
> errors or 'this domain will never work with this device error'.
>
> If we treat all error as temporary and just retry randomly it can
> create a mess. For instance we might fail to attach to a perfectly
> compatible domain due to ENOMEM or something and then go on to
> successfully a create a new 2nd domain, just due to races.
>
> We can certainly code the try everything then allocate scheme, it is
> just much more fragile than having definitive error codes.
Again, not what I was suggesting. In fact the nature of
iommu_attach_group() already rules out bogus devices getting this far,
so all a driver currently has to worry about is compatibility of a
device that it definitely probed with a domain that it definitely
allocated. Therefore, from a caller's point of view, if attaching to an
existing domain returns -EINVAL, try another domain; multiple different
existing domains can be tried, and may also return -EINVAL for the same
or different reasons; the final attempt is to allocate a fresh domain
and attach to that, which should always be nominally valid and *never*
return -EINVAL. If any attempt returns any other error, bail out down
the usual "this should have worked but something went wrong" path. Even
if any driver did have a nonsensical "nothing went wrong, I just can't
attach my device to any of my domains" case, I don't think it would
really need distinguishing from any other general error anyway.
Once multiple drivers are in play, the only addition is that the
"gatekeeper" check inside iommu_attach_group() may also return -EINVAL
if the device is managed by a different driver, since that still fits
the same "try again with a different domain" message to the caller.
It's actually quite neat - basically the exact same thing we've tried to
do with -EMEDIUMTYPE here, but more self-explanatory, since the fact is
that a domain itself should never be invalid for attaching to via its
own ops, and a group should never be inherently invalid for attaching to
a suitable domain, it is only ever a particular combination of group (or
device at the internal level) and domain that may not be valid together.
Thus as long as we can maintain that basic guarantee that attaching a
group to a newly allocated domain can only ever fail for resource
allocation reasons and not some spurious "incompatibility", then we
don't need any obscure trickery, and a single, clear, error code is in
fact enough to say all that needs to be said.
Whether iommu_attach_device() should also join the party and start
rejecting non-singleton-group devices with a different error, or
maintain its current behaviour since its legacy users already have their
expectations set, is another matter in its own right.
Cheers,
Robin.
[1]
https://gitlab.arm.com/linux-arm/linux-rm/-/commit/683cdff1b2d4ae11f56e38d93b37e66e8c939fc9
next prev parent reply other threads:[~2022-09-07 19:41 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-15 18:14 [PATCH v6 0/5] Simplify vfio_iommu_type1 attach/detach routine Nicolin Chen
2022-08-15 18:14 ` [PATCH v6 1/5] iommu: Return -EMEDIUMTYPE for incompatible domain and device/group Nicolin Chen
2022-09-07 12:41 ` Joerg Roedel
2022-09-07 13:47 ` Jason Gunthorpe
2022-09-07 14:06 ` Joerg Roedel
2022-09-07 17:10 ` Jason Gunthorpe
2022-09-08 13:28 ` Joerg Roedel
2022-09-08 16:14 ` Jason Gunthorpe
2022-09-09 3:17 ` Nicolin Chen
2022-09-09 5:00 ` Tian, Kevin
2022-09-09 12:07 ` Jason Gunthorpe
2022-09-13 2:22 ` Tian, Kevin
2022-09-13 5:07 ` Nicolin Chen
2022-09-07 14:23 ` Robin Murphy
2022-09-07 17:00 ` Jason Gunthorpe
2022-09-07 19:41 ` Robin Murphy [this message]
2022-09-08 0:43 ` Jason Gunthorpe
2022-09-08 9:30 ` Tian, Kevin
2022-09-08 12:08 ` Jason Gunthorpe
2022-09-10 23:35 ` Nicolin Chen
2022-09-13 2:24 ` Tian, Kevin
2022-09-13 8:36 ` Nicolin Chen
2022-09-08 9:54 ` Tian, Kevin
2022-09-08 10:25 ` Robin Murphy
2022-08-15 18:14 ` [PATCH v6 2/5] vfio/iommu_type1: Prefer to reuse domains vs match enforced cache coherency Nicolin Chen
2022-08-15 18:14 ` [PATCH v6 3/5] vfio/iommu_type1: Remove the domain->ops comparison Nicolin Chen
2022-08-15 18:14 ` [PATCH v6 4/5] vfio/iommu_type1: Clean up update_dirty_scope in detach_group() Nicolin Chen
2022-08-15 18:14 ` [PATCH v6 5/5] vfio/iommu_type1: Simplify group attachment Nicolin Chen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b466705-3a17-1bbc-7ef2-5adadc22d1ae@arm.com \
--to=robin.murphy@arm.com \
--cc=alex.williamson@redhat.com \
--cc=alyssa@rosenzweig.io \
--cc=asahi@lists.linux.dev \
--cc=baolin.wang@linux.alibaba.com \
--cc=baolu.lu@linux.intel.com \
--cc=christophe.jaillet@wanadoo.fr \
--cc=cohuck@redhat.com \
--cc=dwmw2@infradead.org \
--cc=gerald.schaefer@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jean-philippe@linaro.org \
--cc=jgg@nvidia.com \
--cc=jon@solid-run.com \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=mjrosato@linux.ibm.com \
--cc=nicolinc@nvidia.com \
--cc=orsonzhai@gmail.com \
--cc=robdclark@gmail.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=sven@svenpeter.dev \
--cc=tglx@linutronix.de \
--cc=thierry.reding@gmail.com \
--cc=thunder.leizhen@huawei.com \
--cc=vdumpa@nvidia.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=will@kernel.org \
--cc=yangyingliang@huawei.com \
--cc=zhang.lyra@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).