[PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

[PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

[Nicolin Chen]

Local boot test and VFIO sanity test show that info->iommu can be
used in device_to_iommu() as a fast path. So this patch adds it.

Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
---
 drivers/iommu/intel/iommu.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 2990f80c5e08..412fca5ab9cd 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -777,6 +777,7 @@ static bool iommu_is_dummy(struct intel_iommu *iommu, struct device *dev)
 struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
 {
 	struct dmar_drhd_unit *drhd = NULL;
+	struct device_domain_info *info;
 	struct pci_dev *pdev = NULL;
 	struct intel_iommu *iommu;
 	struct device *tmp;
@@ -786,6 +787,10 @@ struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
 	if (!dev)
 		return NULL;
 
+	info = dev_iommu_priv_get(dev);
+	if (info)
+		return info->iommu;
+
 	if (dev_is_pci(dev)) {
 		struct pci_dev *pf_pdev;
 
-- 
2.17.1

Re: [PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

[Baolu Lu]

On 2022/5/13 08:32, Nicolin Chen wrote:
> Local boot test and VFIO sanity test show that info->iommu can be
> used in device_to_iommu() as a fast path. So this patch adds it.
> 
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ---
>   drivers/iommu/intel/iommu.c | 5 +++++
>   1 file changed, 5 insertions(+)
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 2990f80c5e08..412fca5ab9cd 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -777,6 +777,7 @@ static bool iommu_is_dummy(struct intel_iommu *iommu, struct device *dev)
>   struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
>   {
>   	struct dmar_drhd_unit *drhd = NULL;
> +	struct device_domain_info *info;
>   	struct pci_dev *pdev = NULL;
>   	struct intel_iommu *iommu;
>   	struct device *tmp;
> @@ -786,6 +787,10 @@ struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
>   	if (!dev)
>   		return NULL;
>   
> +	info = dev_iommu_priv_get(dev);
> +	if (info)
> +		return info->iommu;

device_to_iommu() also returns device source id (@bus, @devfn).

Perhaps, we can make device_to_iommu() only for probe_device() where the
per-device info data is not initialized yet. After probe_device(), iommu
and sid are retrieved through other helpers by looking up the device
info directly?

Best regards,
baolu

> +
>   	if (dev_is_pci(dev)) {
>   		struct pci_dev *pf_pdev;
>   

Re: [PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

[Nicolin Chen]

On Fri, May 13, 2022 at 11:32:11AM +0800, Baolu Lu wrote:
> External email: Use caution opening links or attachments
> 
> 
> On 2022/5/13 08:32, Nicolin Chen wrote:
> > Local boot test and VFIO sanity test show that info->iommu can be
> > used in device_to_iommu() as a fast path. So this patch adds it.
> > 
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > ---
> >   drivers/iommu/intel/iommu.c | 5 +++++
> >   1 file changed, 5 insertions(+)
> > 
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 2990f80c5e08..412fca5ab9cd 100644
> > --- a/drivers/iommu/intel/iommu.c
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -777,6 +777,7 @@ static bool iommu_is_dummy(struct intel_iommu *iommu, struct device *dev)
> >   struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
> >   {
> >       struct dmar_drhd_unit *drhd = NULL;
> > +     struct device_domain_info *info;
> >       struct pci_dev *pdev = NULL;
> >       struct intel_iommu *iommu;
> >       struct device *tmp;
> > @@ -786,6 +787,10 @@ struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
> >       if (!dev)
> >               return NULL;
> > 
> > +     info = dev_iommu_priv_get(dev);
> > +     if (info)
> > +             return info->iommu;
> 
> device_to_iommu() also returns device source id (@bus, @devfn).
> 
> Perhaps, we can make device_to_iommu() only for probe_device() where the
> per-device info data is not initialized yet. After probe_device(), iommu
> and sid are retrieved through other helpers by looking up the device
> info directly?

That should work I think. I was just not sure when the priv
could be unset. But it seems that you have cleaned up those
places other than probe/release_device() in recent version :)

Nic

Re: [PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

[Jason Gunthorpe]

On Fri, May 13, 2022 at 11:32:11AM +0800, Baolu Lu wrote:
> On 2022/5/13 08:32, Nicolin Chen wrote:
> > Local boot test and VFIO sanity test show that info->iommu can be
> > used in device_to_iommu() as a fast path. So this patch adds it.
> > 
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> >   drivers/iommu/intel/iommu.c | 5 +++++
> >   1 file changed, 5 insertions(+)
> > 
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 2990f80c5e08..412fca5ab9cd 100644
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -777,6 +777,7 @@ static bool iommu_is_dummy(struct intel_iommu *iommu, struct device *dev)
> >   struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
> >   {
> >   	struct dmar_drhd_unit *drhd = NULL;
> > +	struct device_domain_info *info;
> >   	struct pci_dev *pdev = NULL;
> >   	struct intel_iommu *iommu;
> >   	struct device *tmp;
> > @@ -786,6 +787,10 @@ struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn)
> >   	if (!dev)
> >   		return NULL;
> > +	info = dev_iommu_priv_get(dev);
> > +	if (info)
> > +		return info->iommu;
> 
> device_to_iommu() also returns device source id (@bus, @devfn).

Those are in the info too..

> Perhaps, we can make device_to_iommu() only for probe_device() where the
> per-device info data is not initialized yet. After probe_device(), iommu
> and sid are retrieved through other helpers by looking up the device
> info directly?

This design makes the most sense to me... Nicolin you said there was a
case where attach was happening before probe though??

Jasno

Re: [PATCH] iommu/vt-d: Try info->iommu in device_to_iommu()

[Nicolin Chen]

On Fri, May 13, 2022 at 08:50:32AM -0300, Jason Gunthorpe wrote:

> > Perhaps, we can make device_to_iommu() only for probe_device() where the
> > per-device info data is not initialized yet. After probe_device(), iommu
> > and sid are retrieved through other helpers by looking up the device
> > info directly?
> 
> This design makes the most sense to me... Nicolin you said there was a
> case where attach was happening before probe though??

I was testing on top of our IOMMUFD dev branch actually, which
is still on 5.17 where priv/info seems to be set at the end of
first ->attach_dev() call. In 5.18, Baolu has already cleaned
away some code. So now, just eyeballing here, it should be set
at driver's ->probe_device() call, so it'd be safe to get info
in attach_dev (and in the new op that we are adding).

I am also rebasing our dev branch to more recent version btw.

Thanks
Nic