From: "Liu, Yi L" <yi.l.liu@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: "jgg@nvidia.com" <jgg@nvidia.com>,
"Tian, Kevin" <kevin.tian@intel.com>,
"joro@8bytes.org" <joro@8bytes.org>,
"robin.murphy@arm.com" <robin.murphy@arm.com>,
"cohuck@redhat.com" <cohuck@redhat.com>,
"eric.auger@redhat.com" <eric.auger@redhat.com>,
"nicolinc@nvidia.com" <nicolinc@nvidia.com>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
"chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
"yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
"peterx@redhat.com" <peterx@redhat.com>,
"jasowang@redhat.com" <jasowang@redhat.com>,
"shameerali.kolothum.thodi@huawei.com"
<shameerali.kolothum.thodi@huawei.com>,
"lulu@redhat.com" <lulu@redhat.com>,
"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
"intel-gvt-dev@lists.freedesktop.org"
<intel-gvt-dev@lists.freedesktop.org>,
"intel-gfx@lists.freedesktop.org"
<intel-gfx@lists.freedesktop.org>,
"linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
"Hao, Xudong" <xudong.hao@intel.com>,
"Zhao, Yan Y" <yan.y.zhao@intel.com>,
"Xu, Terrence" <terrence.xu@intel.com>,
"Jiang, Yanting" <yanting.jiang@intel.com>
Subject: RE: [PATCH v8 23/24] vfio: Compile group optionally
Date: Thu, 30 Mar 2023 13:06:37 +0000 [thread overview]
Message-ID: <DS0PR11MB75294C1C51B2B91D6D89958AC38E9@DS0PR11MB7529.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20230329155118.1a76d937.alex.williamson@redhat.com>
> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Thursday, March 30, 2023 5:51 AM
>
> On Mon, 27 Mar 2023 02:40:46 -0700
> Yi Liu <yi.l.liu@intel.com> wrote:
>
> > group code is not needed for vfio device cdev, so with vfio device cdev
> > introduced, the group infrastructures can be compiled out if only cdev
> > is needed.
> >
> > Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> > Tested-by: Terrence Xu <terrence.xu@intel.com>
> > Signed-off-by: Yi Liu <yi.l.liu@intel.com>
> > ---
> > drivers/iommu/iommufd/Kconfig | 4 +-
> > drivers/vfio/Kconfig | 16 ++++-
> > drivers/vfio/Makefile | 2 +-
> > drivers/vfio/vfio.h | 111 ++++++++++++++++++++++++++++++++--
> > include/linux/vfio.h | 13 +++-
> > 5 files changed, 134 insertions(+), 12 deletions(-)
> >
> > diff --git a/drivers/iommu/iommufd/Kconfig b/drivers/iommu/iommufd/Kconfig
> > index ada693ea51a7..1946eed1826a 100644
> > --- a/drivers/iommu/iommufd/Kconfig
> > +++ b/drivers/iommu/iommufd/Kconfig
> > @@ -14,8 +14,8 @@ config IOMMUFD
> > if IOMMUFD
> > config IOMMUFD_VFIO_CONTAINER
> > bool "IOMMUFD provides the VFIO container /dev/vfio/vfio"
> > - depends on VFIO && !VFIO_CONTAINER
> > - default VFIO && !VFIO_CONTAINER
> > + depends on VFIO && VFIO_GROUP && !VFIO_CONTAINER
> > + default VFIO && VFIO_GROUP && !VFIO_CONTAINER
>
> Shouldn't these simply replace VFIO with VFIO_GROUP since VFIO_GROUP
> necessarily depends on VFIO?
looks so.
>
> > help
> > IOMMUFD will provide /dev/vfio/vfio instead of VFIO. This relies on
> > IOMMUFD providing compatibility emulation to give the same ioctls.
> > diff --git a/drivers/vfio/Kconfig b/drivers/vfio/Kconfig
> > index e2105b4dac2d..0942a19601a2 100644
> > --- a/drivers/vfio/Kconfig
> > +++ b/drivers/vfio/Kconfig
> > @@ -4,7 +4,9 @@ menuconfig VFIO
> > select IOMMU_API
> > depends on IOMMUFD || !IOMMUFD
> > select INTERVAL_TREE
> > - select VFIO_CONTAINER if IOMMUFD=n
> > + select VFIO_GROUP if SPAPR_TCE_IOMMU || !IOMMUFD
>
> This needs to be IOMMUFD=n or else VFIO_GROUP cannot be unset when
> IOMMUFD=m
yes. btw. does it mean the "depends on IOMMUFD || !IOMMUFD"
also use IOMMUFD=n?
>
> > + select VFIO_DEVICE_CDEV if !VFIO_GROUP
> > + select VFIO_CONTAINER if IOMMUFD=n && VFIO_GROUP
>
> The fact that CONTAINER depends on GROUP seems to be sufficient that we
> don't need GROUP here. Thanks,
right. I added VFIO_GROUP as I saw a time that container code was comipled
when IOMMUFD=n and VFIO_GROUP=n. This encounters compiling issue since
container code refers device->group->xxx. But this should have been fixed by
selecting VFIO_GROUP when IOMMUFD=n.
Thanks,
Yi Liu
>
> Alex
>
> > help
> > VFIO provides a framework for secure userspace device drivers.
> > See Documentation/driver-api/vfio.rst for more details.
> > @@ -15,6 +17,7 @@ if VFIO
> > config VFIO_DEVICE_CDEV
> > bool "Support for the VFIO cdev /dev/vfio/devices/vfioX"
> > depends on IOMMUFD
> > + default !VFIO_GROUP
> > help
> > The VFIO device cdev is another way for userspace to get device
> > access. Userspace gets device fd by opening device cdev under
> > @@ -23,9 +26,20 @@ config VFIO_DEVICE_CDEV
> >
> > If you don't know what to do here, say N.
> >
> > +config VFIO_GROUP
> > + bool "Support for the VFIO group /dev/vfio/$group_id"
> > + default y
> > + help
> > + VFIO group support provides the traditional model for accessing
> > + devices through VFIO and is used by the majority of userspace
> > + applications and drivers making use of VFIO.
> > +
> > + If you don't know what to do here, say Y.
> > +
> > config VFIO_CONTAINER
> > bool "Support for the VFIO container /dev/vfio/vfio"
> > select VFIO_IOMMU_TYPE1 if MMU && (X86 || S390 || ARM || ARM64)
> > + depends on VFIO_GROUP
> > default y
> > help
> > The VFIO container is the classic interface to VFIO for establishing
> > diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile
> > index 245394aeb94b..57c3515af606 100644
> > --- a/drivers/vfio/Makefile
> > +++ b/drivers/vfio/Makefile
> > @@ -2,9 +2,9 @@
> > obj-$(CONFIG_VFIO) += vfio.o
> >
> > vfio-y += vfio_main.o \
> > - group.o \
> > iova_bitmap.o
> > vfio-$(CONFIG_VFIO_DEVICE_CDEV) += device_cdev.o
> > +vfio-$(CONFIG_VFIO_GROUP) += group.o
> > vfio-$(CONFIG_IOMMUFD) += iommufd.o
> > vfio-$(CONFIG_VFIO_CONTAINER) += container.o
> > vfio-$(CONFIG_VFIO_VIRQFD) += virqfd.o
> > diff --git a/drivers/vfio/vfio.h b/drivers/vfio/vfio.h
> > index c199e410db18..9c7a238ec8dd 100644
> > --- a/drivers/vfio/vfio.h
> > +++ b/drivers/vfio/vfio.h
> > @@ -36,6 +36,12 @@ vfio_allocate_device_file(struct vfio_device *device);
> >
> > extern const struct file_operations vfio_device_fops;
> >
> > +#ifdef CONFIG_VFIO_NOIOMMU
> > +extern bool vfio_noiommu __read_mostly;
> > +#else
> > +enum { vfio_noiommu = false };
> > +#endif
> > +
> > enum vfio_group_type {
> > /*
> > * Physical device with IOMMU backing.
> > @@ -60,6 +66,7 @@ enum vfio_group_type {
> > VFIO_NO_IOMMU,
> > };
> >
> > +#if IS_ENABLED(CONFIG_VFIO_GROUP)
> > struct vfio_group {
> > struct device dev;
> > struct cdev cdev;
> > @@ -113,6 +120,104 @@ static inline void vfio_device_set_noiommu(struct
> vfio_device *device)
> > device->noiommu = IS_ENABLED(CONFIG_VFIO_NOIOMMU) &&
> > device->group->type == VFIO_NO_IOMMU;
> > }
> > +#else
> > +struct vfio_group;
> > +
> > +static inline int vfio_device_block_group(struct vfio_device *device)
> > +{
> > + return 0;
> > +}
> > +
> > +static inline void vfio_device_unblock_group(struct vfio_device *device)
> > +{
> > +}
> > +
> > +static inline int vfio_device_set_group(struct vfio_device *device,
> > + enum vfio_group_type type)
> > +{
> > + return 0;
> > +}
> > +
> > +static inline void vfio_device_remove_group(struct vfio_device *device)
> > +{
> > +}
> > +
> > +static inline void vfio_device_group_register(struct vfio_device *device)
> > +{
> > +}
> > +
> > +static inline void vfio_device_group_unregister(struct vfio_device *device)
> > +{
> > +}
> > +
> > +static inline bool vfio_device_group_uses_container(struct vfio_device_file *df)
> > +{
> > + return false;
> > +}
> > +
> > +static inline int vfio_device_group_use_iommu(struct vfio_device *device)
> > +{
> > + return -EOPNOTSUPP;
> > +}
> > +
> > +static inline void vfio_device_group_unuse_iommu(struct vfio_device *device)
> > +{
> > +}
> > +
> > +static inline void vfio_device_group_close(struct vfio_device_file *df)
> > +{
> > +}
> > +
> > +static inline struct vfio_group *vfio_group_from_file(struct file *file)
> > +{
> > + return NULL;
> > +}
> > +
> > +static inline bool vfio_group_has_dev(struct vfio_group *group,
> > + struct vfio_device *device)
> > +{
> > + return false;
> > +}
> > +
> > +static inline bool vfio_group_enforced_coherent(struct vfio_group *group)
> > +{
> > + return true;
> > +}
> > +
> > +static inline void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
> > +{
> > +}
> > +
> > +static inline bool vfio_device_has_container(struct vfio_device *device)
> > +{
> > + return false;
> > +}
> > +
> > +static inline int __init vfio_group_init(void)
> > +{
> > + return 0;
> > +}
> > +
> > +static inline void vfio_group_cleanup(void)
> > +{
> > +}
> > +
> > +static inline void vfio_device_set_noiommu(struct vfio_device *device)
> > +{
> > + struct iommu_group *iommu_group;
> > +
> > + device->noiommu = false;
> > +
> > + if (!IS_ENABLED(CONFIG_VFIO_NOIOMMU) || !vfio_noiommu)
> > + return;
> > +
> > + iommu_group = iommu_group_get(device->dev);
> > + if (iommu_group)
> > + iommu_group_put(iommu_group);
> > + else
> > + device->noiommu = true;
> > +}
> > +#endif /* CONFIG_VFIO_GROUP */
> >
> > #if IS_ENABLED(CONFIG_VFIO_CONTAINER)
> > /**
> > @@ -356,12 +461,6 @@ static inline void vfio_virqfd_exit(void)
> > }
> > #endif
> >
> > -#ifdef CONFIG_VFIO_NOIOMMU
> > -extern bool vfio_noiommu __read_mostly;
> > -#else
> > -enum { vfio_noiommu = false };
> > -#endif
> > -
> > #ifdef CONFIG_HAVE_KVM
> > void _vfio_device_get_kvm_safe(struct vfio_device *device, struct kvm *kvm);
> > void vfio_device_put_kvm(struct vfio_device *device);
> > diff --git a/include/linux/vfio.h b/include/linux/vfio.h
> > index 8719ec2adbbb..1367605d617c 100644
> > --- a/include/linux/vfio.h
> > +++ b/include/linux/vfio.h
> > @@ -43,7 +43,11 @@ struct vfio_device {
> > */
> > const struct vfio_migration_ops *mig_ops;
> > const struct vfio_log_ops *log_ops;
> > +#if IS_ENABLED(CONFIG_VFIO_GROUP)
> > struct vfio_group *group;
> > + struct list_head group_next;
> > + struct list_head iommu_entry;
> > +#endif
> > struct vfio_device_set *dev_set;
> > struct list_head dev_set_list;
> > unsigned int migration_flags;
> > @@ -58,8 +62,6 @@ struct vfio_device {
> > refcount_t refcount; /* user count on registered device*/
> > unsigned int open_count;
> > struct completion comp;
> > - struct list_head group_next;
> > - struct list_head iommu_entry;
> > struct iommufd_access *iommufd_access;
> > void (*put_kvm)(struct kvm *kvm);
> > #if IS_ENABLED(CONFIG_IOMMUFD)
> > @@ -270,7 +272,14 @@ int vfio_mig_get_next_state(struct vfio_device *device,
> > /*
> > * External user API
> > */
> > +#if IS_ENABLED(CONFIG_VFIO_GROUP)
> > struct iommu_group *vfio_file_iommu_group(struct file *file);
> > +#else
> > +static inline struct iommu_group *vfio_file_iommu_group(struct file *file)
> > +{
> > + return NULL;
> > +}
> > +#endif
> > bool vfio_file_is_valid(struct file *file);
> > bool vfio_file_enforced_coherent(struct file *file);
> > void vfio_file_set_kvm(struct file *file, struct kvm *kvm);
next prev parent reply other threads:[~2023-03-30 13:08 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-27 9:40 [PATCH v8 00/24] Add vfio_device cdev for iommufd support Yi Liu
2023-03-27 9:40 ` [PATCH v8 01/24] vfio: Allocate per device file structure Yi Liu
2023-03-27 9:40 ` [PATCH v8 02/24] vfio: Refine vfio file kAPIs for KVM Yi Liu
2023-03-27 9:40 ` [PATCH v8 03/24] vfio: Remove vfio_file_is_group() Yi Liu
2023-03-30 23:50 ` Jason Gunthorpe
2023-03-27 9:40 ` [PATCH v8 04/24] vfio: Accept vfio device file in the KVM facing kAPI Yi Liu
2023-03-27 9:40 ` [PATCH v8 05/24] kvm/vfio: Rename kvm_vfio_group to prepare for accepting vfio device fd Yi Liu
2023-03-27 9:40 ` [PATCH v8 06/24] kvm/vfio: Accept vfio device file from userspace Yi Liu
2023-03-27 9:40 ` [PATCH v8 07/24] vfio: Pass struct vfio_device_file * to vfio_device_open/close() Yi Liu
2023-03-27 9:40 ` [PATCH v8 08/24] vfio: Block device access via device fd until device is opened Yi Liu
2023-03-28 21:33 ` Alex Williamson
2023-03-29 2:23 ` Liu, Yi L
2023-03-27 9:40 ` [PATCH v8 09/24] vfio: Add cdev_device_open_cnt to vfio_group Yi Liu
2023-03-27 9:40 ` [PATCH v8 10/24] vfio: Make vfio_device_open() single open for device cdev path Yi Liu
2023-03-30 23:52 ` Jason Gunthorpe
2023-03-27 9:40 ` [PATCH v8 11/24] vfio: Make vfio_device_first_open() to accept NULL iommufd for noiommu Yi Liu
2023-03-30 23:56 ` Jason Gunthorpe
2023-03-27 9:40 ` [PATCH v8 12/24] vfio-iommufd: Move noiommu support out of vfio_iommufd_bind() Yi Liu
2023-03-27 9:40 ` [PATCH v8 13/24] vfio-iommufd: Split bind/attach into two steps Yi Liu
2023-03-27 9:40 ` [PATCH v8 14/24] vfio: Record devid in vfio_device_file Yi Liu
2023-03-27 9:40 ` [PATCH v8 15/24] vfio-iommufd: Add detach_ioas support for physical VFIO devices Yi Liu
2023-03-27 9:40 ` [PATCH v8 16/24] iommufd/device: Add iommufd_access_detach() API Yi Liu
2023-03-28 2:23 ` Jon Pan-Doh
2023-03-28 15:54 ` Nicolin Chen
2023-03-29 2:24 ` Liu, Yi L
2023-03-27 9:40 ` [PATCH v8 17/24] vfio-iommufd: Add detach_ioas support for emulated VFIO devices Yi Liu
2023-03-27 9:40 ` [PATCH v8 18/24] vfio: Determine noiommu in vfio_device registration Yi Liu
2023-03-28 6:36 ` Tian, Kevin
2023-03-27 9:40 ` [PATCH v8 19/24] vfio: Name noiommu vfio_device with "noiommu-" prefix Yi Liu
2023-03-28 6:37 ` Tian, Kevin
2023-03-27 9:40 ` [PATCH v8 20/24] vfio: Add cdev for vfio_device Yi Liu
2023-03-29 19:57 ` Alex Williamson
2023-03-30 5:35 ` Liu, Yi L
2023-03-27 9:40 ` [PATCH v8 21/24] vfio: Add VFIO_DEVICE_BIND_IOMMUFD Yi Liu
2023-03-29 21:00 ` Alex Williamson
2023-03-29 23:22 ` Jason Gunthorpe
2023-03-30 12:52 ` Liu, Yi L
2023-03-30 12:59 ` Jason Gunthorpe
2023-03-30 7:09 ` Liu, Yi L
2023-03-30 11:52 ` Jason Gunthorpe
2023-03-30 12:53 ` Liu, Yi L
2023-03-27 9:40 ` [PATCH v8 22/24] vfio: Add VFIO_DEVICE_AT[DE]TACH_IOMMUFD_PT Yi Liu
2023-03-29 21:19 ` Alex Williamson
2023-03-30 13:02 ` Liu, Yi L
2023-03-27 9:40 ` [PATCH v8 23/24] vfio: Compile group optionally Yi Liu
2023-03-29 21:51 ` Alex Williamson
2023-03-30 13:06 ` Liu, Yi L [this message]
2023-03-27 9:40 ` [PATCH v8 24/24] docs: vfio: Add vfio device cdev description Yi Liu
2023-03-29 22:47 ` Alex Williamson
2023-03-29 22:57 ` Alex Williamson
2023-03-30 13:11 ` Liu, Yi L
2023-03-27 17:48 ` [PATCH v8 00/24] Add vfio_device cdev for iommufd support Nicolin Chen
2023-03-31 3:10 ` Jiang, Yanting
2023-03-31 5:01 ` Jiang, Yanting
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DS0PR11MB75294C1C51B2B91D6D89958AC38E9@DS0PR11MB7529.namprd11.prod.outlook.com \
--to=yi.l.liu@intel.com \
--cc=alex.williamson@redhat.com \
--cc=chao.p.peng@linux.intel.com \
--cc=cohuck@redhat.com \
--cc=eric.auger@redhat.com \
--cc=intel-gfx@lists.freedesktop.org \
--cc=intel-gvt-dev@lists.freedesktop.org \
--cc=jasowang@redhat.com \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=lulu@redhat.com \
--cc=mjrosato@linux.ibm.com \
--cc=nicolinc@nvidia.com \
--cc=peterx@redhat.com \
--cc=robin.murphy@arm.com \
--cc=shameerali.kolothum.thodi@huawei.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=terrence.xu@intel.com \
--cc=xudong.hao@intel.com \
--cc=yan.y.zhao@intel.com \
--cc=yanting.jiang@intel.com \
--cc=yi.y.sun@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).