From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jacob Pan <jacob.jun.pan@linux.intel.com>,
Joerg Roedel <joro@8bytes.org>,
Alex Williamson <alex.williamson@redhat.com>,
Lu Baolu <baolu.lu@linux.intel.com>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>,
David Woodhouse <dwmw2@infradead.org>,
Jean-Philippe Brucker <jean-philippe@linaro.com>
Cc: "Raj, Ashok" <ashok.raj@intel.com>, Jonathan Cameron <jic23@kernel.org>
Subject: RE: [PATCH 08/10] iommu/ioasid: Introduce notifier APIs
Date: Fri, 27 Mar 2020 10:03:26 +0000 [thread overview]
Message-ID: <AADFC41AFE54684AB9EE6CBC0274A5D19D7ED5D6@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <1585158931-1825-9-git-send-email-jacob.jun.pan@linux.intel.com>
> From: Jacob Pan <jacob.jun.pan@linux.intel.com>
> Sent: Thursday, March 26, 2020 1:55 AM
>
> IOASID users fit into the publisher-subscriber pattern, a system wide
> blocking notifier chain can be used to inform subscribers of state
> changes. Notifier mechanism also abstracts publisher from knowing the
> private context each subcriber may have.
>
> This patch adds APIs and a global notifier chain, a further optimization
> might be per set notifier for ioasid_set aware users.
>
> Usage example:
> KVM register notifier block such that it can keep its guest-host PASID
> translation table in sync with any IOASID updates.
>
> VFIO publish IOASID change by performing alloc/free, bind/unbind
> operations.
>
> IOMMU driver gets notified when IOASID is freed by VFIO or core mm code
> such that PASID context can be cleaned up.
above example looks mixed. You have KVM registers the notifier but
finally having IOMMU driver to get notified... 😊
>
> Signed-off-by: Liu Yi L <yi.l.liu@intel.com>
> Signed-off-by: Jacob Pan <jacob.jun.pan@linux.intel.com>
> ---
> drivers/iommu/ioasid.c | 61
> ++++++++++++++++++++++++++++++++++++++++++++++++++
> include/linux/ioasid.h | 40 +++++++++++++++++++++++++++++++++
> 2 files changed, 101 insertions(+)
>
> diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c
> index 8612fe6477dc..27dce2cb5af2 100644
> --- a/drivers/iommu/ioasid.c
> +++ b/drivers/iommu/ioasid.c
> @@ -11,6 +11,22 @@
> #include <linux/xarray.h>
>
> static DEFINE_XARRAY_ALLOC(ioasid_sets);
> +/*
> + * An IOASID could have multiple consumers. When a status change occurs,
> + * this notifier chain is used to keep them in sync. Each consumer of the
> + * IOASID service must register notifier block early to ensure no events
> + * are missed.
> + *
> + * This is a publisher-subscriber pattern where publisher can change the
> + * state of each IOASID, e.g. alloc/free, bind IOASID to a device and mm.
> + * On the other hand, subscribers gets notified for the state change and
> + * keep local states in sync.
> + *
> + * Currently, the notifier is global. A further optimization could be per
> + * IOASID set notifier chain.
> + */
> +static BLOCKING_NOTIFIER_HEAD(ioasid_chain);
> +
> /**
> * struct ioasid_set_data - Meta data about ioasid_set
> *
> @@ -408,6 +424,7 @@ static void ioasid_free_locked(ioasid_t ioasid)
> {
> struct ioasid_data *ioasid_data;
> struct ioasid_set_data *sdata;
> + struct ioasid_nb_args args;
>
> ioasid_data = xa_load(&active_allocator->xa, ioasid);
> if (!ioasid_data) {
> @@ -415,6 +432,13 @@ static void ioasid_free_locked(ioasid_t ioasid)
> return;
> }
>
> + args.id = ioasid;
> + args.sid = ioasid_data->sdata->sid;
> + args.pdata = ioasid_data->private;
> + args.set_token = ioasid_data->sdata->token;
> +
> + /* Notify all users that this IOASID is being freed */
> + blocking_notifier_call_chain(&ioasid_chain, IOASID_FREE, &args);
> active_allocator->ops->free(ioasid, active_allocator->ops->pdata);
> /* Custom allocator needs additional steps to free the xa element */
> if (active_allocator->flags & IOASID_ALLOCATOR_CUSTOM) {
> @@ -624,6 +648,43 @@ int ioasid_find_sid(ioasid_t ioasid)
> }
> EXPORT_SYMBOL_GPL(ioasid_find_sid);
>
> +int ioasid_add_notifier(struct notifier_block *nb)
> +{
> + return blocking_notifier_chain_register(&ioasid_chain, nb);
> +}
> +EXPORT_SYMBOL_GPL(ioasid_add_notifier);
> +
> +void ioasid_remove_notifier(struct notifier_block *nb)
> +{
> + blocking_notifier_chain_unregister(&ioasid_chain, nb);
> +}
> +EXPORT_SYMBOL_GPL(ioasid_remove_notifier);
register/unregister
> +
> +int ioasid_notify(ioasid_t ioasid, enum ioasid_notify_val cmd)
add a comment on when this function should be used?
> +{
> + struct ioasid_data *ioasid_data;
> + struct ioasid_nb_args args;
> + int ret = 0;
> +
> + mutex_lock(&ioasid_allocator_lock);
> + ioasid_data = xa_load(&active_allocator->xa, ioasid);
> + if (!ioasid_data) {
> + pr_err("Trying to free unknown IOASID %u\n", ioasid);
why is it fixed to 'free'?
> + mutex_unlock(&ioasid_allocator_lock);
> + return -EINVAL;
> + }
> +
> + args.id = ioasid;
> + args.sid = ioasid_data->sdata->sid;
> + args.pdata = ioasid_data->private;
why no token info as did in ioasid_free?
> +
> + ret = blocking_notifier_call_chain(&ioasid_chain, cmd, &args);
> + mutex_unlock(&ioasid_allocator_lock);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(ioasid_notify);
> +
> MODULE_AUTHOR("Jean-Philippe Brucker <jean-
> philippe.brucker@arm.com>");
> MODULE_AUTHOR("Jacob Pan <jacob.jun.pan@linux.intel.com>");
> MODULE_DESCRIPTION("IO Address Space ID (IOASID) allocator");
> diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h
> index e19c0ad93bd7..32d032913828 100644
> --- a/include/linux/ioasid.h
> +++ b/include/linux/ioasid.h
> @@ -4,6 +4,7 @@
>
> #include <linux/types.h>
> #include <linux/errno.h>
> +#include <linux/notifier.h>
>
> #define INVALID_IOASID ((ioasid_t)-1)
> #define INVALID_IOASID_SET (-1)
> @@ -30,6 +31,27 @@ struct ioasid_allocator_ops {
> void *pdata;
> };
>
> +/* Notification data when IOASID status changed */
> +enum ioasid_notify_val {
> + IOASID_ALLOC = 1,
> + IOASID_FREE,
> + IOASID_BIND,
> + IOASID_UNBIND,
> +};
Curious why IOASID_ALLOC is not notified aumatically within ioasid_alloc
similar to ioasid_free, while leaving to the publisher? BIND/UNBIND is
a publisher thing but a bit strange to see ALLOC/FREE with different policy here.
> +
> +/**
> + * struct ioasid_nb_args - Argument provided by IOASID core when notifier
> + * is called.
> + * @id: the IOASID being notified
> + * @sid: the IOASID set @id belongs to
> + * @pdata: the private data attached to the IOASID
> + */
> +struct ioasid_nb_args {
> + ioasid_t id;
> + int sid;
> + struct ioasid_set *set_token;
> + void *pdata;
> +};
> /* Shared IOASID set for reserved for host system use */
> extern int system_ioasid_sid;
>
> @@ -43,11 +65,15 @@ void *ioasid_find(int sid, ioasid_t ioasid, bool
> (*getter)(void *));
> int ioasid_register_allocator(struct ioasid_allocator_ops *allocator);
> void ioasid_unregister_allocator(struct ioasid_allocator_ops *allocator);
> int ioasid_attach_data(ioasid_t ioasid, void *data);
> +int ioasid_add_notifier(struct notifier_block *nb);
> +void ioasid_remove_notifier(struct notifier_block *nb);
> void ioasid_install_capacity(ioasid_t total);
> int ioasid_alloc_system_set(int quota);
> int ioasid_alloc_set(struct ioasid_set *token, ioasid_t quota, int *sid);
> void ioasid_free_set(int sid, bool destroy_set);
> int ioasid_find_sid(ioasid_t ioasid);
> +int ioasid_notify(ioasid_t id, enum ioasid_notify_val cmd);
> +
> #else /* !CONFIG_IOASID */
> static inline ioasid_t ioasid_alloc(int sid, ioasid_t min,
> ioasid_t max, void *private)
> @@ -73,6 +99,20 @@ static inline void *ioasid_find(int sid, ioasid_t ioasid,
> bool (*getter)(void *)
> return NULL;
> }
>
> +static inline int ioasid_add_notifier(struct notifier_block *nb)
> +{
> + return -ENOTSUPP;
> +}
> +
> +static inline void ioasid_remove_notifier(struct notifier_block *nb)
> +{
> +}
> +
> +int ioasid_notify(ioasid_t ioasid, enum ioasid_notify_val cmd)
> +{
> + return -ENOTSUPP;
> +}
> +
> static inline int ioasid_register_allocator(struct ioasid_allocator_ops
> *allocator)
> {
> return -ENOTSUPP;
> --
> 2.7.4
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2020-03-27 10:03 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-25 17:55 [PATCH 00/10] IOASID extensions for guest SVA Jacob Pan
2020-03-25 17:55 ` [PATCH 01/10] iommu/ioasid: Introduce system-wide capacity Jacob Pan
2020-03-27 8:07 ` Tian, Kevin
2020-03-27 16:08 ` Jacob Pan
2020-04-01 13:45 ` Jean-Philippe Brucker
2020-04-01 22:50 ` Jacob Pan
2020-03-25 17:55 ` [PATCH 02/10] iommu/vt-d: Set IOASID capacity when SVM is enabled Jacob Pan
2020-03-27 8:08 ` Tian, Kevin
2020-03-25 17:55 ` [PATCH 03/10] iommu/ioasid: Introduce per set allocation APIs Jacob Pan
2020-03-26 2:12 ` Lu Baolu
2020-03-26 21:30 ` Jacob Pan
2020-03-27 8:38 ` Tian, Kevin
2020-03-27 16:59 ` Jacob Pan
2020-03-28 6:32 ` Tian, Kevin
2020-04-01 13:47 ` Jean-Philippe Brucker
2020-04-06 20:02 ` Jacob Pan
2020-04-07 11:01 ` Jean-Philippe Brucker
2020-04-21 21:51 ` Jacob Pan
2020-03-25 17:55 ` [PATCH 04/10] iommu/ioasid: Rename ioasid_set_data to avoid confusion with ioasid_set Jacob Pan
2020-03-27 9:35 ` Tian, Kevin
2020-03-25 17:55 ` [PATCH 05/10] iommu/ioasid: Create an IOASID set for host SVA use Jacob Pan
2020-03-27 9:41 ` Tian, Kevin
2020-03-27 17:28 ` Jacob Pan
2020-03-28 6:33 ` Tian, Kevin
2020-04-01 13:53 ` Jean-Philippe Brucker
2020-04-06 15:33 ` Jacob Pan
2020-04-07 11:01 ` Jean-Philippe Brucker
2020-04-13 22:06 ` Jacob Pan
2020-04-15 15:10 ` Jean-Philippe Brucker
2020-03-25 17:55 ` [PATCH 06/10] iommu/ioasid: Convert to set aware allocations Jacob Pan
2020-03-27 9:54 ` Tian, Kevin
2020-03-27 17:41 ` Jacob Pan
2020-03-28 6:40 ` Tian, Kevin
2020-04-06 20:07 ` Jacob Pan
2020-04-01 13:55 ` Jean-Philippe Brucker
2020-04-01 22:45 ` Jacob Pan
2020-03-25 17:55 ` [PATCH 07/10] iommu/ioasid: Use mutex instead of spinlock Jacob Pan
2020-03-27 9:55 ` Tian, Kevin
2020-04-01 13:58 ` Jean-Philippe Brucker
2020-03-25 17:55 ` [PATCH 08/10] iommu/ioasid: Introduce notifier APIs Jacob Pan
2020-03-27 10:03 ` Tian, Kevin [this message]
2020-03-27 18:36 ` Jacob Pan
2020-03-28 6:43 ` Tian, Kevin
2020-03-31 15:13 ` Jacob Pan
2020-04-01 14:00 ` Jean-Philippe Brucker
2020-04-10 15:43 ` Jacob Pan
2020-03-25 17:55 ` [PATCH 09/10] iommu/ioasid: Support ioasid_set quota adjustment Jacob Pan
2020-03-27 10:09 ` Tian, Kevin
2020-03-27 23:30 ` Jacob Pan
2020-03-28 6:44 ` Tian, Kevin
2020-03-25 17:55 ` [PATCH 10/10] iommu/vt-d: Register PASID notifier for status change Jacob Pan
2020-03-27 10:22 ` Tian, Kevin
2020-03-27 23:47 ` Jacob Pan
2020-04-01 14:03 ` [PATCH 00/10] IOASID extensions for guest SVA Jean-Philippe Brucker
2020-04-01 23:38 ` Jacob Pan
2020-04-02 12:26 ` Jean-Philippe Brucker
2020-04-02 16:09 ` Jacob Pan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AADFC41AFE54684AB9EE6CBC0274A5D19D7ED5D6@SHSMSX104.ccr.corp.intel.com \
--to=kevin.tian@intel.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@linux.intel.com \
--cc=dwmw2@infradead.org \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jean-philippe@linaro.com \
--cc=jic23@kernel.org \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).