From: Alex Elder <elder@linaro.org>
To: Elliot Berman <quic_eberman@quicinc.com>,
Srinivas Kandagatla <srinivas.kandagatla@linaro.org>,
Prakruthi Deepak Heragu <quic_pheragu@quicinc.com>
Cc: Murali Nalajala <quic_mnalajal@quicinc.com>,
Trilok Soni <quic_tsoni@quicinc.com>,
Srivatsa Vaddagiri <quic_svaddagi@quicinc.com>,
Carl van Schaik <quic_cvanscha@quicinc.com>,
Dmitry Baryshkov <dmitry.baryshkov@linaro.org>,
Bjorn Andersson <andersson@kernel.org>,
Konrad Dybcio <konrad.dybcio@linaro.org>,
Arnd Bergmann <arnd@arndb.de>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Rob Herring <robh+dt@kernel.org>,
Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
Jonathan Corbet <corbet@lwn.net>,
Bagas Sanjaya <bagasdotme@gmail.com>,
Will Deacon <will@kernel.org>, Andy Gross <agross@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Jassi Brar <jassisinghbrar@gmail.com>,
linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v11 20/26] virt: gunyah: Add resource tickets
Date: Fri, 31 Mar 2023 09:27:01 -0500 [thread overview]
Message-ID: <dd56d297-2483-5b7d-4c63-fd103070af5c@linaro.org> (raw)
In-Reply-To: <20230304010632.2127470-21-quic_eberman@quicinc.com>
On 3/3/23 7:06 PM, Elliot Berman wrote:
> Some VM functions need to acquire Gunyah resources. For instance, Gunyah
> vCPUs are exposed to the host as a resource. The Gunyah vCPU function
> will register a resource ticket and be able to interact with the
> hypervisor once the resource ticket is filled.
>
> Resource tickets are the mechanism for functions to acquire ownership of
> Gunyah resources. Gunyah functions can be created before the VM's
> resources are created and made available to Linux. A resource ticket
> identifies a type of resource and a label of a resource which the ticket
> holder is interested in.
>
> Resources are created by Gunyah as configured in the VM's devicetree
> configuration. Gunyah doesn't process the label and that makes it
> possible for userspace to create multiple resources with the same label.
> Resource ticket owners need to be prepared for populate to be called
> multiple times if userspace created multiple resources with the same
> label.
>
> Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
One possibly substantive suggestion here, plus a couple suggestions
to add or revise comments.
-Alex
> ---
> drivers/virt/gunyah/vm_mgr.c | 112 +++++++++++++++++++++++++++++++++-
> drivers/virt/gunyah/vm_mgr.h | 4 ++
> include/linux/gunyah_vm_mgr.h | 14 +++++
> 3 files changed, 129 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/virt/gunyah/vm_mgr.c b/drivers/virt/gunyah/vm_mgr.c
> index 88db011395ec..0269bcdaf692 100644
> --- a/drivers/virt/gunyah/vm_mgr.c
> +++ b/drivers/virt/gunyah/vm_mgr.c
> @@ -165,6 +165,74 @@ static long gh_vm_rm_function(struct gh_vm *ghvm, struct gh_fn_desc *f)
> return r;
> }
>
> +int gh_vm_add_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket)
> +{
> + struct gh_vm_resource_ticket *iter;
> + struct gh_resource *ghrsc;
> + int ret = 0;
> +
> + mutex_lock(&ghvm->resources_lock);
> + list_for_each_entry(iter, &ghvm->resource_tickets, list) {
> + if (iter->resource_type == ticket->resource_type && iter->label == ticket->label) {
> + ret = -EEXIST;
> + goto out;
> + }
> + }
> +
> + if (!try_module_get(ticket->owner)) {
> + ret = -ENODEV;
> + goto out;
> + }
> +
> + list_add(&ticket->list, &ghvm->resource_tickets);
> + INIT_LIST_HEAD(&ticket->resources);
> +
> + list_for_each_entry(ghrsc, &ghvm->resources, list) {
> + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) {
> + if (!ticket->populate(ticket, ghrsc))
> + list_move(&ghrsc->list, &ticket->resources);
> + }
> + }
> +out:
> + mutex_unlock(&ghvm->resources_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(gh_vm_add_resource_ticket);
> +
> +void gh_vm_remove_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket)
> +{
> + struct gh_resource *ghrsc, *iter;
> +
> + mutex_lock(&ghvm->resources_lock);
> + list_for_each_entry_safe(ghrsc, iter, &ticket->resources, list) {
> + ticket->unpopulate(ticket, ghrsc);
> + list_move(&ghrsc->list, &ghvm->resources);
> + }
> +
> + module_put(ticket->owner);
> + list_del(&ticket->list);
> + mutex_unlock(&ghvm->resources_lock);
> +}
> +EXPORT_SYMBOL_GPL(gh_vm_remove_resource_ticket);
> +
> +static void gh_vm_add_resource(struct gh_vm *ghvm, struct gh_resource *ghrsc)
> +{
> + struct gh_vm_resource_ticket *ticket;
> +
> + mutex_lock(&ghvm->resources_lock);
> + list_for_each_entry(ticket, &ghvm->resource_tickets, list) {
> + if (ghrsc->type == ticket->resource_type && ghrsc->rm_label == ticket->label) {
> + if (!ticket->populate(ticket, ghrsc)) {
> + list_add(&ghrsc->list, &ticket->resources);
> + goto found;
> + }
I think the "goto found" belongs here, unconditionally.
You disallow adding more than one ticket of a given type
with the same label. So you will never match another
ticket once you've matched this one.
The populate function generally shouldn't fail. I think
it only fails if you find a duplicate, and again, I think
you prevent that from happening. (But if it does, you
silently ignore it...)
> + }
> + }
> + list_add(&ghrsc->list, &ghvm->resources);
> +found:
> + mutex_unlock(&ghvm->resources_lock);
> +}
> +
> static int gh_vm_rm_notification_status(struct gh_vm *ghvm, void *data)
> {
> struct gh_rm_vm_status_payload *payload = data;
> @@ -230,6 +298,8 @@ static void gh_vm_free(struct work_struct *work)
> {
> struct gh_vm *ghvm = container_of(work, struct gh_vm, free_work);
> struct gh_vm_function_instance *inst, *iiter;
> + struct gh_vm_resource_ticket *ticket, *titer;
> + struct gh_resource *ghrsc, *riter;
> struct gh_vm_mem *mapping, *tmp;
> int ret;
>
> @@ -246,6 +316,25 @@ static void gh_vm_free(struct work_struct *work)
> }
> mutex_unlock(&ghvm->fn_lock);
>
> + mutex_lock(&ghvm->resources_lock);
> + if (!list_empty(&ghvm->resource_tickets)) {
> + dev_warn(ghvm->parent, "Dangling resource tickets:\n");
> + list_for_each_entry_safe(ticket, titer, &ghvm->resource_tickets, list) {
> + dev_warn(ghvm->parent, " %pS\n", ticket->populate);
> + gh_vm_remove_resource_ticket(ghvm, ticket);
> + }
> + }
> +
> + list_for_each_entry_safe(ghrsc, riter, &ghvm->resources, list) {
> + gh_rm_free_resource(ghrsc);
> + }
> + mutex_unlock(&ghvm->resources_lock);
> +
> + ret = gh_rm_vm_reset(ghvm->rm, ghvm->vmid);
> + if (ret)
> + dev_err(ghvm->parent, "Failed to reset the vm: %d\n", ret);
> + wait_event(ghvm->vm_status_wait, ghvm->vm_status == GH_RM_VM_STATUS_RESET);
> +
> mutex_lock(&ghvm->mm_lock);
> list_for_each_entry_safe(mapping, tmp, &ghvm->memory_mappings, list) {
> gh_vm_mem_reclaim(ghvm, mapping);
> @@ -329,6 +418,9 @@ static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm)
> init_rwsem(&ghvm->status_lock);
> INIT_WORK(&ghvm->free_work, gh_vm_free);
> kref_init(&ghvm->kref);
> + mutex_init(&ghvm->resources_lock);
> + INIT_LIST_HEAD(&ghvm->resources);
> + INIT_LIST_HEAD(&ghvm->resource_tickets);
> INIT_LIST_HEAD(&ghvm->functions);
> ghvm->vm_status = GH_RM_VM_STATUS_LOAD;
>
> @@ -338,9 +430,11 @@ static __must_check struct gh_vm *gh_vm_alloc(struct gh_rm *rm)
> static int gh_vm_start(struct gh_vm *ghvm)
> {
> struct gh_vm_mem *mapping;
> + struct gh_rm_hyp_resources *resources;
> + struct gh_resource *ghrsc;
> u64 dtb_offset;
> u32 mem_handle;
> - int ret;
> + int ret, i, n;
>
> down_write(&ghvm->status_lock);
> if (ghvm->vm_status != GH_RM_VM_STATUS_LOAD) {
> @@ -394,6 +488,22 @@ static int gh_vm_start(struct gh_vm *ghvm)
> goto err;
> }
>
> + ret = gh_rm_get_hyp_resources(ghvm->rm, ghvm->vmid, &resources);
> + if (ret) {
> + dev_warn(ghvm->parent, "Failed to get hypervisor resources for VM: %d\n", ret);
> + goto err;
> + }
> +
> + for (i = 0, n = le32_to_cpu(resources->n_entries); i < n; i++) {
> + ghrsc = gh_rm_alloc_resource(ghvm->rm, &resources->entries[i]);
> + if (!ghrsc) {
> + ret = -ENOMEM;
> + goto err;
> + }
> +
> + gh_vm_add_resource(ghvm, ghrsc);
> + }
> +
> ret = gh_rm_vm_start(ghvm->rm, ghvm->vmid);
> if (ret) {
> dev_warn(ghvm->parent, "Failed to start VM: %d\n", ret);
> diff --git a/drivers/virt/gunyah/vm_mgr.h b/drivers/virt/gunyah/vm_mgr.h
> index 7bd271bad721..18d0e1effd25 100644
> --- a/drivers/virt/gunyah/vm_mgr.h
> +++ b/drivers/virt/gunyah/vm_mgr.h
> @@ -7,6 +7,7 @@
> #define _GH_PRIV_VM_MGR_H
>
> #include <linux/gunyah_rsc_mgr.h>
> +#include <linux/gunyah_vm_mgr.h>
> #include <linux/list.h>
> #include <linux/kref.h>
> #include <linux/miscdevice.h>
> @@ -51,6 +52,9 @@ struct gh_vm {
> struct list_head memory_mappings;
> struct mutex fn_lock;
> struct list_head functions;
> + struct mutex resources_lock;
> + struct list_head resources;
> + struct list_head resource_tickets;
> };
>
> int gh_vm_mem_alloc(struct gh_vm *ghvm, struct gh_userspace_memory_region *region);
> diff --git a/include/linux/gunyah_vm_mgr.h b/include/linux/gunyah_vm_mgr.h
> index 3825c951790a..01b1761b5923 100644
> --- a/include/linux/gunyah_vm_mgr.h
> +++ b/include/linux/gunyah_vm_mgr.h
> @@ -70,4 +70,18 @@ void gh_vm_function_unregister(struct gh_vm_function *f);
> DECLARE_GH_VM_FUNCTION(_name, _type, _bind, _unbind); \
> module_gh_vm_function(_name)
>
> +struct gh_vm_resource_ticket {
> + struct list_head list; /* for gh_vm's resources list */
Maybe "resource lists" above (it's for the resources list and
resource_tickets list).
> + struct list_head resources; /* for gh_resources's list */
Maybe: /* resources associated with this ticket */
> + enum gh_resource_type resource_type;
> + u32 label;
> +
> + struct module *owner;
> + int (*populate)(struct gh_vm_resource_ticket *ticket, struct gh_resource *ghrsc);
> + void (*unpopulate)(struct gh_vm_resource_ticket *ticket, struct gh_resource *ghrsc);
> +};
> +
> +int gh_vm_add_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket);
> +void gh_vm_remove_resource_ticket(struct gh_vm *ghvm, struct gh_vm_resource_ticket *ticket);
> +
> #endif
next prev parent reply other threads:[~2023-03-31 14:27 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-03-04 1:06 [PATCH v11 00/26] Drivers for gunyah hypervisor Elliot Berman
2023-03-04 1:06 ` [PATCH v11 01/26] docs: gunyah: Introduce Gunyah Hypervisor Elliot Berman
2023-03-04 1:06 ` [PATCH v11 02/26] dt-bindings: Add binding for gunyah hypervisor Elliot Berman
2023-03-04 1:06 ` [PATCH v11 03/26] gunyah: Common types and error codes for Gunyah hypercalls Elliot Berman
2023-03-21 14:23 ` Srinivas Kandagatla
2023-03-31 14:24 ` Alex Elder
2023-04-03 19:44 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 04/26] virt: gunyah: Add hypercalls to identify Gunyah Elliot Berman
2023-03-21 14:22 ` Srinivas Kandagatla
2023-03-31 14:24 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 05/26] virt: gunyah: Identify hypervisor version Elliot Berman
2023-03-21 15:48 ` Srinivas Kandagatla
2023-03-31 14:24 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 06/26] virt: gunyah: msgq: Add hypercalls to send and receive messages Elliot Berman
2023-03-21 15:49 ` Srinivas Kandagatla
2023-03-31 14:25 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 07/26] mailbox: Add Gunyah message queue mailbox Elliot Berman
2023-03-21 14:22 ` Srinivas Kandagatla
2023-03-31 14:25 ` Alex Elder
2023-04-03 20:15 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 08/26] gunyah: rsc_mgr: Add resource manager RPC core Elliot Berman
2023-03-31 14:25 ` Alex Elder
2023-04-03 20:34 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 09/26] gunyah: rsc_mgr: Add VM lifecycle RPC Elliot Berman
2023-03-31 14:25 ` Alex Elder
2023-04-03 21:09 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 10/26] gunyah: vm_mgr: Introduce basic VM Manager Elliot Berman
2023-03-21 14:23 ` Srinivas Kandagatla
2023-03-31 14:25 ` Alex Elder
2023-04-11 20:48 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 11/26] gunyah: rsc_mgr: Add RPC for sharing memory Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 12/26] gunyah: vm_mgr: Add/remove user memory regions Elliot Berman
2023-03-24 18:37 ` Will Deacon
2023-04-11 20:34 ` Elliot Berman
2023-04-11 21:19 ` Will Deacon
2023-04-12 20:48 ` Elliot Berman
2023-04-13 9:54 ` Will Deacon
2023-03-31 14:26 ` Alex Elder
2023-04-11 21:04 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 13/26] gunyah: vm_mgr: Add ioctls to support basic non-proxy VM boot Elliot Berman
2023-03-21 14:24 ` Srinivas Kandagatla
2023-04-11 21:07 ` Elliot Berman
2023-04-11 21:09 ` Alex Elder
2023-03-31 14:26 ` Alex Elder
2023-04-11 21:16 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 14/26] samples: Add sample userspace Gunyah VM Manager Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 15/26] gunyah: rsc_mgr: Add platform ops on mem_lend/mem_reclaim Elliot Berman
2023-03-21 14:23 ` Srinivas Kandagatla
2023-03-22 19:17 ` Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 16/26] firmware: qcom_scm: Register Gunyah platform ops Elliot Berman
2023-03-21 14:24 ` Srinivas Kandagatla
2023-03-21 18:40 ` Elliot Berman
2023-03-21 20:19 ` Srinivas Kandagatla
2023-03-04 1:06 ` [PATCH v11 17/26] docs: gunyah: Document Gunyah VM Manager Elliot Berman
2023-03-04 1:06 ` [PATCH v11 18/26] virt: gunyah: Translate gh_rm_hyp_resource into gunyah_resource Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-04-18 0:25 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 19/26] gunyah: vm_mgr: Add framework to add VM Functions Elliot Berman
2023-03-31 14:26 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 20/26] virt: gunyah: Add resource tickets Elliot Berman
2023-03-31 14:27 ` Alex Elder [this message]
2023-04-17 22:57 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 21/26] virt: gunyah: Add IO handlers Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 22/26] virt: gunyah: Add proxy-scheduled vCPUs Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-04-17 22:41 ` Elliot Berman
2023-04-18 12:46 ` Alex Elder
2023-04-18 17:18 ` Elliot Berman
2023-04-18 17:31 ` Alex Elder
2023-04-18 18:35 ` Elliot Berman
2023-03-04 1:06 ` [PATCH v11 23/26] virt: gunyah: Add hypercalls for sending doorbell Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 24/26] virt: gunyah: Add irqfd interface Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-04-17 22:55 ` Elliot Berman
2023-04-18 12:55 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 25/26] virt: gunyah: Add ioeventfd Elliot Berman
2023-03-31 14:27 ` Alex Elder
2023-03-04 1:06 ` [PATCH v11 26/26] MAINTAINERS: Add Gunyah hypervisor drivers section Elliot Berman
2023-03-31 14:24 ` [PATCH v11 00/26] Drivers for gunyah hypervisor Alex Elder
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=dd56d297-2483-5b7d-4c63-fd103070af5c@linaro.org \
--to=elder@linaro.org \
--cc=agross@kernel.org \
--cc=andersson@kernel.org \
--cc=arnd@arndb.de \
--cc=bagasdotme@gmail.com \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=devicetree@vger.kernel.org \
--cc=dmitry.baryshkov@linaro.org \
--cc=gregkh@linuxfoundation.org \
--cc=jassisinghbrar@gmail.com \
--cc=konrad.dybcio@linaro.org \
--cc=krzysztof.kozlowski+dt@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=quic_cvanscha@quicinc.com \
--cc=quic_eberman@quicinc.com \
--cc=quic_mnalajal@quicinc.com \
--cc=quic_pheragu@quicinc.com \
--cc=quic_svaddagi@quicinc.com \
--cc=quic_tsoni@quicinc.com \
--cc=robh+dt@kernel.org \
--cc=srinivas.kandagatla@linaro.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).