From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6ABC2C636CD for ; Tue, 7 Feb 2023 14:32:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zqVPKy6tkr9e4LXQUlFsIHJD/y05c/AapgWrCHLtvRM=; b=cyLjtUVNO6ACQk nuN5pyOTsW5UYVle+kOVZAwNCHo5nw7I6myanEyL4VuqnBHowx77hxogKx85VG630nYQ/AzclFo/r 31J8uxrLdirqJmUh+s3Yq5htwSKjdMm6MjfPAtg1+BYzxGBbXVMGCiYN/k3mzsLrauhZicd3pizNb X02SWegSzbImzzV1QLe4QJFXEJcHYQzFPNWAJp9o+RCIYMQ6QWc0otZ+h4uZdYleEph2sUHIcihu0 0F3vO7cBYQLE+2e4kpwX/soxPIU7GYqPAIQcgyy0vFovO+SUBET1DSp9rOGd72t0oIKWDULx4Q+YU TwnSMgk3afiRRXnPgGlw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pPOzo-00CVwC-K6; Tue, 07 Feb 2023 14:30:56 +0000 Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pPOzj-00CVuM-2h for linux-arm-kernel@lists.infradead.org; Tue, 07 Feb 2023 14:30:53 +0000 Received: by mail-wm1-x32f.google.com with SMTP id n28-20020a05600c3b9c00b003ddca7a2bcbso11609674wms.3 for ; Tue, 07 Feb 2023 06:30:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=YOpPEC2Awvgh2+il+1cdOelaxS0kr6969CW5gfZdK9s=; b=mvmIayQ6kbXX2V1Crl0tHuqqPSYFLB/Jc8zYwglHDcxyt934fkl+bWG6tLuCpQ1g5M Jb9KOLyEms2XLQ3wir9yiiaWxp2b+zzlB/xYywUOiCIuPwI6UKMcX3PMOvoSNkWe8fkN Nw8Ws+PMzukriLOCXNBIBORq3847BjCpUi1rIvGpmgEUWjDNwdRHURkSNd1oA39bkNTP QaLerpe41EPHHKUA1OnajCdSskROLxC2RJ9KgPNEV15H9cLQdIO+8S7kqxWcA1KA7BRx r6PN8jB5gMHWV7Ltvt/T0Tm+PHOqMBpmj3WxvPkvSnBGvvC/3B63h/jtFoGsKnOVsqij tYLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YOpPEC2Awvgh2+il+1cdOelaxS0kr6969CW5gfZdK9s=; b=uxLIr9Z4L2D4sOeGHBdI1fxbJcG9g6uSVXjOHgeZ+olYhihA41YP1F+G0a4m3p/ed9 YPS6lLNVtO3SzveR4rfIDX/zNB4Mt+62nHngTy0Nh+gCKzQ5Lw0SaOIQ7aT6HvbXOmLh Vl4ZSulD9CBCw2lTfdmOBGBRwE+uZmwVXCmIQxWYT3DfolDAXLH14hkbdHaP3k81x/Jp ajGAjhodtAgdDuMKGV9XMbaDRNKQWMh7tfAtfBt57K13ZSGNSiia8HMCeaUAmiut70Ih plHxfz1j7LnvI8GRNXpw9vEMrUh9LVIrrU5cgXeqE6KjzVCyalyAFJYZa4/re5cFKgMz L63g== X-Gm-Message-State: AO0yUKWvrg043heK6rXcaEpbkG88ATK/d3iTdXYx51g6RABrAtfSjecx K7QY9goy0HqLPWYJUY/ijBAsWg== X-Google-Smtp-Source: AK7set8jcs1Fi07SodeYKm3KsVNy19Z8Ks0GBdzOiILFELxjwbjn+1rpRCWpD+Y0NUJHjqjrlI6mSA== X-Received: by 2002:a05:600c:920:b0:3df:d817:df98 with SMTP id m32-20020a05600c092000b003dfd817df98mr3295961wmp.10.1675780248891; Tue, 07 Feb 2023 06:30:48 -0800 (PST) Received: from [192.168.1.195] ([5.133.47.210]) by smtp.googlemail.com with ESMTPSA id j14-20020a05600c190e00b003daf681d05dsm15344363wmq.26.2023.02.07.06.30.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 07 Feb 2023 06:30:47 -0800 (PST) Message-ID: <724a79fe-304d-f8db-c66c-9fdbfac873c8@linaro.org> Date: Tue, 7 Feb 2023 14:30:46 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.4.2 Subject: Re: [PATCH v9 26/27] virt: gunyah: Add irqfd interface Content-Language: en-US To: Elliot Berman , Bjorn Andersson , Alex Elder , Murali Nalajala , Jonathan Corbet Cc: Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Prakruthi Deepak Heragu , Dmitry Baryshkov , Arnd Bergmann , Greg Kroah-Hartman , Rob Herring , Krzysztof Kozlowski , Bagas Sanjaya , Catalin Marinas , Will Deacon , Marc Zyngier , Jassi Brar , Sudeep Holla , linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org References: <20230120224627.4053418-1-quic_eberman@quicinc.com> <20230120224627.4053418-27-quic_eberman@quicinc.com> From: Srinivas Kandagatla In-Reply-To: <20230120224627.4053418-27-quic_eberman@quicinc.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230207_063051_161631_84C0D75A X-CRM114-Status: GOOD ( 37.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 20/01/2023 22:46, Elliot Berman wrote: > Enable support for creating irqfds which can raise an interrupt on a > Gunyah virtual machine. irqfds are exposed to userspace as a Gunyah VM > function with the name "irqfd". If the VM devicetree is not configured > to create a doorbell with the corresponding label, userspace will still > be able to assert the eventfd but no interrupt will be raised on the > guest. > > Co-developed-by: Prakruthi Deepak Heragu > Signed-off-by: Prakruthi Deepak Heragu > Signed-off-by: Elliot Berman > --- > Documentation/virt/gunyah/vm-manager.rst | 22 +++ > drivers/virt/gunyah/Kconfig | 9 ++ > drivers/virt/gunyah/Makefile | 1 + > drivers/virt/gunyah/gunyah_irqfd.c | 166 +++++++++++++++++++++++ > include/linux/gunyah.h | 5 + > include/uapi/linux/gunyah.h | 11 +- > 6 files changed, 213 insertions(+), 1 deletion(-) > create mode 100644 drivers/virt/gunyah/gunyah_irqfd.c > > diff --git a/Documentation/virt/gunyah/vm-manager.rst b/Documentation/virt/gunyah/vm-manager.rst > index d11267d59802..b6cf8db826b8 100644 > --- a/Documentation/virt/gunyah/vm-manager.rst > +++ b/Documentation/virt/gunyah/vm-manager.rst > @@ -142,3 +142,25 @@ The vcpu type will register with the VM Manager to expect to control > vCPU number `vcpu_id`. It returns a file descriptor allowing interaction with > the vCPU. See the Gunyah vCPU API description sections for interacting with > the Gunyah vCPU file descriptors. > + > +Type: "irqfd" > +^^^^^^^^^^^^^ > + > +:: > + > + struct gh_fn_irqfd_arg { > + __u32 fd; > + __u32 label; > + #define GH_IRQFD_LEVEL (1UL << 0) > + #define GH_IRQFD_DEASSIGN (1UL << 1) > + __u32 flags; > + }; > + > +Allows setting an eventfd to directly trigger a guest interrupt. > +irqfd.fd specifies the file descriptor to use as the eventfd. > +irqfd.label corresponds to the doorbell label used in the guest VM's devicetree. > +The irqfd is removed using the GH_IRQFD_DEASSIGN flag and specifying at least > +the irqfd.label. > + > +GH_IRQFD_LEVEL configures the corresponding doorbell to behave like a level > +triggered interrupt. > diff --git a/drivers/virt/gunyah/Kconfig b/drivers/virt/gunyah/Kconfig > index 4c1c6110b50e..2cde24d429d1 100644 > --- a/drivers/virt/gunyah/Kconfig > +++ b/drivers/virt/gunyah/Kconfig > @@ -26,3 +26,12 @@ config GUNYAH_VCPU > VMMs can also handle stage 2 faults of the vCPUs. > > Say Y/M here if unsure and you want to support Gunyah VMMs. > + > +config GUNYAH_IRQFD > + tristate "Gunyah irqfd interface" > + depends on GUNYAH > + help > + Enable kernel support for creating irqfds which can raise an interrupt > + on Gunyah virtual machine. > + > + Say Y/M here if unsure and you want to support Gunyah VMMs. > diff --git a/drivers/virt/gunyah/Makefile b/drivers/virt/gunyah/Makefile > index 2d1b604a7b03..6cf756bfa3c2 100644 > --- a/drivers/virt/gunyah/Makefile > +++ b/drivers/virt/gunyah/Makefile > @@ -7,3 +7,4 @@ gunyah_rsc_mgr-y += rsc_mgr.o rsc_mgr_rpc.o vm_mgr.o vm_mgr_mm.o > obj-$(CONFIG_GUNYAH) += gunyah_rsc_mgr.o > > obj-$(CONFIG_GUNYAH_VCPU) += gunyah_vcpu.o > +obj-$(CONFIG_GUNYAH_IRQFD) += gunyah_irqfd.o > diff --git a/drivers/virt/gunyah/gunyah_irqfd.c b/drivers/virt/gunyah/gunyah_irqfd.c > new file mode 100644 > index 000000000000..a3be9ca2377a > --- /dev/null > +++ b/drivers/virt/gunyah/gunyah_irqfd.c > @@ -0,0 +1,166 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. > + */ > + > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > +#include > + > +#include > + > +struct gunyah_irqfd { > + struct gunyah_resource *ghrsc; > + struct gunyah_vm_resource_ticket ticket; > + struct gunyah_vm_function *f; > + > + struct kref kref; > + bool level; > + > + struct eventfd_ctx *ctx; > + wait_queue_entry_t wait; > + poll_table pt; > + struct fd fd; > +}; > + > +static void gh_irqfd_cleanup(struct kref *kref) > +{ > + struct gunyah_irqfd *irqfd = container_of(kref, struct gunyah_irqfd, kref); > + > + kfree(irqfd); > +} > + > +static int irqfd_wakeup(wait_queue_entry_t *wait, unsigned int mode, int sync, void *key) > +{ > + struct gunyah_irqfd *irqfd = container_of(wait, struct gunyah_irqfd, wait); > + __poll_t flags = key_to_poll(key); > + u64 enable_mask = GH_DBL_NONBLOCK; > + u64 old_flags; > + int ret = 0; > + > + if (flags & EPOLLIN) { > + if (irqfd->ghrsc) { > + ret = gh_hypercall_dbl_send(irqfd->ghrsc->capid, enable_mask, &old_flags); > + if (ret) > + pr_err("Failed to assert irq %d\n", irqfd->f->fn.irqfd.label); > + } > + } > + > + return 0; > +} > + > +static void irqfd_ptable_queue_proc(struct file *file, wait_queue_head_t *wqh, poll_table *pt) > +{ > + struct gunyah_irqfd *irq_ctx = container_of(pt, struct gunyah_irqfd, pt); > + > + add_wait_queue(wqh, &irq_ctx->wait); > +} > + > +static int gunyah_irqfd_populate(struct gunyah_vm_resource_ticket *ticket, > + struct gunyah_resource *ghrsc) > +{ > + struct gunyah_irqfd *irqfd = container_of(ticket, struct gunyah_irqfd, ticket); > + u64 enable_mask = GH_DBL_NONBLOCK; > + u64 ack_mask = ~0; > + int ret = 0; > + > + irqfd->ghrsc = ghrsc; > + if (irqfd->level) { > + ret = gh_hypercall_dbl_set_mask(irqfd->ghrsc->capid, enable_mask, ack_mask); > + if (ret) > + pr_warn("irq %d couldn't be set as level triggered. Might cause IRQ storm if asserted\n", > + irqfd->f->fn.irqfd.label); > + } > + kref_get(&irqfd->kref); > + > + return 0; > +} > + > +static void gunyah_irqfd_unpopulate(struct gunyah_vm_resource_ticket *ticket, > + struct gunyah_resource *ghrsc) > +{ > + struct gunyah_irqfd *irqfd = container_of(ticket, struct gunyah_irqfd, ticket); > + u64 cnt; > + > + eventfd_ctx_remove_wait_queue(irqfd->ctx, &irqfd->wait, &cnt); > + eventfd_ctx_put(irqfd->ctx); > + fdput(irqfd->fd); <-- > + irqfd->ctx = NULL; > + irqfd->fd.file = NULL; > + irqfd->ghrsc = NULL; --> How do we know that this is the last reference ? > + kref_put(&irqfd->kref, gh_irqfd_cleanup); > +} > + > +static long gunyah_irqfd_bind(struct gunyah_vm_function *f) > +{ > + __poll_t events; > + struct gunyah_irqfd *irqfd; > + long r; > + > + irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL); > + if (!irqfd) > + return -ENOMEM; > + > + irqfd->f = f; > + f->data = irqfd; > + > + irqfd->fd = fdget(f->fn.irqfd.fd); > + if (!irqfd->fd.file) { > + r = -EBADF; > + goto err_free; > + } > + > + irqfd->ctx = eventfd_ctx_fileget(irqfd->fd.file); > + if (IS_ERR(irqfd->ctx)) { > + r = PTR_ERR(irqfd->ctx); > + goto err_fdput; > + } > + > + if (f->fn.irqfd.flags & GH_IRQFD_LEVEL) > + irqfd->level = true; > + > + init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup); > + init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc); > + kref_init(&irqfd->kref); > + > + irqfd->ticket.resource_type = GUNYAH_RESOURCE_TYPE_BELL_TX; > + irqfd->ticket.label = f->fn.irqfd.label; > + irqfd->ticket.owner = THIS_MODULE; > + irqfd->ticket.populate = gunyah_irqfd_populate; > + irqfd->ticket.unpopulate = gunyah_irqfd_unpopulate; > + > + r = ghvm_add_resource_ticket(f->ghvm, &irqfd->ticket); > + if (r) > + goto err_ctx; > + > + events = vfs_poll(irqfd->fd.file, &irqfd->pt); > + if (events & EPOLLIN) > + pr_warn("Premature injection of interrupt\n"); > + > + return 0; > +err_ctx: kref_put missing? > + eventfd_ctx_put(irqfd->ctx); > +err_fdput: > + fdput(irqfd->fd); > +err_free: > + kfree(irqfd); > + return r; > +} > + > +static void gunyah_irqfd_release(struct gunyah_vm_function *f) > +{ > + struct gunyah_irqfd *irqfd = f->data; > + > + /* unpopulate will trigger clean up of the eventfd */ > + ghvm_remove_resource_ticket(irqfd->f->ghvm, &irqfd->ticket); > +} > + > +DECLARE_GUNYAH_VM_FUNCTION_INIT(irqfd, gunyah_irqfd_bind, gunyah_irqfd_release); > +MODULE_DESCRIPTION("Gunyah irqfds"); > +MODULE_LICENSE("GPL"); > diff --git a/include/linux/gunyah.h b/include/linux/gunyah.h > index ac4879940c10..6b363707a901 100644 > --- a/include/linux/gunyah.h > +++ b/include/linux/gunyah.h > @@ -33,6 +33,11 @@ struct gunyah_resource { > u32 rm_label; > }; > > +/** > + * Gunyah Doorbells > + */ > +#define GH_DBL_NONBLOCK BIT(32) > + > /** > * Gunyah Message Queues > */ > diff --git a/include/uapi/linux/gunyah.h b/include/uapi/linux/gunyah.h > index b4afb11f538a..a947f0317ca9 100644 > --- a/include/uapi/linux/gunyah.h > +++ b/include/uapi/linux/gunyah.h > @@ -57,10 +57,19 @@ struct gh_fn_vcpu_arg { > __u32 vcpu_id; > }; > > +struct gh_fn_irqfd_arg { > + __u32 fd; > + __u32 label; > +#define GH_IRQFD_LEVEL (1UL << 0) > +#define GH_IRQFD_DEASSIGN (1UL << 1) > + __u32 flags; same issue here, this is not naturaly aligned. for details take a look at Documentation/driver-api/ioctl.rst > +}; > + > struct gh_vm_function { > char name[GUNYAH_FUNCTION_NAME_SIZE]; > union { > - struct gh_device_vcpu_arg vcpu; > + struct gh_fn_vcpu_arg vcpu; > + struct gh_fn_irqfd_arg irqfd; > char data[GUNYAH_FUNCTION_MAX_ARG_SIZE]; > }; > }; _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel