From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0FF3C433E6 for ; Mon, 21 Dec 2020 04:50:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 93F4B22CB2 for ; Mon, 21 Dec 2020 04:50:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728328AbgLUEuf (ORCPT ); Sun, 20 Dec 2020 23:50:35 -0500 Received: from szxga05-in.huawei.com ([45.249.212.191]:9543 "EHLO szxga05-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727219AbgLUEud (ORCPT ); Sun, 20 Dec 2020 23:50:33 -0500 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Czn7737k1zhv3g; Mon, 21 Dec 2020 12:49:07 +0800 (CST) Received: from [10.174.185.179] (10.174.185.179) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.498.0; Mon, 21 Dec 2020 12:49:40 +0800 Subject: Re: [PATCH] genirq/msi: Initialize msi_alloc_info to zero for msi_prepare API To: Marc Zyngier CC: , , , , References: <20201218060039.1770-1-yuzenghui@huawei.com> <87v9czqaj9.wl-maz@kernel.org> From: Zenghui Yu Message-ID: <48a318c3-ed5e-98fa-fcb1-502df088b78c@huawei.com> Date: Mon, 21 Dec 2020 12:49:38 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <87v9czqaj9.wl-maz@kernel.org> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.174.185.179] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Marc, On 2020/12/19 1:38, Marc Zyngier wrote: > Hi Zenghui, > > On Fri, 18 Dec 2020 06:00:39 +0000, > Zenghui Yu wrote: >> >> Since commit 5fe71d271df8 ("irqchip/gic-v3-its: Tag ITS device as shared if >> allocating for a proxy device"), some of the devices are wrongly marked as >> "shared" by the ITS driver on systems equipped with the ITS(es). The >> problem is that the @info->flags may not be initialized anywhere and we end >> up looking at random bits on the stack. That's obviously not good. >> >> The straightforward fix is to properly initialize msi_alloc_info inside the >> .prepare callback of affected MSI domains (its-pci-msi, its-platform-msi, >> etc). We can also perform the initialization in IRQ core layer for >> msi_domain_prepare_irqs() API and it looks much neater to me. >> >> Signed-off-by: Zenghui Yu >> --- >> >> This was noticed when I was playing with the assigned devices on arm64 and >> VFIO failed to enable MSI-X vectors for almost all VFs (CCed kvm list in >> case others will hit the same issue). It turned out that these VFs are >> marked as "shared" by mistake and have trouble with the following sequence: >> >> pci_alloc_irq_vectors(pdev, 1, 1, flag); >> pci_free_irq_vectors(pdev); >> pci_alloc_irq_vectors(pdev, 1, 2, flag); --> we can only get >> *one* vector >> >> But besides VFIO, I guess there are already some devices get into trouble >> at probe time and can't work properly. >> >> kernel/irq/msi.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c >> index 2c0c4d6d0f83..dc0e2d7fbdfd 100644 >> --- a/kernel/irq/msi.c >> +++ b/kernel/irq/msi.c >> @@ -402,7 +402,7 @@ int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, >> struct msi_domain_ops *ops = info->ops; >> struct irq_data *irq_data; >> struct msi_desc *desc; >> - msi_alloc_info_t arg; >> + msi_alloc_info_t arg = { }; >> int i, ret, virq; >> bool can_reserve; > > Thanks for having investigated this. I guess my only worry with this > is that msi_alloc_info_t is a pretty large structure on x86, and > zeroing it isn't totally free. It seems that x86 will zero the whole msi_alloc_info_t structure at the beginning of its .prepare (pci_msi_prepare()/init_irq_alloc_info()). If this really affects something, I think we can easily address it with some cleanup (on top of this patch). > But this definitely looks nicer than > some of the alternatives (.prepare isn't a good option, as we do rely > on the flag being set in __platform_msi_create_device_domain(), which > calls itself .prepare). Indeed, thanks for fixing the commit message. > I'll queue it, and we can always revisit this later if Thomas (or > anyone else) has a better idea. Thanks! Zenghui From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFF3CC433E0 for ; Mon, 21 Dec 2020 04:52:13 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 90E1022B2D for ; Mon, 21 Dec 2020 04:52:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 90E1022B2D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mNmPTvzI+tG1zIei1kGPeLAz7aKPc8n5VKboturOhJ8=; b=AaljXqT+79y1IxItTh09qQ0ze Sic1ppQvzO8cxxjpqgmbwO+iOuK/1TgnAZ8siT/zKzC8HS0ZrB4cXYPxqT3R1ynln47toISH1QVt4 kSXEUnn8cOGyg7NNwOm+NuBfLLshPzWLDKv/B5FpBwHF7Io8YJgqXCLhCOnARZUOvUSJ4IIH6/T+o /OBu8+ACWBKBYO9yCTFJNDHxquH45+6uK5v1e+Q4DKyVScy8hT4fATeKjlPIcs8H4p3y2rst3gJ+G 9Pxs6wnEdk7GOfQYV71TdSeyJxehNiXLq5tTqBmFtfTxhZ+cqUtls2SVjVAkFwmouLeuYgGU4tljK tBeMKp0Eg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1krD9B-0002S6-Lw; Mon, 21 Dec 2020 04:50:13 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1krD97-0002RL-2e for linux-arm-kernel@lists.infradead.org; Mon, 21 Dec 2020 04:50:11 +0000 Received: from DGGEMS408-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Czn7737k1zhv3g; Mon, 21 Dec 2020 12:49:07 +0800 (CST) Received: from [10.174.185.179] (10.174.185.179) by DGGEMS408-HUB.china.huawei.com (10.3.19.208) with Microsoft SMTP Server id 14.3.498.0; Mon, 21 Dec 2020 12:49:40 +0800 Subject: Re: [PATCH] genirq/msi: Initialize msi_alloc_info to zero for msi_prepare API To: Marc Zyngier References: <20201218060039.1770-1-yuzenghui@huawei.com> <87v9czqaj9.wl-maz@kernel.org> From: Zenghui Yu Message-ID: <48a318c3-ed5e-98fa-fcb1-502df088b78c@huawei.com> Date: Mon, 21 Dec 2020 12:49:38 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <87v9czqaj9.wl-maz@kernel.org> Content-Language: en-US X-Originating-IP: [10.174.185.179] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201220_235010_864435_C37156A6 X-CRM114-Status: GOOD ( 22.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: wanghaibin.wang@huawei.com, tglx@linutronix.de, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Marc, On 2020/12/19 1:38, Marc Zyngier wrote: > Hi Zenghui, > > On Fri, 18 Dec 2020 06:00:39 +0000, > Zenghui Yu wrote: >> >> Since commit 5fe71d271df8 ("irqchip/gic-v3-its: Tag ITS device as shared if >> allocating for a proxy device"), some of the devices are wrongly marked as >> "shared" by the ITS driver on systems equipped with the ITS(es). The >> problem is that the @info->flags may not be initialized anywhere and we end >> up looking at random bits on the stack. That's obviously not good. >> >> The straightforward fix is to properly initialize msi_alloc_info inside the >> .prepare callback of affected MSI domains (its-pci-msi, its-platform-msi, >> etc). We can also perform the initialization in IRQ core layer for >> msi_domain_prepare_irqs() API and it looks much neater to me. >> >> Signed-off-by: Zenghui Yu >> --- >> >> This was noticed when I was playing with the assigned devices on arm64 and >> VFIO failed to enable MSI-X vectors for almost all VFs (CCed kvm list in >> case others will hit the same issue). It turned out that these VFs are >> marked as "shared" by mistake and have trouble with the following sequence: >> >> pci_alloc_irq_vectors(pdev, 1, 1, flag); >> pci_free_irq_vectors(pdev); >> pci_alloc_irq_vectors(pdev, 1, 2, flag); --> we can only get >> *one* vector >> >> But besides VFIO, I guess there are already some devices get into trouble >> at probe time and can't work properly. >> >> kernel/irq/msi.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion(-) >> >> diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c >> index 2c0c4d6d0f83..dc0e2d7fbdfd 100644 >> --- a/kernel/irq/msi.c >> +++ b/kernel/irq/msi.c >> @@ -402,7 +402,7 @@ int __msi_domain_alloc_irqs(struct irq_domain *domain, struct device *dev, >> struct msi_domain_ops *ops = info->ops; >> struct irq_data *irq_data; >> struct msi_desc *desc; >> - msi_alloc_info_t arg; >> + msi_alloc_info_t arg = { }; >> int i, ret, virq; >> bool can_reserve; > > Thanks for having investigated this. I guess my only worry with this > is that msi_alloc_info_t is a pretty large structure on x86, and > zeroing it isn't totally free. It seems that x86 will zero the whole msi_alloc_info_t structure at the beginning of its .prepare (pci_msi_prepare()/init_irq_alloc_info()). If this really affects something, I think we can easily address it with some cleanup (on top of this patch). > But this definitely looks nicer than > some of the alternatives (.prepare isn't a good option, as we do rely > on the flag being set in __platform_msi_create_device_domain(), which > calls itself .prepare). Indeed, thanks for fixing the commit message. > I'll queue it, and we can always revisit this later if Thomas (or > anyone else) has a better idea. Thanks! Zenghui _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel