From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3B79C433B4 for ; Fri, 7 May 2021 10:00:42 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D9DF613D6 for ; Fri, 7 May 2021 10:00:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D9DF613D6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=hisilicon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:CC:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mkgGI59haQBx1VmCPYL63G2tqAzFI9WarKKlu4aOI0s=; b=crseD9/nE3C8lp7eHxn9Ae7Sj 2YrMVBn1ot17zYW+5rWNmfmBFqhVJQ3GS7OxaFBXGaSTZ0ocruKj5t7g7pZoqCc0dwzwKFvuhPBCa g0cL7nkACJlvk0U9sqEFpYYwjq/biLlYXKIWKLphu/cjaYCkRNlTLqzqf3rA0rVbVnPf1Xl1C7o1U HvJqiFg1Xkma8xyHXKy331CmovPQuhcM8CNM/CavSgjAtLIbGdBF2EcsZC3wG/4hQMbsRzG2vLLWA G2Qd9yr8bbpoO4+3ZesonyoTXl+pXc/kZcqjv+4SBicL5hkekJDXsHK8XAlLNrTFmkFe2+w+qj8tI zDyVfhc6g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lexFx-006kJN-73; Fri, 07 May 2021 09:58:49 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lexFq-006kIJ-D0 for linux-arm-kernel@desiato.infradead.org; Fri, 07 May 2021 09:58:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: Content-Type:In-Reply-To:MIME-Version:Date:Message-ID:From:References:CC:To: Subject:Sender:Reply-To:Content-ID:Content-Description; bh=NNl5wRFaJeCcVW+lHEIg4g9wr85q5QPcVAI4DAyh5zQ=; b=BKXTEBXEXUY2CJ6KvPG2O6TjF+ UWjnHHer02cX6Qmvzn5lVbAIcue7LLSoR6jfI7eZV5V4mcJiNgS2sbqGoO4J3uy7q43RV01SEVCvW l1bc+fqWOyPalxNHM0uCUhpcO+bILd2L3zKlM9CssUd1E5pHf9Yz9i8K8DqAqx2yIpsp9QSUeJ8B6 QgZDgGwGpJ7Scyd1lvlOKJ5cUJ2fdtOYe7FEZV5NuBQaa9/X/2RLbBhwu61DSl1tbjAdzzGjiad1e a0gBZdYu9fg1n7G+GQ4D1wh4OkV3maqHOPdKlSV7ZbUnB9dJJNkkR8mfqyygw/UXTgMgo7r0k2+ip CT/vliEw==; Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lexFn-006lHd-DH for linux-arm-kernel@lists.infradead.org; Fri, 07 May 2021 09:58:41 +0000 Received: from DGGEMS401-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Fc5Rw5QgTzkX9h; Fri, 7 May 2021 17:55:56 +0800 (CST) Received: from [10.67.77.175] (10.67.77.175) by DGGEMS401-HUB.china.huawei.com (10.3.19.201) with Microsoft SMTP Server id 14.3.498.0; Fri, 7 May 2021 17:58:23 +0800 Subject: Re: Question on guest enable msi fail when using GICv4/4.1 To: Marc Zyngier CC: , , , , Alex Williamson , Cornelia Huck , Nianyao Tang , Bjorn Helgaas References: <3a2c66d6-6ca0-8478-d24b-61e8e3241b20@hisilicon.com> <87k0oaq5jf.wl-maz@kernel.org> From: Shaokun Zhang Message-ID: Date: Fri, 7 May 2021 17:58:23 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 MIME-Version: 1.0 In-Reply-To: <87k0oaq5jf.wl-maz@kernel.org> Content-Language: en-US X-Originating-IP: [10.67.77.175] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210507_025839_642440_B10514B2 X-CRM114-Status: GOOD ( 21.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Marc, Thanks for your quick reply. On 2021/5/7 17:03, Marc Zyngier wrote: > On Fri, 07 May 2021 06:57:04 +0100, > Shaokun Zhang wrote: >> >> [This letter comes from Nianyao Tang] >> >> Hi, >> >> Using GICv4/4.1 and msi capability, guest vf driver requires 3 >> vectors and enable msi, will lead to guest stuck. > > Stuck how? Guest serial does not response anymore and guest network shutdown. > >> Qemu gets number of interrupts from Multiple Message Capable field >> set by guest. This field is aligned to a power of 2(if a function >> requires 3 vectors, it initializes it to 2). > > So I guess this is a MultiMSI device with 4 vectors, right? > Yes, it can support maximum of 32 msi interrupts, and vf driver only use 3 msi. >> However, guest driver just sends 3 mapi-cmd to vits and 3 ite >> entries is recorded in host. Vfio initializes msi interrupts using >> the number of interrupts 4 provide by qemu. When it comes to the >> 4th msi without ite in vits, in irq_bypass_register_producer, >> producer and consumer will __connect fail, due to find_ite fail, and >> do not resume guest. > > Let me rephrase this to check that I understand it: > - The device has 4 vectors > - The guest only create mappings for 3 of them > - VFIO calls kvm_vgic_v4_set_forwarding() for each vector > - KVM doesn't have a mapping for the 4th vector and returns an error > - VFIO disable this 4th vector > > Is that correct? If yes, I don't understand why that impacts the guest > at all. From what I can see, vfio_msi_set_vector_signal() just prints > a message on the console and carries on. > function calls: --> vfio_msi_set_vector_signal --> irq_bypass_register_producer -->__connect in __connect, add_producer finally calls kvm_vgic_v4_set_forwarding and fails to get the 4th mapping. When add_producer fail, it does not call cons->start, calls kvm_arch_irq_bypass_start and then kvm_arm_resume_guest. Thanks, Shaokun >> Do we support this case, Guest function using msi interrupts number >> not aligned to a power of 2? Or qemu should provide correct msi >> interrupts number? > > QEMU cannot know how many vectors are in use, and the guest is free to > issue mappings for the exact number of vectors it wants to service. > > Please describe what breaks the guest here. > > Thanks, > > M. > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel