From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F7EDC433B4 for ; Tue, 4 May 2021 15:30:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5410B6101A for ; Tue, 4 May 2021 15:30:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230525AbhEDPa6 convert rfc822-to-8bit (ORCPT ); Tue, 4 May 2021 11:30:58 -0400 Received: from 5.mo51.mail-out.ovh.net ([188.165.49.213]:33974 "EHLO 5.mo51.mail-out.ovh.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230212AbhEDPa5 (ORCPT ); Tue, 4 May 2021 11:30:57 -0400 X-Greylist: delayed 7764 seconds by postgrey-1.27 at vger.kernel.org; Tue, 04 May 2021 11:30:57 EDT Received: from mxplan5.mail.ovh.net (unknown [10.109.143.103]) by mo51.mail-out.ovh.net (Postfix) with ESMTPS id 1829F297B71; Tue, 4 May 2021 16:11:55 +0200 (CEST) Received: from kaod.org (37.59.142.104) by DAG8EX1.mxp5.local (172.16.2.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Tue, 4 May 2021 16:11:32 +0200 Authentication-Results: garm.ovh; auth=pass (GARM-104R0059937f50a-f377-4106-9467-b3b86bf2979e, 233BADB9E061AA125F593C9F78707CF28220F307) smtp.auth=groug@kaod.org X-OVh-ClientIp: 78.197.208.248 Date: Tue, 4 May 2021 16:11:31 +0200 From: Greg Kurz To: Greg Kroah-Hartman CC: Christoph Hellwig , Michael Ellerman , Alex Williamson , Jason Gunthorpe , , David Airlie , , , "Paul Mackerras" , Daniel Vetter , , , , Subject: Re: remove the nvlink2 pci_vfio subdriver v2 Message-ID: <20210504161131.2ed74d7b@bahia.lan> In-Reply-To: References: <20210326061311.1497642-1-hch@lst.de> <20210504142236.76994047@bahia.lan> <20210504152034.18e41ec3@bahia.lan> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8BIT X-Originating-IP: [37.59.142.104] X-ClientProxiedBy: DAG3EX1.mxp5.local (172.16.2.21) To DAG8EX1.mxp5.local (172.16.2.71) X-Ovh-Tracer-GUID: effd68a3-6459-416d-a28c-4964cefb2245 X-Ovh-Tracer-Id: 6202864065056512385 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduledrvdefiedgjeehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhepfffhvffukfgjfhfogggtgfhisehtqhertdertdejnecuhfhrohhmpefirhgvghcumfhurhiiuceoghhrohhugheskhgrohgurdhorhhgqeenucggtffrrghtthgvrhhnpeevlefhtddufffhieevhefhleegleelgfetffetkedugeehjeffgfehhfefueduffenucfkpheptddrtddrtddrtddpfeejrdehledrudegvddruddtgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhhouggvpehsmhhtphdqohhuthdphhgvlhhopehmgihplhgrnhehrdhmrghilhdrohhvhhdrnhgvthdpihhnvghtpedtrddtrddtrddtpdhmrghilhhfrhhomhepghhrohhugheskhgrohgurdhorhhgpdhrtghpthhtohepqhgvmhhuqdhpphgtsehnohhnghhnuhdrohhrgh Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 4 May 2021 15:30:15 +0200 Greg Kroah-Hartman wrote: > On Tue, May 04, 2021 at 03:20:34PM +0200, Greg Kurz wrote: > > On Tue, 4 May 2021 14:59:07 +0200 > > Greg Kroah-Hartman wrote: > > > > > On Tue, May 04, 2021 at 02:22:36PM +0200, Greg Kurz wrote: > > > > On Fri, 26 Mar 2021 07:13:09 +0100 > > > > Christoph Hellwig wrote: > > > > > > > > > Hi all, > > > > > > > > > > the nvlink2 vfio subdriver is a weird beast. It supports a hardware > > > > > feature without any open source component - what would normally be > > > > > the normal open source userspace that we require for kernel drivers, > > > > > although in this particular case user space could of course be a > > > > > kernel driver in a VM. It also happens to be a complete mess that > > > > > does not properly bind to PCI IDs, is hacked into the vfio_pci driver > > > > > and also pulles in over 1000 lines of code always build into powerpc > > > > > kernels that have Power NV support enabled. Because of all these > > > > > issues and the lack of breaking userspace when it is removed I think > > > > > the best idea is to simply kill. > > > > > > > > > > Changes since v1: > > > > > - document the removed subtypes as reserved > > > > > - add the ACK from Greg > > > > > > > > > > Diffstat: > > > > > arch/powerpc/platforms/powernv/npu-dma.c | 705 --------------------------- > > > > > b/arch/powerpc/include/asm/opal.h | 3 > > > > > b/arch/powerpc/include/asm/pci-bridge.h | 1 > > > > > b/arch/powerpc/include/asm/pci.h | 7 > > > > > b/arch/powerpc/platforms/powernv/Makefile | 2 > > > > > b/arch/powerpc/platforms/powernv/opal-call.c | 2 > > > > > b/arch/powerpc/platforms/powernv/pci-ioda.c | 185 ------- > > > > > b/arch/powerpc/platforms/powernv/pci.c | 11 > > > > > b/arch/powerpc/platforms/powernv/pci.h | 17 > > > > > b/arch/powerpc/platforms/pseries/pci.c | 23 > > > > > b/drivers/vfio/pci/Kconfig | 6 > > > > > b/drivers/vfio/pci/Makefile | 1 > > > > > b/drivers/vfio/pci/vfio_pci.c | 18 > > > > > b/drivers/vfio/pci/vfio_pci_private.h | 14 > > > > > b/include/uapi/linux/vfio.h | 38 - > > > > > > > > > > > > Hi Christoph, > > > > > > > > FYI, these uapi changes break build of QEMU. > > > > > > What uapi changes? > > > > > > > All macros and structure definitions that are being removed > > from include/uapi/linux/vfio.h by patch 1. > > > > > What exactly breaks? > > > > > > > These macros and types are used by the current QEMU code base. > > Next time the QEMU source tree updates its copy of the kernel > > headers, the compilation of affected code will fail. > > So does QEMU use this api that is being removed, or does it just have > some odd build artifacts of the uapi things? > These are region subtypes definition and associated capabilities. QEMU basically gets information on VFIO regions from the kernel driver and for those regions with a nvlink2 subtype, it tries to extract some more nvlink2 related info. > What exactly is the error messages here? > [55/143] Compiling C object libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o FAILED: libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o cc -Ilibqemu-ppc64-softmmu.fa.p -I. -I../.. -Itarget/ppc -I../../target/ppc -I../../capstone/include/capstone -Iqapi -Itrace -Iui -Iui/shader -I/usr/include/pixman-1 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fdiagnostics-color=auto -pipe -Wall -Winvalid-pch -Werror -std=gnu99 -O2 -g -isystem /home/greg/Work/qemu/qemu-virtiofs/linux-headers -isystem linux-headers -iquote . -iquote /home/greg/Work/qemu/qemu-virtiofs -iquote /home/greg/Work/qemu/qemu-virtiofs/include -iquote /home/greg/Work/qemu/qemu-virtiofs/disas/libvixl -iquote /home/greg/Work/qemu/qemu-virtiofs/tcg/ppc -iquote /home/greg/Work/qemu/qemu-virtiofs/accel/tcg -pthread -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wold-style-declaration -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -W empty-body -Wnested-externs -Wendif-labels -Wexpansion-to-defined -Wimplicit-fallthrough=2 -Wno-missing-include-dirs -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -fPIC -isystem../../linux-headers -isystemlinux-headers -DNEED_CPU_H '-DCONFIG_TARGET="ppc64-softmmu-config-target.h"' '-DCONFIG_DEVICES="ppc64-softmmu-config-devices.h"' -MD -MQ libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o -MF libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o.d -o libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o -c ../../hw/vfio/pci-quirks.c ../../hw/vfio/pci-quirks.c: In function ‘vfio_pci_nvidia_v100_ram_init’: ../../hw/vfio/pci-quirks.c:1597:36: error: ‘VFIO_REGION_SUBTYPE_NVIDIA_NVLINK2_RAM’ undeclared (first use in this function); did you mean ‘VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD’? VFIO_REGION_SUBTYPE_NVIDIA_NVLINK2_RAM, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD ../../hw/vfio/pci-quirks.c:1597:36: note: each undeclared identifier is reported only once for each function it appears in ../../hw/vfio/pci-quirks.c:1603:44: error: ‘VFIO_REGION_INFO_CAP_NVLINK2_SSATGT’ undeclared (first use in this function); did you mean ‘VFIO_REGION_INFO_CAP_SPARSE_MMAP’? hdr = vfio_get_region_info_cap(nv2reg, VFIO_REGION_INFO_CAP_NVLINK2_SSATGT); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1624:49: error: dereferencing pointer to incomplete type ‘struct vfio_region_info_cap_nvlink2_ssatgt’ (void *) (uintptr_t) cap->tgt); ^~ ../../hw/vfio/pci-quirks.c: In function ‘vfio_pci_nvlink2_init’: ../../hw/vfio/pci-quirks.c:1646:36: error: ‘VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD’ undeclared (first use in this function); did you mean ‘VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD’? VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD ../../hw/vfio/pci-quirks.c:1653:36: error: ‘VFIO_REGION_INFO_CAP_NVLINK2_SSATGT’ undeclared (first use in this function); did you mean ‘VFIO_REGION_INFO_CAP_SPARSE_MMAP’? VFIO_REGION_INFO_CAP_NVLINK2_SSATGT); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1661:36: error: ‘VFIO_REGION_INFO_CAP_NVLINK2_LNKSPD’ undeclared (first use in this function); did you mean ‘VFIO_REGION_INFO_CAP_SPARSE_MMAP’? VFIO_REGION_INFO_CAP_NVLINK2_LNKSPD); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1685:52: error: dereferencing pointer to incomplete type ‘struct vfio_region_info_cap_nvlink2_ssatgt’ (void *) (uintptr_t) captgt->tgt); ^~ ../../hw/vfio/pci-quirks.c:1691:54: error: dereferencing pointer to incomplete type ‘struct vfio_region_info_cap_nvlink2_lnkspd’ (void *) (uintptr_t) capspeed->link_speed); ^~ > And if we put the uapi .h file stuff back, is that sufficient for qemu > to work, as it should be checking at runtime what the kernel has / has > not anyway, right? > Right. This will just be dead code in QEMU for newer kernels. Anyway, as said in some other mail, it is probably time for QEMU to start deprecating this code as well. > thanks, > > greg k-h From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_2 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77700C433B4 for ; Tue, 4 May 2021 14:30:34 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C43DB613B3 for ; Tue, 4 May 2021 14:30:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C43DB613B3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4FZMh66vlCz30BC for ; Wed, 5 May 2021 00:30:30 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kaod.org (client-ip=46.105.77.235; helo=10.mo51.mail-out.ovh.net; envelope-from=groug@kaod.org; receiver=) X-Greylist: delayed 602 seconds by postgrey-1.36 at boromir; Wed, 05 May 2021 00:30:07 AEST Received: from 10.mo51.mail-out.ovh.net (10.mo51.mail-out.ovh.net [46.105.77.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4FZMgg2Nfdz2y88 for ; Wed, 5 May 2021 00:30:04 +1000 (AEST) Received: from mxplan5.mail.ovh.net (unknown [10.109.143.103]) by mo51.mail-out.ovh.net (Postfix) with ESMTPS id 1829F297B71; Tue, 4 May 2021 16:11:55 +0200 (CEST) Received: from kaod.org (37.59.142.104) by DAG8EX1.mxp5.local (172.16.2.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Tue, 4 May 2021 16:11:32 +0200 Authentication-Results: garm.ovh; auth=pass (GARM-104R0059937f50a-f377-4106-9467-b3b86bf2979e, 233BADB9E061AA125F593C9F78707CF28220F307) smtp.auth=groug@kaod.org X-OVh-ClientIp: 78.197.208.248 Date: Tue, 4 May 2021 16:11:31 +0200 From: Greg Kurz To: Greg Kroah-Hartman Subject: Re: remove the nvlink2 pci_vfio subdriver v2 Message-ID: <20210504161131.2ed74d7b@bahia.lan> In-Reply-To: References: <20210326061311.1497642-1-hch@lst.de> <20210504142236.76994047@bahia.lan> <20210504152034.18e41ec3@bahia.lan> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [37.59.142.104] X-ClientProxiedBy: DAG3EX1.mxp5.local (172.16.2.21) To DAG8EX1.mxp5.local (172.16.2.71) X-Ovh-Tracer-GUID: effd68a3-6459-416d-a28c-4964cefb2245 X-Ovh-Tracer-Id: 6202864065056512385 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduledrvdefiedgjeehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhepfffhvffukfgjfhfogggtgfhisehtqhertdertdejnecuhfhrohhmpefirhgvghcumfhurhiiuceoghhrohhugheskhgrohgurdhorhhgqeenucggtffrrghtthgvrhhnpeevlefhtddufffhieevhefhleegleelgfetffetkedugeehjeffgfehhfefueduffenucfkpheptddrtddrtddrtddpfeejrdehledrudegvddruddtgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhhouggvpehsmhhtphdqohhuthdphhgvlhhopehmgihplhgrnhehrdhmrghilhdrohhvhhdrnhgvthdpihhnvghtpedtrddtrddtrddtpdhmrghilhhfrhhomhepghhrohhugheskhgrohgurdhorhhgpdhrtghpthhtohepqhgvmhhuqdhpphgtsehnohhnghhnuhdrohhrgh X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , kvm@vger.kernel.org, David Airlie , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, qemu-devel@nongnu.org, Alex Williamson , Paul Mackerras , Jason Gunthorpe , linux-api@vger.kernel.org, qemu-ppc@nongnu.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Tue, 4 May 2021 15:30:15 +0200 Greg Kroah-Hartman wrote: > On Tue, May 04, 2021 at 03:20:34PM +0200, Greg Kurz wrote: > > On Tue, 4 May 2021 14:59:07 +0200 > > Greg Kroah-Hartman wrote: > >=20 > > > On Tue, May 04, 2021 at 02:22:36PM +0200, Greg Kurz wrote: > > > > On Fri, 26 Mar 2021 07:13:09 +0100 > > > > Christoph Hellwig wrote: > > > >=20 > > > > > Hi all, > > > > >=20 > > > > > the nvlink2 vfio subdriver is a weird beast. It supports a hardw= are > > > > > feature without any open source component - what would normally be > > > > > the normal open source userspace that we require for kernel drive= rs, > > > > > although in this particular case user space could of course be a > > > > > kernel driver in a VM. It also happens to be a complete mess that > > > > > does not properly bind to PCI IDs, is hacked into the vfio_pci dr= iver > > > > > and also pulles in over 1000 lines of code always build into powe= rpc > > > > > kernels that have Power NV support enabled. Because of all these > > > > > issues and the lack of breaking userspace when it is removed I th= ink > > > > > the best idea is to simply kill. > > > > >=20 > > > > > Changes since v1: > > > > > - document the removed subtypes as reserved > > > > > - add the ACK from Greg > > > > >=20 > > > > > Diffstat: > > > > > arch/powerpc/platforms/powernv/npu-dma.c | 705 ------------= --------------- > > > > > b/arch/powerpc/include/asm/opal.h | 3=20 > > > > > b/arch/powerpc/include/asm/pci-bridge.h | 1=20 > > > > > b/arch/powerpc/include/asm/pci.h | 7=20 > > > > > b/arch/powerpc/platforms/powernv/Makefile | 2=20 > > > > > b/arch/powerpc/platforms/powernv/opal-call.c | 2=20 > > > > > b/arch/powerpc/platforms/powernv/pci-ioda.c | 185 ------- > > > > > b/arch/powerpc/platforms/powernv/pci.c | 11=20 > > > > > b/arch/powerpc/platforms/powernv/pci.h | 17=20 > > > > > b/arch/powerpc/platforms/pseries/pci.c | 23=20 > > > > > b/drivers/vfio/pci/Kconfig | 6=20 > > > > > b/drivers/vfio/pci/Makefile | 1=20 > > > > > b/drivers/vfio/pci/vfio_pci.c | 18=20 > > > > > b/drivers/vfio/pci/vfio_pci_private.h | 14=20 > > > > > b/include/uapi/linux/vfio.h | 38 - > > > >=20 > > > >=20 > > > > Hi Christoph, > > > >=20 > > > > FYI, these uapi changes break build of QEMU. > > >=20 > > > What uapi changes? > > >=20 > >=20 > > All macros and structure definitions that are being removed > > from include/uapi/linux/vfio.h by patch 1. > >=20 > > > What exactly breaks? > > >=20 > >=20 > > These macros and types are used by the current QEMU code base. > > Next time the QEMU source tree updates its copy of the kernel > > headers, the compilation of affected code will fail. >=20 > So does QEMU use this api that is being removed, or does it just have > some odd build artifacts of the uapi things? >=20 These are region subtypes definition and associated capabilities. QEMU basically gets information on VFIO regions from the kernel driver and for those regions with a nvlink2 subtype, it tries to extract some more nvlink2 related info. > What exactly is the error messages here? >=20 [55/143] Compiling C object libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c= .o FAILED: libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o=20 cc -Ilibqemu-ppc64-softmmu.fa.p -I. -I../.. -Itarget/ppc -I../../target/ppc= -I../../capstone/include/capstone -Iqapi -Itrace -Iui -Iui/shader -I/usr/i= nclude/pixman-1 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fdia= gnostics-color=3Dauto -pipe -Wall -Winvalid-pch -Werror -std=3Dgnu99 -O2 -g= -isystem /home/greg/Work/qemu/qemu-virtiofs/linux-headers -isystem linux-h= eaders -iquote . -iquote /home/greg/Work/qemu/qemu-virtiofs -iquote /home/g= reg/Work/qemu/qemu-virtiofs/include -iquote /home/greg/Work/qemu/qemu-virti= ofs/disas/libvixl -iquote /home/greg/Work/qemu/qemu-virtiofs/tcg/ppc -iquot= e /home/greg/Work/qemu/qemu-virtiofs/accel/tcg -pthread -U_FORTIFY_SOURCE -= D_FORTIFY_SOURCE=3D2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=3D64 -D_LARGEFILE_SO= URCE -Wstrict-prototypes -Wredundant-decls -Wundef -Wwrite-strings -Wmissin= g-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wold-style-declarati= on -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Win= it-self -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -= Wexpansion-to-defined -Wimplicit-fallthrough=3D2 -Wno-missing-include-dirs = -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -fPIC -isyste= m../../linux-headers -isystemlinux-headers -DNEED_CPU_H '-DCONFIG_TARGET=3D= "ppc64-softmmu-config-target.h"' '-DCONFIG_DEVICES=3D"ppc64-softmmu-config-= devices.h"' -MD -MQ libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o -MF l= ibqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o.d -o libqemu-ppc64-softmmu= .fa.p/hw_vfio_pci-quirks.c.o -c ../../hw/vfio/pci-quirks.c ../../hw/vfio/pci-quirks.c: In function =E2=80=98vfio_pci_nvidia_v100_ram_i= nit=E2=80=99: ../../hw/vfio/pci-quirks.c:1597:36: error: =E2=80=98VFIO_REGION_SUBTYPE_NVI= DIA_NVLINK2_RAM=E2=80=99 undeclared (first use in this function); did you m= ean =E2=80=98VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD=E2=80=99? VFIO_REGION_SUBTYPE_NVIDIA_NVLINK2_RAM, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD ../../hw/vfio/pci-quirks.c:1597:36: note: each undeclared identifier is rep= orted only once for each function it appears in ../../hw/vfio/pci-quirks.c:1603:44: error: =E2=80=98VFIO_REGION_INFO_CAP_NV= LINK2_SSATGT=E2=80=99 undeclared (first use in this function); did you mean= =E2=80=98VFIO_REGION_INFO_CAP_SPARSE_MMAP=E2=80=99? hdr =3D vfio_get_region_info_cap(nv2reg, VFIO_REGION_INFO_CAP_NVLINK2_= SSATGT); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~= ~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1624:49: error: dereferencing pointer to incompl= ete type =E2=80=98struct vfio_region_info_cap_nvlink2_ssatgt=E2=80=99 (void *) (uintptr_t) cap->tgt); ^~ ../../hw/vfio/pci-quirks.c: In function =E2=80=98vfio_pci_nvlink2_init=E2= =80=99: ../../hw/vfio/pci-quirks.c:1646:36: error: =E2=80=98VFIO_REGION_SUBTYPE_IBM= _NVLINK2_ATSD=E2=80=99 undeclared (first use in this function); did you mea= n =E2=80=98VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD=E2=80=99? VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD ../../hw/vfio/pci-quirks.c:1653:36: error: =E2=80=98VFIO_REGION_INFO_CAP_NV= LINK2_SSATGT=E2=80=99 undeclared (first use in this function); did you mean= =E2=80=98VFIO_REGION_INFO_CAP_SPARSE_MMAP=E2=80=99? VFIO_REGION_INFO_CAP_NVLINK2_SSATGT); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1661:36: error: =E2=80=98VFIO_REGION_INFO_CAP_NV= LINK2_LNKSPD=E2=80=99 undeclared (first use in this function); did you mean= =E2=80=98VFIO_REGION_INFO_CAP_SPARSE_MMAP=E2=80=99? VFIO_REGION_INFO_CAP_NVLINK2_LNKSPD); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1685:52: error: dereferencing pointer to incompl= ete type =E2=80=98struct vfio_region_info_cap_nvlink2_ssatgt=E2=80=99 (void *) (uintptr_t) captgt->tgt); ^~ ../../hw/vfio/pci-quirks.c:1691:54: error: dereferencing pointer to incompl= ete type =E2=80=98struct vfio_region_info_cap_nvlink2_lnkspd=E2=80=99 (void *) (uintptr_t) capspeed->link_speed); ^~ > And if we put the uapi .h file stuff back, is that sufficient for qemu > to work, as it should be checking at runtime what the kernel has / has > not anyway, right? >=20 Right. This will just be dead code in QEMU for newer kernels. Anyway, as said in some other mail, it is probably time for QEMU to start deprecating this code as well. > thanks, >=20 > greg k-h From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 562B3C433ED for ; Tue, 4 May 2021 14:20:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5F3A610FA for ; Tue, 4 May 2021 14:20:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5F3A610FA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:37556 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ldvuN-0001Z7-T6 for qemu-devel@archiver.kernel.org; Tue, 04 May 2021 10:20:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54078) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldvme-0003nS-2j for qemu-devel@nongnu.org; Tue, 04 May 2021 10:12:20 -0400 Received: from 8.mo51.mail-out.ovh.net ([46.105.45.231]:59611) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ldvmX-0008H2-Ea for qemu-devel@nongnu.org; Tue, 04 May 2021 10:12:19 -0400 Received: from mxplan5.mail.ovh.net (unknown [10.109.143.103]) by mo51.mail-out.ovh.net (Postfix) with ESMTPS id 1829F297B71; Tue, 4 May 2021 16:11:55 +0200 (CEST) Received: from kaod.org (37.59.142.104) by DAG8EX1.mxp5.local (172.16.2.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Tue, 4 May 2021 16:11:32 +0200 Authentication-Results: garm.ovh; auth=pass (GARM-104R0059937f50a-f377-4106-9467-b3b86bf2979e, 233BADB9E061AA125F593C9F78707CF28220F307) smtp.auth=groug@kaod.org X-OVh-ClientIp: 78.197.208.248 Date: Tue, 4 May 2021 16:11:31 +0200 From: Greg Kurz To: Greg Kroah-Hartman Subject: Re: remove the nvlink2 pci_vfio subdriver v2 Message-ID: <20210504161131.2ed74d7b@bahia.lan> In-Reply-To: References: <20210326061311.1497642-1-hch@lst.de> <20210504142236.76994047@bahia.lan> <20210504152034.18e41ec3@bahia.lan> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Originating-IP: [37.59.142.104] X-ClientProxiedBy: DAG3EX1.mxp5.local (172.16.2.21) To DAG8EX1.mxp5.local (172.16.2.71) X-Ovh-Tracer-GUID: effd68a3-6459-416d-a28c-4964cefb2245 X-Ovh-Tracer-Id: 6202864065056512385 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduledrvdefiedgjeehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhepfffhvffukfgjfhfogggtgfhisehtqhertdertdejnecuhfhrohhmpefirhgvghcumfhurhiiuceoghhrohhugheskhgrohgurdhorhhgqeenucggtffrrghtthgvrhhnpeevlefhtddufffhieevhefhleegleelgfetffetkedugeehjeffgfehhfefueduffenucfkpheptddrtddrtddrtddpfeejrdehledrudegvddruddtgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhhouggvpehsmhhtphdqohhuthdphhgvlhhopehmgihplhgrnhehrdhmrghilhdrohhvhhdrnhgvthdpihhnvghtpedtrddtrddtrddtpdhmrghilhhfrhhomhepghhrohhugheskhgrohgurdhorhhgpdhrtghpthhtohepqhgvmhhuqdhpphgtsehnohhnghhnuhdrohhrgh Received-SPF: pass client-ip=46.105.45.231; envelope-from=groug@kaod.org; helo=8.mo51.mail-out.ovh.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Daniel Vetter , kvm@vger.kernel.org, David Airlie , Michael Ellerman , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, qemu-devel@nongnu.org, Alex Williamson , Paul Mackerras , Jason Gunthorpe , linux-api@vger.kernel.org, qemu-ppc@nongnu.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" On Tue, 4 May 2021 15:30:15 +0200 Greg Kroah-Hartman wrote: > On Tue, May 04, 2021 at 03:20:34PM +0200, Greg Kurz wrote: > > On Tue, 4 May 2021 14:59:07 +0200 > > Greg Kroah-Hartman wrote: > >=20 > > > On Tue, May 04, 2021 at 02:22:36PM +0200, Greg Kurz wrote: > > > > On Fri, 26 Mar 2021 07:13:09 +0100 > > > > Christoph Hellwig wrote: > > > >=20 > > > > > Hi all, > > > > >=20 > > > > > the nvlink2 vfio subdriver is a weird beast. It supports a hardw= are > > > > > feature without any open source component - what would normally be > > > > > the normal open source userspace that we require for kernel drive= rs, > > > > > although in this particular case user space could of course be a > > > > > kernel driver in a VM. It also happens to be a complete mess that > > > > > does not properly bind to PCI IDs, is hacked into the vfio_pci dr= iver > > > > > and also pulles in over 1000 lines of code always build into powe= rpc > > > > > kernels that have Power NV support enabled. Because of all these > > > > > issues and the lack of breaking userspace when it is removed I th= ink > > > > > the best idea is to simply kill. > > > > >=20 > > > > > Changes since v1: > > > > > - document the removed subtypes as reserved > > > > > - add the ACK from Greg > > > > >=20 > > > > > Diffstat: > > > > > arch/powerpc/platforms/powernv/npu-dma.c | 705 ------------= --------------- > > > > > b/arch/powerpc/include/asm/opal.h | 3=20 > > > > > b/arch/powerpc/include/asm/pci-bridge.h | 1=20 > > > > > b/arch/powerpc/include/asm/pci.h | 7=20 > > > > > b/arch/powerpc/platforms/powernv/Makefile | 2=20 > > > > > b/arch/powerpc/platforms/powernv/opal-call.c | 2=20 > > > > > b/arch/powerpc/platforms/powernv/pci-ioda.c | 185 ------- > > > > > b/arch/powerpc/platforms/powernv/pci.c | 11=20 > > > > > b/arch/powerpc/platforms/powernv/pci.h | 17=20 > > > > > b/arch/powerpc/platforms/pseries/pci.c | 23=20 > > > > > b/drivers/vfio/pci/Kconfig | 6=20 > > > > > b/drivers/vfio/pci/Makefile | 1=20 > > > > > b/drivers/vfio/pci/vfio_pci.c | 18=20 > > > > > b/drivers/vfio/pci/vfio_pci_private.h | 14=20 > > > > > b/include/uapi/linux/vfio.h | 38 - > > > >=20 > > > >=20 > > > > Hi Christoph, > > > >=20 > > > > FYI, these uapi changes break build of QEMU. > > >=20 > > > What uapi changes? > > >=20 > >=20 > > All macros and structure definitions that are being removed > > from include/uapi/linux/vfio.h by patch 1. > >=20 > > > What exactly breaks? > > >=20 > >=20 > > These macros and types are used by the current QEMU code base. > > Next time the QEMU source tree updates its copy of the kernel > > headers, the compilation of affected code will fail. >=20 > So does QEMU use this api that is being removed, or does it just have > some odd build artifacts of the uapi things? >=20 These are region subtypes definition and associated capabilities. QEMU basically gets information on VFIO regions from the kernel driver and for those regions with a nvlink2 subtype, it tries to extract some more nvlink2 related info. > What exactly is the error messages here? >=20 [55/143] Compiling C object libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c= .o FAILED: libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o=20 cc -Ilibqemu-ppc64-softmmu.fa.p -I. -I../.. -Itarget/ppc -I../../target/ppc= -I../../capstone/include/capstone -Iqapi -Itrace -Iui -Iui/shader -I/usr/i= nclude/pixman-1 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fdia= gnostics-color=3Dauto -pipe -Wall -Winvalid-pch -Werror -std=3Dgnu99 -O2 -g= -isystem /home/greg/Work/qemu/qemu-virtiofs/linux-headers -isystem linux-h= eaders -iquote . -iquote /home/greg/Work/qemu/qemu-virtiofs -iquote /home/g= reg/Work/qemu/qemu-virtiofs/include -iquote /home/greg/Work/qemu/qemu-virti= ofs/disas/libvixl -iquote /home/greg/Work/qemu/qemu-virtiofs/tcg/ppc -iquot= e /home/greg/Work/qemu/qemu-virtiofs/accel/tcg -pthread -U_FORTIFY_SOURCE -= D_FORTIFY_SOURCE=3D2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=3D64 -D_LARGEFILE_SO= URCE -Wstrict-prototypes -Wredundant-decls -Wundef -Wwrite-strings -Wmissin= g-prototypes -fno-strict-aliasing -fno-common -fwrapv -Wold-style-declarati= on -Wold-style-definition -Wtype-limits -Wformat-security -Wformat-y2k -Win= it-self -Wignored-qualifiers -Wempty-body -Wnested-externs -Wendif-labels -= Wexpansion-to-defined -Wimplicit-fallthrough=3D2 -Wno-missing-include-dirs = -Wno-shift-negative-value -Wno-psabi -fstack-protector-strong -fPIC -isyste= m../../linux-headers -isystemlinux-headers -DNEED_CPU_H '-DCONFIG_TARGET=3D= "ppc64-softmmu-config-target.h"' '-DCONFIG_DEVICES=3D"ppc64-softmmu-config-= devices.h"' -MD -MQ libqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o -MF l= ibqemu-ppc64-softmmu.fa.p/hw_vfio_pci-quirks.c.o.d -o libqemu-ppc64-softmmu= .fa.p/hw_vfio_pci-quirks.c.o -c ../../hw/vfio/pci-quirks.c ../../hw/vfio/pci-quirks.c: In function =E2=80=98vfio_pci_nvidia_v100_ram_i= nit=E2=80=99: ../../hw/vfio/pci-quirks.c:1597:36: error: =E2=80=98VFIO_REGION_SUBTYPE_NVI= DIA_NVLINK2_RAM=E2=80=99 undeclared (first use in this function); did you m= ean =E2=80=98VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD=E2=80=99? VFIO_REGION_SUBTYPE_NVIDIA_NVLINK2_RAM, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD ../../hw/vfio/pci-quirks.c:1597:36: note: each undeclared identifier is rep= orted only once for each function it appears in ../../hw/vfio/pci-quirks.c:1603:44: error: =E2=80=98VFIO_REGION_INFO_CAP_NV= LINK2_SSATGT=E2=80=99 undeclared (first use in this function); did you mean= =E2=80=98VFIO_REGION_INFO_CAP_SPARSE_MMAP=E2=80=99? hdr =3D vfio_get_region_info_cap(nv2reg, VFIO_REGION_INFO_CAP_NVLINK2_= SSATGT); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~= ~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1624:49: error: dereferencing pointer to incompl= ete type =E2=80=98struct vfio_region_info_cap_nvlink2_ssatgt=E2=80=99 (void *) (uintptr_t) cap->tgt); ^~ ../../hw/vfio/pci-quirks.c: In function =E2=80=98vfio_pci_nvlink2_init=E2= =80=99: ../../hw/vfio/pci-quirks.c:1646:36: error: =E2=80=98VFIO_REGION_SUBTYPE_IBM= _NVLINK2_ATSD=E2=80=99 undeclared (first use in this function); did you mea= n =E2=80=98VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD=E2=80=99? VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD ../../hw/vfio/pci-quirks.c:1653:36: error: =E2=80=98VFIO_REGION_INFO_CAP_NV= LINK2_SSATGT=E2=80=99 undeclared (first use in this function); did you mean= =E2=80=98VFIO_REGION_INFO_CAP_SPARSE_MMAP=E2=80=99? VFIO_REGION_INFO_CAP_NVLINK2_SSATGT); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1661:36: error: =E2=80=98VFIO_REGION_INFO_CAP_NV= LINK2_LNKSPD=E2=80=99 undeclared (first use in this function); did you mean= =E2=80=98VFIO_REGION_INFO_CAP_SPARSE_MMAP=E2=80=99? VFIO_REGION_INFO_CAP_NVLINK2_LNKSPD); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ VFIO_REGION_INFO_CAP_SPARSE_MMAP ../../hw/vfio/pci-quirks.c:1685:52: error: dereferencing pointer to incompl= ete type =E2=80=98struct vfio_region_info_cap_nvlink2_ssatgt=E2=80=99 (void *) (uintptr_t) captgt->tgt); ^~ ../../hw/vfio/pci-quirks.c:1691:54: error: dereferencing pointer to incompl= ete type =E2=80=98struct vfio_region_info_cap_nvlink2_lnkspd=E2=80=99 (void *) (uintptr_t) capspeed->link_speed); ^~ > And if we put the uapi .h file stuff back, is that sufficient for qemu > to work, as it should be checking at runtime what the kernel has / has > not anyway, right? >=20 Right. This will just be dead code in QEMU for newer kernels. Anyway, as said in some other mail, it is probably time for QEMU to start deprecating this code as well. > thanks, >=20 > greg k-h From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_2 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23CC5C433ED for ; Tue, 4 May 2021 16:49:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A138761177 for ; Tue, 4 May 2021 16:49:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A138761177 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kaod.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 68D606EB30; Tue, 4 May 2021 16:49:49 +0000 (UTC) X-Greylist: delayed 1073 seconds by postgrey-1.36 at gabe; Tue, 04 May 2021 14:30:02 UTC Received: from 9.mo51.mail-out.ovh.net (9.mo51.mail-out.ovh.net [46.105.48.137]) by gabe.freedesktop.org (Postfix) with ESMTPS id F0BBD6EB0F for ; Tue, 4 May 2021 14:30:02 +0000 (UTC) Received: from mxplan5.mail.ovh.net (unknown [10.109.143.103]) by mo51.mail-out.ovh.net (Postfix) with ESMTPS id 1829F297B71; Tue, 4 May 2021 16:11:55 +0200 (CEST) Received: from kaod.org (37.59.142.104) by DAG8EX1.mxp5.local (172.16.2.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.4; Tue, 4 May 2021 16:11:32 +0200 Authentication-Results: garm.ovh; auth=pass (GARM-104R0059937f50a-f377-4106-9467-b3b86bf2979e, 233BADB9E061AA125F593C9F78707CF28220F307) smtp.auth=groug@kaod.org X-OVh-ClientIp: 78.197.208.248 Date: Tue, 4 May 2021 16:11:31 +0200 From: Greg Kurz To: Greg Kroah-Hartman Subject: Re: remove the nvlink2 pci_vfio subdriver v2 Message-ID: <20210504161131.2ed74d7b@bahia.lan> In-Reply-To: References: <20210326061311.1497642-1-hch@lst.de> <20210504142236.76994047@bahia.lan> <20210504152034.18e41ec3@bahia.lan> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-redhat-linux-gnu) MIME-Version: 1.0 X-Originating-IP: [37.59.142.104] X-ClientProxiedBy: DAG3EX1.mxp5.local (172.16.2.21) To DAG8EX1.mxp5.local (172.16.2.71) X-Ovh-Tracer-GUID: effd68a3-6459-416d-a28c-4964cefb2245 X-Ovh-Tracer-Id: 6202864065056512385 X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrgeduledrvdefiedgjeehucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuqfggjfdpvefjgfevmfevgfenuceurghilhhouhhtmecuhedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhepfffhvffukfgjfhfogggtgfhisehtqhertdertdejnecuhfhrohhmpefirhgvghcumfhurhiiuceoghhrohhugheskhgrohgurdhorhhgqeenucggtffrrghtthgvrhhnpeevlefhtddufffhieevhefhleegleelgfetffetkedugeehjeffgfehhfefueduffenucfkpheptddrtddrtddrtddpfeejrdehledrudegvddruddtgeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhhouggvpehsmhhtphdqohhuthdphhgvlhhopehmgihplhgrnhehrdhmrghilhdrohhvhhdrnhgvthdpihhnvghtpedtrddtrddtrddtpdhmrghilhhfrhhomhepghhrohhugheskhgrohgurdhorhhgpdhrtghpthhtohepqhgvmhhuqdhpphgtsehnohhnghhnuhdrohhrgh X-Mailman-Approved-At: Tue, 04 May 2021 16:49:47 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kvm@vger.kernel.org, David Airlie , Michael Ellerman , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, qemu-devel@nongnu.org, Alex Williamson , Paul Mackerras , Jason Gunthorpe , linux-api@vger.kernel.org, qemu-ppc@nongnu.org, linuxppc-dev@lists.ozlabs.org, Christoph Hellwig Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T24gVHVlLCA0IE1heSAyMDIxIDE1OjMwOjE1ICswMjAwCkdyZWcgS3JvYWgtSGFydG1hbiA8Z3Jl Z2toQGxpbnV4Zm91bmRhdGlvbi5vcmc+IHdyb3RlOgoKPiBPbiBUdWUsIE1heSAwNCwgMjAyMSBh dCAwMzoyMDozNFBNICswMjAwLCBHcmVnIEt1cnogd3JvdGU6Cj4gPiBPbiBUdWUsIDQgTWF5IDIw MjEgMTQ6NTk6MDcgKzAyMDAKPiA+IEdyZWcgS3JvYWgtSGFydG1hbiA8Z3JlZ2toQGxpbnV4Zm91 bmRhdGlvbi5vcmc+IHdyb3RlOgo+ID4gCj4gPiA+IE9uIFR1ZSwgTWF5IDA0LCAyMDIxIGF0IDAy OjIyOjM2UE0gKzAyMDAsIEdyZWcgS3VyeiB3cm90ZToKPiA+ID4gPiBPbiBGcmksIDI2IE1hciAy MDIxIDA3OjEzOjA5ICswMTAwCj4gPiA+ID4gQ2hyaXN0b3BoIEhlbGx3aWcgPGhjaEBsc3QuZGU+ IHdyb3RlOgo+ID4gPiA+IAo+ID4gPiA+ID4gSGkgYWxsLAo+ID4gPiA+ID4gCj4gPiA+ID4gPiB0 aGUgbnZsaW5rMiB2ZmlvIHN1YmRyaXZlciBpcyBhIHdlaXJkIGJlYXN0LiAgSXQgc3VwcG9ydHMg YSBoYXJkd2FyZQo+ID4gPiA+ID4gZmVhdHVyZSB3aXRob3V0IGFueSBvcGVuIHNvdXJjZSBjb21w b25lbnQgLSB3aGF0IHdvdWxkIG5vcm1hbGx5IGJlCj4gPiA+ID4gPiB0aGUgbm9ybWFsIG9wZW4g c291cmNlIHVzZXJzcGFjZSB0aGF0IHdlIHJlcXVpcmUgZm9yIGtlcm5lbCBkcml2ZXJzLAo+ID4g PiA+ID4gYWx0aG91Z2ggaW4gdGhpcyBwYXJ0aWN1bGFyIGNhc2UgdXNlciBzcGFjZSBjb3VsZCBv ZiBjb3Vyc2UgYmUgYQo+ID4gPiA+ID4ga2VybmVsIGRyaXZlciBpbiBhIFZNLiAgSXQgYWxzbyBo YXBwZW5zIHRvIGJlIGEgY29tcGxldGUgbWVzcyB0aGF0Cj4gPiA+ID4gPiBkb2VzIG5vdCBwcm9w ZXJseSBiaW5kIHRvIFBDSSBJRHMsIGlzIGhhY2tlZCBpbnRvIHRoZSB2ZmlvX3BjaSBkcml2ZXIK PiA+ID4gPiA+IGFuZCBhbHNvIHB1bGxlcyBpbiBvdmVyIDEwMDAgbGluZXMgb2YgY29kZSBhbHdh eXMgYnVpbGQgaW50byBwb3dlcnBjCj4gPiA+ID4gPiBrZXJuZWxzIHRoYXQgaGF2ZSBQb3dlciBO ViBzdXBwb3J0IGVuYWJsZWQuICBCZWNhdXNlIG9mIGFsbCB0aGVzZQo+ID4gPiA+ID4gaXNzdWVz IGFuZCB0aGUgbGFjayBvZiBicmVha2luZyB1c2Vyc3BhY2Ugd2hlbiBpdCBpcyByZW1vdmVkIEkg dGhpbmsKPiA+ID4gPiA+IHRoZSBiZXN0IGlkZWEgaXMgdG8gc2ltcGx5IGtpbGwuCj4gPiA+ID4g PiAKPiA+ID4gPiA+IENoYW5nZXMgc2luY2UgdjE6Cj4gPiA+ID4gPiAgLSBkb2N1bWVudCB0aGUg cmVtb3ZlZCBzdWJ0eXBlcyBhcyByZXNlcnZlZAo+ID4gPiA+ID4gIC0gYWRkIHRoZSBBQ0sgZnJv bSBHcmVnCj4gPiA+ID4gPiAKPiA+ID4gPiA+IERpZmZzdGF0Ogo+ID4gPiA+ID4gIGFyY2gvcG93 ZXJwYy9wbGF0Zm9ybXMvcG93ZXJudi9ucHUtZG1hLmMgICAgIHwgIDcwNSAtLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0KPiA+ID4gPiA+ICBiL2FyY2gvcG93ZXJwYy9pbmNsdWRlL2FzbS9vcGFs LmggICAgICAgICAgICB8ICAgIDMgCj4gPiA+ID4gPiAgYi9hcmNoL3Bvd2VycGMvaW5jbHVkZS9h c20vcGNpLWJyaWRnZS5oICAgICAgfCAgICAxIAo+ID4gPiA+ID4gIGIvYXJjaC9wb3dlcnBjL2lu Y2x1ZGUvYXNtL3BjaS5oICAgICAgICAgICAgIHwgICAgNyAKPiA+ID4gPiA+ICBiL2FyY2gvcG93 ZXJwYy9wbGF0Zm9ybXMvcG93ZXJudi9NYWtlZmlsZSAgICB8ICAgIDIgCj4gPiA+ID4gPiAgYi9h cmNoL3Bvd2VycGMvcGxhdGZvcm1zL3Bvd2VybnYvb3BhbC1jYWxsLmMgfCAgICAyIAo+ID4gPiA+ ID4gIGIvYXJjaC9wb3dlcnBjL3BsYXRmb3Jtcy9wb3dlcm52L3BjaS1pb2RhLmMgIHwgIDE4NSAt LS0tLS0tCj4gPiA+ID4gPiAgYi9hcmNoL3Bvd2VycGMvcGxhdGZvcm1zL3Bvd2VybnYvcGNpLmMg ICAgICAgfCAgIDExIAo+ID4gPiA+ID4gIGIvYXJjaC9wb3dlcnBjL3BsYXRmb3Jtcy9wb3dlcm52 L3BjaS5oICAgICAgIHwgICAxNyAKPiA+ID4gPiA+ICBiL2FyY2gvcG93ZXJwYy9wbGF0Zm9ybXMv cHNlcmllcy9wY2kuYyAgICAgICB8ICAgMjMgCj4gPiA+ID4gPiAgYi9kcml2ZXJzL3ZmaW8vcGNp L0tjb25maWcgICAgICAgICAgICAgICAgICAgfCAgICA2IAo+ID4gPiA+ID4gIGIvZHJpdmVycy92 ZmlvL3BjaS9NYWtlZmlsZSAgICAgICAgICAgICAgICAgIHwgICAgMSAKPiA+ID4gPiA+ICBiL2Ry aXZlcnMvdmZpby9wY2kvdmZpb19wY2kuYyAgICAgICAgICAgICAgICB8ICAgMTggCj4gPiA+ID4g PiAgYi9kcml2ZXJzL3ZmaW8vcGNpL3ZmaW9fcGNpX3ByaXZhdGUuaCAgICAgICAgfCAgIDE0IAo+ ID4gPiA+ID4gIGIvaW5jbHVkZS91YXBpL2xpbnV4L3ZmaW8uaCAgICAgICAgICAgICAgICAgIHwg ICAzOCAtCj4gPiA+ID4gCj4gPiA+ID4gCj4gPiA+ID4gSGkgQ2hyaXN0b3BoLAo+ID4gPiA+IAo+ ID4gPiA+IEZZSSwgdGhlc2UgdWFwaSBjaGFuZ2VzIGJyZWFrIGJ1aWxkIG9mIFFFTVUuCj4gPiA+ IAo+ID4gPiBXaGF0IHVhcGkgY2hhbmdlcz8KPiA+ID4gCj4gPiAKPiA+IEFsbCBtYWNyb3MgYW5k IHN0cnVjdHVyZSBkZWZpbml0aW9ucyB0aGF0IGFyZSBiZWluZyByZW1vdmVkCj4gPiBmcm9tIGlu Y2x1ZGUvdWFwaS9saW51eC92ZmlvLmggYnkgcGF0Y2ggMS4KPiA+IAo+ID4gPiBXaGF0IGV4YWN0 bHkgYnJlYWtzPwo+ID4gPiAKPiA+IAo+ID4gVGhlc2UgbWFjcm9zIGFuZCB0eXBlcyBhcmUgdXNl ZCBieSB0aGUgY3VycmVudCBRRU1VIGNvZGUgYmFzZS4KPiA+IE5leHQgdGltZSB0aGUgUUVNVSBz b3VyY2UgdHJlZSB1cGRhdGVzIGl0cyBjb3B5IG9mIHRoZSBrZXJuZWwKPiA+IGhlYWRlcnMsIHRo ZSBjb21waWxhdGlvbiBvZiBhZmZlY3RlZCBjb2RlIHdpbGwgZmFpbC4KPiAKPiBTbyBkb2VzIFFF TVUgdXNlIHRoaXMgYXBpIHRoYXQgaXMgYmVpbmcgcmVtb3ZlZCwgb3IgZG9lcyBpdCBqdXN0IGhh dmUKPiBzb21lIG9kZCBidWlsZCBhcnRpZmFjdHMgb2YgdGhlIHVhcGkgdGhpbmdzPwo+IAoKVGhl c2UgYXJlIHJlZ2lvbiBzdWJ0eXBlcyBkZWZpbml0aW9uIGFuZCBhc3NvY2lhdGVkIGNhcGFiaWxp dGllcy4KUUVNVSBiYXNpY2FsbHkgZ2V0cyBpbmZvcm1hdGlvbiBvbiBWRklPIHJlZ2lvbnMgZnJv bSB0aGUga2VybmVsCmRyaXZlciBhbmQgZm9yIHRob3NlIHJlZ2lvbnMgd2l0aCBhIG52bGluazIg c3VidHlwZSwgaXQgdHJpZXMKdG8gZXh0cmFjdCBzb21lIG1vcmUgbnZsaW5rMiByZWxhdGVkIGlu Zm8uCgo+IFdoYXQgZXhhY3RseSBpcyB0aGUgZXJyb3IgbWVzc2FnZXMgaGVyZT8KPiAKCls1NS8x NDNdIENvbXBpbGluZyBDIG9iamVjdCBsaWJxZW11LXBwYzY0LXNvZnRtbXUuZmEucC9od192Zmlv X3BjaS1xdWlya3MuYy5vCkZBSUxFRDogbGlicWVtdS1wcGM2NC1zb2Z0bW11LmZhLnAvaHdfdmZp b19wY2ktcXVpcmtzLmMubyAKY2MgLUlsaWJxZW11LXBwYzY0LXNvZnRtbXUuZmEucCAtSS4gLUku Li8uLiAtSXRhcmdldC9wcGMgLUkuLi8uLi90YXJnZXQvcHBjIC1JLi4vLi4vY2Fwc3RvbmUvaW5j bHVkZS9jYXBzdG9uZSAtSXFhcGkgLUl0cmFjZSAtSXVpIC1JdWkvc2hhZGVyIC1JL3Vzci9pbmNs dWRlL3BpeG1hbi0xIC1JL3Vzci9pbmNsdWRlL2dsaWItMi4wIC1JL3Vzci9saWI2NC9nbGliLTIu MC9pbmNsdWRlIC1mZGlhZ25vc3RpY3MtY29sb3I9YXV0byAtcGlwZSAtV2FsbCAtV2ludmFsaWQt cGNoIC1XZXJyb3IgLXN0ZD1nbnU5OSAtTzIgLWcgLWlzeXN0ZW0gL2hvbWUvZ3JlZy9Xb3JrL3Fl bXUvcWVtdS12aXJ0aW9mcy9saW51eC1oZWFkZXJzIC1pc3lzdGVtIGxpbnV4LWhlYWRlcnMgLWlx dW90ZSAuIC1pcXVvdGUgL2hvbWUvZ3JlZy9Xb3JrL3FlbXUvcWVtdS12aXJ0aW9mcyAtaXF1b3Rl IC9ob21lL2dyZWcvV29yay9xZW11L3FlbXUtdmlydGlvZnMvaW5jbHVkZSAtaXF1b3RlIC9ob21l L2dyZWcvV29yay9xZW11L3FlbXUtdmlydGlvZnMvZGlzYXMvbGlidml4bCAtaXF1b3RlIC9ob21l L2dyZWcvV29yay9xZW11L3FlbXUtdmlydGlvZnMvdGNnL3BwYyAtaXF1b3RlIC9ob21lL2dyZWcv V29yay9xZW11L3FlbXUtdmlydGlvZnMvYWNjZWwvdGNnIC1wdGhyZWFkIC1VX0ZPUlRJRllfU09V UkNFIC1EX0ZPUlRJRllfU09VUkNFPTIgLURfR05VX1NPVVJDRSAtRF9GSUxFX09GRlNFVF9CSVRT PTY0IC1EX0xBUkdFRklMRV9TT1VSQ0UgLVdzdHJpY3QtcHJvdG90eXBlcyAtV3JlZHVuZGFudC1k ZWNscyAtV3VuZGVmIC1Xd3JpdGUtc3RyaW5ncyAtV21pc3NpbmctcHJvdG90eXBlcyAtZm5vLXN0 cmljdC1hbGlhc2luZyAtZm5vLWNvbW1vbiAtZndyYXB2IC1Xb2xkLXN0eWxlLWRlY2xhcmF0aW9u IC1Xb2xkLXN0eWxlLWRlZmluaXRpb24gLVd0eXBlLWxpbWl0cyAtV2Zvcm1hdC1zZWN1cml0eSAt V2Zvcm1hdC15MmsgLVdpbml0LXNlbGYgLVdpZ25vcmVkLXF1YWxpZmllcnMgLVdlbXB0eS1ib2R5 IC1XbmVzdGVkLWV4dGVybnMgLVdlbmRpZi1sYWJlbHMgLVdleHBhbnNpb24tdG8tZGVmaW5lZCAt V2ltcGxpY2l0LWZhbGx0aHJvdWdoPTIgLVduby1taXNzaW5nLWluY2x1ZGUtZGlycyAtV25vLXNo aWZ0LW5lZ2F0aXZlLXZhbHVlIC1Xbm8tcHNhYmkgLWZzdGFjay1wcm90ZWN0b3Itc3Ryb25nIC1m UElDIC1pc3lzdGVtLi4vLi4vbGludXgtaGVhZGVycyAtaXN5c3RlbWxpbnV4LWhlYWRlcnMgLURO RUVEX0NQVV9IICctRENPTkZJR19UQVJHRVQ9InBwYzY0LXNvZnRtbXUtY29uZmlnLXRhcmdldC5o IicgJy1EQ09ORklHX0RFVklDRVM9InBwYzY0LXNvZnRtbXUtY29uZmlnLWRldmljZXMuaCInIC1N RCAtTVEgbGlicWVtdS1wcGM2NC1zb2Z0bW11LmZhLnAvaHdfdmZpb19wY2ktcXVpcmtzLmMubyAt TUYgbGlicWVtdS1wcGM2NC1zb2Z0bW11LmZhLnAvaHdfdmZpb19wY2ktcXVpcmtzLmMuby5kIC1v IGxpYnFlbXUtcHBjNjQtc29mdG1tdS5mYS5wL2h3X3ZmaW9fcGNpLXF1aXJrcy5jLm8gLWMgLi4v Li4vaHcvdmZpby9wY2ktcXVpcmtzLmMKLi4vLi4vaHcvdmZpby9wY2ktcXVpcmtzLmM6IEluIGZ1 bmN0aW9uIOKAmHZmaW9fcGNpX252aWRpYV92MTAwX3JhbV9pbml04oCZOgouLi8uLi9ody92Zmlv L3BjaS1xdWlya3MuYzoxNTk3OjM2OiBlcnJvcjog4oCYVkZJT19SRUdJT05fU1VCVFlQRV9OVklE SUFfTlZMSU5LMl9SQU3igJkgdW5kZWNsYXJlZCAoZmlyc3QgdXNlIGluIHRoaXMgZnVuY3Rpb24p OyBkaWQgeW91IG1lYW4g4oCYVkZJT19SRUdJT05fU1VCVFlQRV9DQ1dfQVNZTkNfQ01E4oCZPwog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWRklPX1JFR0lPTl9TVUJUWVBFX05W SURJQV9OVkxJTksyX1JBTSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgXn5+ fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4KICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgVkZJT19SRUdJT05fU1VCVFlQRV9DQ1dfQVNZTkNfQ01ECi4uLy4uL2h3 L3ZmaW8vcGNpLXF1aXJrcy5jOjE1OTc6MzY6IG5vdGU6IGVhY2ggdW5kZWNsYXJlZCBpZGVudGlm aWVyIGlzIHJlcG9ydGVkIG9ubHkgb25jZSBmb3IgZWFjaCBmdW5jdGlvbiBpdCBhcHBlYXJzIGlu Ci4uLy4uL2h3L3ZmaW8vcGNpLXF1aXJrcy5jOjE2MDM6NDQ6IGVycm9yOiDigJhWRklPX1JFR0lP Tl9JTkZPX0NBUF9OVkxJTksyX1NTQVRHVOKAmSB1bmRlY2xhcmVkIChmaXJzdCB1c2UgaW4gdGhp cyBmdW5jdGlvbik7IGRpZCB5b3UgbWVhbiDigJhWRklPX1JFR0lPTl9JTkZPX0NBUF9TUEFSU0Vf TU1BUOKAmT8KICAgICBoZHIgPSB2ZmlvX2dldF9yZWdpb25faW5mb19jYXAobnYycmVnLCBWRklP X1JFR0lPTl9JTkZPX0NBUF9OVkxJTksyX1NTQVRHVCk7CiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgXn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn4K ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWRklPX1JFR0lPTl9J TkZPX0NBUF9TUEFSU0VfTU1BUAouLi8uLi9ody92ZmlvL3BjaS1xdWlya3MuYzoxNjI0OjQ5OiBl cnJvcjogZGVyZWZlcmVuY2luZyBwb2ludGVyIHRvIGluY29tcGxldGUgdHlwZSDigJhzdHJ1Y3Qg dmZpb19yZWdpb25faW5mb19jYXBfbnZsaW5rMl9zc2F0Z3TigJkKICAgICAgICAgICAgICAgICAg ICAgICAgICh2b2lkICopICh1aW50cHRyX3QpIGNhcC0+dGd0KTsKICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+Ci4uLy4uL2h3L3ZmaW8vcGNpLXF1aXJr cy5jOiBJbiBmdW5jdGlvbiDigJh2ZmlvX3BjaV9udmxpbmsyX2luaXTigJk6Ci4uLy4uL2h3L3Zm aW8vcGNpLXF1aXJrcy5jOjE2NDY6MzY6IGVycm9yOiDigJhWRklPX1JFR0lPTl9TVUJUWVBFX0lC TV9OVkxJTksyX0FUU0TigJkgdW5kZWNsYXJlZCAoZmlyc3QgdXNlIGluIHRoaXMgZnVuY3Rpb24p OyBkaWQgeW91IG1lYW4g4oCYVkZJT19SRUdJT05fU1VCVFlQRV9DQ1dfQVNZTkNfQ01E4oCZPwog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWRklPX1JFR0lPTl9TVUJUWVBFX0lC TV9OVkxJTksyX0FUU0QsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+ fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICBWRklPX1JFR0lPTl9TVUJUWVBFX0NDV19BU1lOQ19DTUQKLi4vLi4vaHcvdmZp by9wY2ktcXVpcmtzLmM6MTY1MzozNjogZXJyb3I6IOKAmFZGSU9fUkVHSU9OX0lORk9fQ0FQX05W TElOSzJfU1NBVEdU4oCZIHVuZGVjbGFyZWQgKGZpcnN0IHVzZSBpbiB0aGlzIGZ1bmN0aW9uKTsg ZGlkIHlvdSBtZWFuIOKAmFZGSU9fUkVHSU9OX0lORk9fQ0FQX1NQQVJTRV9NTUFQ4oCZPwogICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBWRklPX1JFR0lPTl9JTkZPX0NBUF9OVkxJ TksyX1NTQVRHVCk7CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+fn5+fn5+ fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIFZGSU9fUkVHSU9OX0lORk9fQ0FQX1NQQVJTRV9NTUFQCi4uLy4uL2h3L3ZmaW8vcGNp LXF1aXJrcy5jOjE2NjE6MzY6IGVycm9yOiDigJhWRklPX1JFR0lPTl9JTkZPX0NBUF9OVkxJTksy X0xOS1NQROKAmSB1bmRlY2xhcmVkIChmaXJzdCB1c2UgaW4gdGhpcyBmdW5jdGlvbik7IGRpZCB5 b3UgbWVhbiDigJhWRklPX1JFR0lPTl9JTkZPX0NBUF9TUEFSU0VfTU1BUOKAmT8KICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgVkZJT19SRUdJT05fSU5GT19DQVBfTlZMSU5LMl9M TktTUEQpOwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+fn5+ fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBWRklPX1JFR0lPTl9JTkZPX0NBUF9TUEFSU0VfTU1BUAouLi8uLi9ody92ZmlvL3BjaS1xdWly a3MuYzoxNjg1OjUyOiBlcnJvcjogZGVyZWZlcmVuY2luZyBwb2ludGVyIHRvIGluY29tcGxldGUg dHlwZSDigJhzdHJ1Y3QgdmZpb19yZWdpb25faW5mb19jYXBfbnZsaW5rMl9zc2F0Z3TigJkKICAg ICAgICAgICAgICAgICAgICAgICAgICh2b2lkICopICh1aW50cHRyX3QpIGNhcHRndC0+dGd0KTsK ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIF5+Ci4u Ly4uL2h3L3ZmaW8vcGNpLXF1aXJrcy5jOjE2OTE6NTQ6IGVycm9yOiBkZXJlZmVyZW5jaW5nIHBv aW50ZXIgdG8gaW5jb21wbGV0ZSB0eXBlIOKAmHN0cnVjdCB2ZmlvX3JlZ2lvbl9pbmZvX2NhcF9u dmxpbmsyX2xua3NwZOKAmQogICAgICAgICAgICAgICAgICAgICAgICAgKHZvaWQgKikgKHVpbnRw dHJfdCkgY2Fwc3BlZWQtPmxpbmtfc3BlZWQpOwogICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICBefgoKPiBBbmQgaWYgd2UgcHV0IHRoZSB1YXBpIC5o IGZpbGUgc3R1ZmYgYmFjaywgaXMgdGhhdCBzdWZmaWNpZW50IGZvciBxZW11Cj4gdG8gd29yaywg YXMgaXQgc2hvdWxkIGJlIGNoZWNraW5nIGF0IHJ1bnRpbWUgd2hhdCB0aGUga2VybmVsIGhhcyAv IGhhcwo+IG5vdCBhbnl3YXksIHJpZ2h0Pwo+IAoKUmlnaHQuIFRoaXMgd2lsbCBqdXN0IGJlIGRl YWQgY29kZSBpbiBRRU1VIGZvciBuZXdlciBrZXJuZWxzLgoKQW55d2F5LCBhcyBzYWlkIGluIHNv bWUgb3RoZXIgbWFpbCwgaXQgaXMgcHJvYmFibHkgdGltZSBmb3IgUUVNVSB0bwpzdGFydCBkZXBy ZWNhdGluZyB0aGlzIGNvZGUgYXMgd2VsbC4KCj4gdGhhbmtzLAo+IAo+IGdyZWcgay1oCgpfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpkcmktZGV2ZWwgbWFp bGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5m cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9kcmktZGV2ZWwK