From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EEA1C4363D for ; Fri, 25 Sep 2020 15:29:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 17BF8235F9 for ; Fri, 25 Sep 2020 15:29:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ExCmKxeP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729400AbgIYP3d (ORCPT ); Fri, 25 Sep 2020 11:29:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:59088 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728801AbgIYP3c (ORCPT ); Fri, 25 Sep 2020 11:29:32 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601047771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p7Afe48pkEQZLqWPSXWRulCj196P+48pQyU4ztEKPbI=; b=ExCmKxePDxbZ5FJ4NYvz+U4pMum66vt1nGiAGB4Yr9GAYRs4ncKEwUuFBMFlNI4M3GKoHD yYAhSYAaf//wld4AjsRji1BD0gn+z2dHZw1pmczyHg0F+TsnPR0qzErhJdEXFBS95cqekK DwAQ7svzKraioPgeDvtY6Y0KqQcfOdc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-CloU21ZHNWSunAjJZOKQ-g-1; Fri, 25 Sep 2020 11:29:26 -0400 X-MC-Unique: CloU21ZHNWSunAjJZOKQ-g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 19D6B10082EB; Fri, 25 Sep 2020 15:29:22 +0000 (UTC) Received: from ovpn-66-87.rdu2.redhat.com (unknown [10.10.67.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4371D1002382; Fri, 25 Sep 2020 15:29:14 +0000 (UTC) Message-ID: <3c12bdec2c4ecdabcccd9ece3d495d792e9fc231.camel@redhat.com> Subject: Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI From: Qian Cai To: Thomas Gleixner , LKML , Stephen Rothwell , linux-next@vger.kernel.org Cc: x86@kernel.org, Joerg Roedel , iommu@lists.linux-foundation.org, linux-hyperv@vger.kernel.org, Haiyang Zhang , Jon Derrick , Lu Baolu , Wei Liu , "K. Y. Srinivasan" , Stephen Hemminger , Steve Wahl , Dimitri Sivanich , Russ Anderson , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Konrad Rzeszutek Wilk , xen-devel@lists.xenproject.org, Juergen Gross , Boris Ostrovsky , Stefano Stabellini , Marc Zyngier , Greg Kroah-Hartman , "Rafael J. Wysocki" , Megha Dey , Jason Gunthorpe , Dave Jiang , Alex Williamson , Jacob Pan , Baolu Lu , Kevin Tian , Dan Williams Date: Fri, 25 Sep 2020 11:29:13 -0400 In-Reply-To: <20200826111628.794979401@linutronix.de> References: <20200826111628.794979401@linutronix.de> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2020-08-26 at 13:16 +0200, Thomas Gleixner wrote: > This is the second version of providing a base to support device MSI (non > PCI based) and on top of that support for IMS (Interrupt Message Storm) > based devices in a halfways architecture independent way. > > The first version can be found here: > > https://lore.kernel.org/r/20200821002424.119492231@linutronix.de > > It's still a mixed bag of bug fixes, cleanups and general improvements > which are worthwhile independent of device MSI. Reverting the part of this patchset on the top of today's linux-next fixed an boot issue on HPE ProLiant DL560 Gen10, i.e., $ git revert --no-edit 13b90cadfc29..bc95fd0d7c42 .config: https://gitlab.com/cailca/linux-mm/-/blob/master/x86.config It looks like the crashes happen in the interrupt remapping code where they are only able to to generate partial call traces. [ 1.912386][ T0] ACPI: X2APIC_NMI (uid[0xf5] high level 9983][ T0] ... MAX_LOCK_DEPTH: 48 [ 7.914876][ T0] ... MAX_LOCKDEP_KEYS: 8192 [ 7.919942][ T0] ... CLASSHASH_SIZE: 4096 [ 7.925009][ T0] ... MAX_LOCKDEP_ENTRIES: 32768 [ 7.930163][ T0] ... MAX_LOCKDEP_CHAINS: 65536 [ 7.935318][ T0] ... CHAINHASH_SIZE: 32768 [ 7.940473][ T0] memory used by lock dependency info: 6301 kB [ 7.946586][ T0] memory used for stack traces: 4224 kB [ 7.952088][ T0] per task-struct memory footprint: 1920 bytes [ 7.968312][ T0] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 7.980281][ T0] ACPI: Core revision 20200717 [ 7.993343][ T0] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns [ 8.003270][ T0] APIC: Switch to symmetric I/O mode setup [ 8.008951][ T0] DMAR: Host address width 46 [ 8.013512][ T0] DMAR: DRHD base: 0x000000e5ffc000 flags: 0x0 [ 8.019680][ T0] DMAR: dmar0: reg_base_addr e5ffc000 ver 1:0 cap 8d2078c106f0466 [ T0] DMAR-IR: IOAPIC id 15 under DRHD base 0xe5ffc000 IOMMU 0 [ 8.420990][ T0] DMAR-IR: IOAPIC id 8 under DRHD base 0xddffc000 IOMMU 15 [ 8.428166][ T0] DMAR-IR: IOAPIC id 9 under DRHD base 0xddffc000 IOMMU 15 [ 8.435341][ T0] DMAR-IR: HPET id 0 under DRHD base 0xddffc000 [ 8.441456][ T0] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 8.457911][ T0] DMAR-IR: Enabled IRQ remapping in x2apic mode [ 8.466614][ T0] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 8.474295][ T0] #PF: supervisor instruction fetch in kernel mode [ 8.480669][ T0] #PF: error_code(0x0010) - not-present page [ 8.486518][ T0] PGD 0 P4D 0 [ 8.489757][ T0] Oops: 0010 [#1] SMP KASAN PTI [ 8.494476][ T0] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G I 5.9.0-rc6-next-20200925 #2 [ 8.503987][ T0] Hardware name: HPE ProLiant DL560 Gen10/ProLiant DL560 Gen10, BIOS U34 11/13/2019 [ 8.513238][ T0] RIP: 0010:0x0 [ 8.516562][ T0] Code: Bad RIP v or [ 2.906744][ T0] ACPI: X2API32, address 0xfec68000, GSI 128-135 [ 2.907063][ T0] IOAPIC[15]: apic_id 29, version 32, address 0xfec70000, GSI 136-143 [ 2.907071][ T0] IOAPIC[16]: apic_id 30, version 32, address 0xfec78000, GSI 144-151 [ 2.907079][ T0] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 2.907084][ T0] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 2.907100][ T0] Using ACPI (MADT) for SMP configuration information [ 2.907105][ T0] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 2.907116][ T0] ACPI: SPCR: console: uart,mmio,0x0,115200 [ 2.907121][ T0] TSC deadline timer available [ 2.907126][ T0] smpboot: Allowing 144 CPUs, 0 hotplug CPUs [ 2.907163][ T0] [mem 0xd0000000-0xfdffffff] available for PCI devices [ 2.907175][ T0] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 2.914541][ T0] setup_percpu: NR_CPUS:256 nr_cpumask_bits:144 nr_cpu_ids:144 nr_node_ids:4 [ 2.926109][ 466 ecap f020df [ 9.134709][ T0] DMAR: DRHD base: 0x000000f5ffc000 flags: 0x0 [ 9.140867][ T0] DMAR: dmar8: reg_base_addr f5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.149610][ T0] DMAR: DRHD base: 0x000000f7ffc000 flags: 0x0 [ 9.155762][ T0] DMAR: dmar9: reg_base_addr f7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.164491][ T0] DMAR: DRHD base: 0x000000f9ffc000 flags: 0x0 [ 9.170645][ T0] DMAR: dmar10: reg_base_addr f9ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.179476][ T0] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0 [ 9.185626][ T0] DMAR: dmar11: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.194442][ T0] DMAR: DRHD base: 0x000000dfffc000 flags: 0x0 [ 9.200587][ T0] DMAR: dmar12: reg_base_addr dfffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.209418][ T0] DMAR: DRHD base: 0x000000e1ffc000 flags: 0x0 [ 9.215551][ T0] DMAR: dmar13: reg_base_addr e1ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.224367][ T0] DMAR: DRHD base: 0x000000e3ffc83][ T0] msi_domain_alloc+0x8e/0x280 [ 9.615015][ T0] __irq_domain_a8992cd [ 9.711906][ T0] R10: ffffffff85407d78 R11: fffffbfff18992cc R12: ffffffff8546ffc0 [ 9.719761][ T0] R13: 0000000000000098 R14: ffff888106e63a40 R15: 0000000000000001 [ 9.727617][ T0] FS: 0000000000000000(0000) GS:ffff8887df800000(0000) knlGS:0000000000000000 [ 9.736431][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 9.742892][ T0] CR2: ffffffffffffffd6 CR3: 0000001ba7814001 CR4: 00000000000606b0 [ 9.750747][ T0] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 9.758601][ T0] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 9.766456][ T0] Kernel panic - not syncing: Fatal exception [ 9.772547][ T0] ---[ end Kernel panic - not syncing: Fatal exception ]--- The working boot (without those patches) looks like this: [ 1.913963][ T0] ACPI: X2APIC_NMI (uid[0xf4] high level lint[0x1]) [ 1.913967][ T0] ACPI: X2APIC_NMI (uid[0xf5] high level lint[0x1]) [ 1.913970][ T0] ACPI: X2APIC_NMI (uid[0xf6] high level lint[0x1]) [ 1.913974][ T0] ACPI: X2APIC_NMI (uid[0xf7] high level lint[0x1]) [ 1.914017][ T0] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 [ 1.914032][ T0] IOAPIC[1]: apic_id 9, version 32, address 0xfec01000, GSI 24-31 [ 1.914039][ T0] IOAPIC[2]: apic_id 10, version 32, address 0xfec08000, GSI 32-39 [ 1.914047][ T0] IOAPIC[3]: apic_id 11, version 32, address 0xfec10000, GSI 40-47 [ 1.914054][ T0] IOAPIC[4]: apic_id 12, version 32, address 0xfec18000, GSI 48-55 [ 1.914062][ T0] IOAPIC[5]: apic_id 15, version 32, address 0xfec20000, GSI 56-63 [ 1.[ 7.994567][ T0] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 8.006541][ T0] ACPI: Core revision 20200717 [ 8.019713][ T0] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns [ 8.029672][ T0] APIC: Switch to symmetric I/O mode setup [ 8.035354][ T0] DMAR: Host address width 46 [ 8.039915][ T0] DMAR: DRHD base: 0x000000e5ffc000 flags: 0x0 [ 8.046095][ T0] DMAR: dmar0: reg_base_addr e5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 8.054840][ T0] DMAR: DRHD base: 0x000000e7ffc000 flags: 0x0 [ 8.060997][ T0] DMAR: dmar1: reg_base_addr e7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 8.069740][ T0] DMAR: DRHD base: 0x000000e9ffc000 flags: 0x0 [ 8.075872][ T0] DMAR: dmar2: reg_base_addr e9ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 8.084615][ T0] DMAR: DRHD base: 0x000000ebffc000 flags: 0x0 [ 8.090761][ T0] DMAR: dmar3: reg_base_addr ebffc000 ver 1:0 cap 8d2078c106f0466 ecap fMAR-IR: Enabled IRQ remapping in x2apic mode [ 8.513491][ T0] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 8.568289][ T0] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2b3e459bf4c, max_idle_ns: 440795289890 ns [ 8.579576][ T0] Calibrating delay loop (skipped), value calculated using timer frequency.. 6000.00 BogoMIPS (lpj=30000000) [ 8.589574][ T0] pid_max: default: 147456 minimum: 1152 [ 8.714025][ T0] efi: memattr: Entry attributes invalid: RO and XP bits both cleared [ 8.719577][ T0] efi: memattr: ! 0x0000a057a000-0x0000a05b4fff [Runtime Code |RUN| | | | | | | | | | | | ] [ 8.775355][ T0] Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes, vmalloc) [ 8.798868][ T0] Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes, vmalloc) [ 8.811550][ T0] Mount-cache hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 8.820076][ T0] Mountpoint-cache hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 8.879327][ T0] mce: CPU0: Thermal mo[ 8.996916][ T1] Performance Events: PEBS fmt3+, Skylake events, 32-deep LBR, full-width counters, Intel PMU driver. [ 8.999591][ T1] ... version: 4 [ 9.004310][ T1] ... bit width: 48 [ 9.009118][ T1] ... generic registers: 4 [ 9.009574][ T1] ... value mask: 0000ffffffffffff [ 9.015601][ T1] ... max period: 00007fffffffffff [ 9.019574][ T1] ... fixed-purpose events: 3 [ 9.024294][ T1] ... event mask: 000000070000000f [ 9.034357][ T1] rcu: Hierarchical SRCU implementation. [ 9.062516][ T5] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. > > There are quite a bunch of issues to solve: > > - X86 does not use the device::msi_domain pointer for historical reasons > and due to XEN, which makes it impossible to create an architecture > agnostic device MSI infrastructure. > > - X86 has it's own msi_alloc_info data type which is pointlessly > different from the generic version and does not allow to share code. > > - The logic of composing MSI messages in an hierarchy is busted at the > core level and of course some (x86) drivers depend on that. > > - A few minor shortcomings as usual > > This series addresses that in several steps: > > 1) Accidental bug fixes > > iommu/amd: Prevent NULL pointer dereference > > 2) Janitoring > > x86/init: Remove unused init ops > PCI: vmd: Dont abuse vector irqomain as parent > x86/msi: Remove pointless vcpu_affinity callback > > 3) Sanitizing the composition of MSI messages in a hierarchy > > genirq/chip: Use the first chip in irq_chip_compose_msi_msg() > x86/msi: Move compose message callback where it belongs > > 4) Simplification of the x86 specific interrupt allocation mechanism > > x86/irq: Rename X86_IRQ_ALLOC_TYPE_MSI* to reflect PCI dependency > x86/irq: Add allocation type for parent domain retrieval > iommu/vt-d: Consolidate irq domain getter > iommu/amd: Consolidate irq domain getter > iommu/irq_remapping: Consolidate irq domain lookup > > 5) Consolidation of the X86 specific interrupt allocation mechanism to be as > close > as possible to the generic MSI allocation mechanism which allows to get > rid > of quite a bunch of x86'isms which are pointless > > x86/irq: Prepare consolidation of irq_alloc_info > x86/msi: Consolidate HPET allocation > x86/ioapic: Consolidate IOAPIC allocation > x86/irq: Consolidate DMAR irq allocation > x86/irq: Consolidate UV domain allocation > PCI/MSI: Rework pci_msi_domain_calc_hwirq() > x86/msi: Consolidate MSI allocation > x86/msi: Use generic MSI domain ops > > 6) x86 specific cleanups to remove the dependency on arch_*_msi_irqs() > > x86/irq: Move apic_post_init() invocation to one place > x86/pci: Reducde #ifdeffery in PCI init code > x86/irq: Initialize PCI/MSI domain at PCI init time > irqdomain/msi: Provide DOMAIN_BUS_VMD_MSI > PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI > PCI/MSI: Provide pci_dev_has_special_msi_domain() helper > x86/xen: Make xen_msi_init() static and rename it to xen_hvm_msi_init() > x86/xen: Rework MSI teardown > x86/xen: Consolidate XEN-MSI init > irqdomain/msi: Allow to override msi_domain_alloc/free_irqs() > x86/xen: Wrap XEN MSI management into irqdomain > iommm/vt-d: Store irq domain in struct device > iommm/amd: Store irq domain in struct device > x86/pci: Set default irq domain in pcibios_add_device() > PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectable > x86/irq: Cleanup the arch_*_msi_irqs() leftovers > x86/irq: Make most MSI ops XEN private > iommu/vt-d: Remove domain search for PCI/MSI[X] > iommu/amd: Remove domain search for PCI/MSI > > 7) X86 specific preparation for device MSI > > x86/irq: Add DEV_MSI allocation type > x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI > > 8) Generic device MSI infrastructure > platform-msi: Provide default irq_chip:: Ack > genirq/proc: Take buslock on affinity write > genirq/msi: Provide and use msi_domain_set_default_info_flags() > platform-msi: Add device MSI infrastructure > irqdomain/msi: Provide msi_alloc/free_store() callbacks > > 9) POC of IMS (Interrupt Message Storm) irq domain and irqchip > implementations for both device array and queue storage. > > irqchip: Add IMS (Interrupt Message Storm) driver - NOT FOR MERGING > > Changes vs. V1: > > - Addressed various review comments and addressed the 0day fallout. > - Corrected the XEN logic (Jürgen) > - Make the arch fallback in PCI/MSI opt-in not opt-out (Bjorn) > > - Fixed the compose MSI message inconsistency > > - Ensure that the necessary flags are set for device SMI > > - Make the irq bus logic work for affinity setting to prepare > support for IMS storage in queue memory. It turned out to be > less scary than I feared. > > - Remove leftovers in iommu/intel|amd > > - Reworked the IMS POC driver to cover queue storage so Jason can have a > look whether that fits the needs of MLX devices. > > The whole lot is also available from git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git device-msi > > This has been tested on Intel/AMD/KVM but lacks testing on: > > - HYPERV (-ENODEV) > - VMD enabled systems (-ENODEV) > - XEN (-ENOCLUE) > - IMS (-ENODEV) > > - Any non-X86 code which might depend on the broken compose MSI message > logic. Marc excpects not much fallout, but agrees that we need to fix > it anyway. > > #1 - #3 should be applied unconditionally for obvious reasons > #4 - #6 are wortwhile cleanups which should be done independent of device MSI > > #7 - #8 look promising to cleanup the platform MSI implementation > independent of #8, but I neither had cycles nor the stomach to > tackle that. > > #9 is obviously just for the folks interested in IMS > > Thanks, > > tglx From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABA23C4727C for ; Fri, 25 Sep 2020 15:29:40 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3606623718 for ; Fri, 25 Sep 2020 15:29:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ExCmKxeP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3606623718 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id B8BEA20480; Fri, 25 Sep 2020 15:29:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WeIyquUjhz6G; Fri, 25 Sep 2020 15:29:35 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by silver.osuosl.org (Postfix) with ESMTP id C1B132E16B; Fri, 25 Sep 2020 15:29:35 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9DBC5C0859; Fri, 25 Sep 2020 15:29:35 +0000 (UTC) Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id EAB82C0051 for ; Fri, 25 Sep 2020 15:29:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D90CF875F2 for ; Fri, 25 Sep 2020 15:29:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cQ21xAjLDQd4 for ; Fri, 25 Sep 2020 15:29:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by hemlock.osuosl.org (Postfix) with ESMTPS id 348C3875EB for ; Fri, 25 Sep 2020 15:29:32 +0000 (UTC) Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601047771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p7Afe48pkEQZLqWPSXWRulCj196P+48pQyU4ztEKPbI=; b=ExCmKxePDxbZ5FJ4NYvz+U4pMum66vt1nGiAGB4Yr9GAYRs4ncKEwUuFBMFlNI4M3GKoHD yYAhSYAaf//wld4AjsRji1BD0gn+z2dHZw1pmczyHg0F+TsnPR0qzErhJdEXFBS95cqekK DwAQ7svzKraioPgeDvtY6Y0KqQcfOdc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-CloU21ZHNWSunAjJZOKQ-g-1; Fri, 25 Sep 2020 11:29:26 -0400 X-MC-Unique: CloU21ZHNWSunAjJZOKQ-g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 19D6B10082EB; Fri, 25 Sep 2020 15:29:22 +0000 (UTC) Received: from ovpn-66-87.rdu2.redhat.com (unknown [10.10.67.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4371D1002382; Fri, 25 Sep 2020 15:29:14 +0000 (UTC) Message-ID: <3c12bdec2c4ecdabcccd9ece3d495d792e9fc231.camel@redhat.com> Subject: Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI From: Qian Cai To: Thomas Gleixner , LKML , Stephen Rothwell , linux-next@vger.kernel.org Date: Fri, 25 Sep 2020 11:29:13 -0400 In-Reply-To: <20200826111628.794979401@linutronix.de> References: <20200826111628.794979401@linutronix.de> Mime-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Cc: Dimitri Sivanich , linux-hyperv@vger.kernel.org, Steve Wahl , linux-pci@vger.kernel.org, "K. Y. Srinivasan" , Dan Williams , Wei Liu , Stephen Hemminger , Baolu Lu , Marc Zyngier , x86@kernel.org, Jason Gunthorpe , Megha Dey , xen-devel@lists.xenproject.org, Kevin Tian , Konrad Rzeszutek Wilk , Haiyang Zhang , Alex Williamson , Stefano Stabellini , Bjorn Helgaas , Dave Jiang , Boris Ostrovsky , Jon Derrick , Juergen Gross , Russ Anderson , Greg Kroah-Hartman , iommu@lists.linux-foundation.org, Jacob Pan , "Rafael J. Wysocki" X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gV2VkLCAyMDIwLTA4LTI2IGF0IDEzOjE2ICswMjAwLCBUaG9tYXMgR2xlaXhuZXIgd3JvdGU6 Cj4gVGhpcyBpcyB0aGUgc2Vjb25kIHZlcnNpb24gb2YgcHJvdmlkaW5nIGEgYmFzZSB0byBzdXBw b3J0IGRldmljZSBNU0kgKG5vbgo+IFBDSSBiYXNlZCkgYW5kIG9uIHRvcCBvZiB0aGF0IHN1cHBv cnQgZm9yIElNUyAoSW50ZXJydXB0IE1lc3NhZ2UgU3Rvcm0pCj4gYmFzZWQgZGV2aWNlcyBpbiBh IGhhbGZ3YXlzIGFyY2hpdGVjdHVyZSBpbmRlcGVuZGVudCB3YXkuCj4gCj4gVGhlIGZpcnN0IHZl cnNpb24gY2FuIGJlIGZvdW5kIGhlcmU6Cj4gCj4gICAgIGh0dHBzOi8vbG9yZS5rZXJuZWwub3Jn L3IvMjAyMDA4MjEwMDI0MjQuMTE5NDkyMjMxQGxpbnV0cm9uaXguZGUKPiAKPiBJdCdzIHN0aWxs IGEgbWl4ZWQgYmFnIG9mIGJ1ZyBmaXhlcywgY2xlYW51cHMgYW5kIGdlbmVyYWwgaW1wcm92ZW1l bnRzCj4gd2hpY2ggYXJlIHdvcnRod2hpbGUgaW5kZXBlbmRlbnQgb2YgZGV2aWNlIE1TSS4KClJl dmVydGluZyB0aGUgcGFydCBvZiB0aGlzIHBhdGNoc2V0IG9uIHRoZSB0b3Agb2YgdG9kYXkncyBs aW51eC1uZXh0IGZpeGVkIGFuCmJvb3QgaXNzdWUgb24gSFBFIFByb0xpYW50IERMNTYwIEdlbjEw LCBpLmUuLAoKJCBnaXQgcmV2ZXJ0IC0tbm8tZWRpdCAxM2I5MGNhZGZjMjkuLmJjOTVmZDBkN2M0 MgoKLmNvbmZpZzogaHR0cHM6Ly9naXRsYWIuY29tL2NhaWxjYS9saW51eC1tbS8tL2Jsb2IvbWFz dGVyL3g4Ni5jb25maWcKCkl0IGxvb2tzIGxpa2UgdGhlIGNyYXNoZXMgaGFwcGVuIGluIHRoZSBp bnRlcnJ1cHQgcmVtYXBwaW5nIGNvZGUgd2hlcmUgdGhleSBhcmUKb25seSBhYmxlIHRvIHRvIGdl bmVyYXRlIHBhcnRpYWwgY2FsbCB0cmFjZXMuCgpbICAgIDEuOTEyMzg2XVsgICAgVDBdIEFDUEk6 IFgyQVBJQ19OTUkgKHVpZFsweGY1XSBoaWdoIGxldmVsIDk5ODNdWyAgICBUMF0gLi4uIE1BWF9M T0NLX0RFUFRIOiAgICAgICAgICA0OApbICAgIDcuOTE0ODc2XVsgICAgVDBdIC4uLiBNQVhfTE9D S0RFUF9LRVlTOiAgICAgICAgODE5MgpbICAgIDcuOTE5OTQyXVsgICAgVDBdIC4uLiBDTEFTU0hB U0hfU0laRTogICAgICAgICAgNDA5NgpbICAgIDcuOTI1MDA5XVsgICAgVDBdIC4uLiBNQVhfTE9D S0RFUF9FTlRSSUVTOiAgICAgMzI3NjgKWyAgICA3LjkzMDE2M11bICAgIFQwXSAuLi4gTUFYX0xP Q0tERVBfQ0hBSU5TOiAgICAgIDY1NTM2ClsgICAgNy45MzUzMThdWyAgICBUMF0gLi4uIENIQUlO SEFTSF9TSVpFOiAgICAgICAgICAzMjc2OApbICAgIDcuOTQwNDczXVsgICAgVDBdICBtZW1vcnkg dXNlZCBieSBsb2NrIGRlcGVuZGVuY3kgaW5mbzogNjMwMSBrQgpbICAgIDcuOTQ2NTg2XVsgICAg VDBdICBtZW1vcnkgdXNlZCBmb3Igc3RhY2sgdHJhY2VzOiA0MjI0IGtCClsgICAgNy45NTIwODhd WyAgICBUMF0gIHBlciB0YXNrLXN0cnVjdCBtZW1vcnkgZm9vdHByaW50OiAxOTIwIGJ5dGVzClsg ICAgNy45NjgzMTJdWyAgICBUMF0gbWVtcG9saWN5OiBFbmFibGluZyBhdXRvbWF0aWMgTlVNQSBi YWxhbmNpbmcuIENvbmZpZ3VyZSB3aXRoIG51bWFfYmFsYW5jaW5nPSBvciB0aGUga2VybmVsLm51 bWFfYmFsYW5jaW5nIHN5c2N0bApbICAgIDcuOTgwMjgxXVsgICAgVDBdIEFDUEk6IENvcmUgcmV2 aXNpb24gMjAyMDA3MTcKWyAgICA3Ljk5MzM0M11bICAgIFQwXSBjbG9ja3NvdXJjZTogaHBldDog bWFzazogMHhmZmZmZmZmZiBtYXhfY3ljbGVzOiAweGZmZmZmZmZmLCBtYXhfaWRsZV9uczogNzk2 MzU4NTUyNDUgbnMKWyAgICA4LjAwMzI3MF1bICAgIFQwXSBBUElDOiBTd2l0Y2ggdG8gc3ltbWV0 cmljIEkvTyBtb2RlIHNldHVwClsgICAgOC4wMDg5NTFdWyAgICBUMF0gRE1BUjogSG9zdCBhZGRy ZXNzIHdpZHRoIDQ2ClsgICAgOC4wMTM1MTJdWyAgICBUMF0gRE1BUjogRFJIRCBiYXNlOiAweDAw MDAwMGU1ZmZjMDAwIGZsYWdzOiAweDAKWyAgICA4LjAxOTY4MF1bICAgIFQwXSBETUFSOiBkbWFy MDogcmVnX2Jhc2VfYWRkciBlNWZmYzAwMCB2ZXIgMTowIGNhcCA4ZDIwNzhjMTA2ZjA0NjYgWyAg ICBUMF0gRE1BUi1JUjogSU9BUElDIGlkIDE1IHVuZGVyIERSSEQgYmFzZSAgMHhlNWZmYzAwMCBJ T01NVSAwClsgICAgOC40MjA5OTBdWyAgICBUMF0gRE1BUi1JUjogSU9BUElDIGlkIDggdW5kZXIg RFJIRCBiYXNlICAweGRkZmZjMDAwIElPTU1VIDE1ClsgICAgOC40MjgxNjZdWyAgICBUMF0gRE1B Ui1JUjogSU9BUElDIGlkIDkgdW5kZXIgRFJIRCBiYXNlICAweGRkZmZjMDAwIElPTU1VIDE1Clsg ICAgOC40MzUzNDFdWyAgICBUMF0gRE1BUi1JUjogSFBFVCBpZCAwIHVuZGVyIERSSEQgYmFzZSAw eGRkZmZjMDAwClsgICAgOC40NDE0NTZdWyAgICBUMF0gRE1BUi1JUjogUXVldWVkIGludmFsaWRh dGlvbiB3aWxsIGJlIGVuYWJsZWQgdG8gc3VwcG9ydCB4MmFwaWMgYW5kIEludHItcmVtYXBwaW5n LgpbICAgIDguNDU3OTExXVsgICAgVDBdIERNQVItSVI6IEVuYWJsZWQgSVJRIHJlbWFwcGluZyBp biB4MmFwaWMgbW9kZQpbICAgIDguNDY2NjE0XVsgICAgVDBdIEJVRzoga2VybmVsIE5VTEwgcG9p bnRlciBkZXJlZmVyZW5jZSwgYWRkcmVzczogMDAwMDAwMDAwMDAwMDAwMApbICAgIDguNDc0Mjk1 XVsgICAgVDBdICNQRjogc3VwZXJ2aXNvciBpbnN0cnVjdGlvbiBmZXRjaCBpbiBrZXJuZWwgbW9k ZQpbICAgIDguNDgwNjY5XVsgICAgVDBdICNQRjogZXJyb3JfY29kZSgweDAwMTApIC0gbm90LXBy ZXNlbnQgcGFnZQpbICAgIDguNDg2NTE4XVsgICAgVDBdIFBHRCAwIFA0RCAwIApbICAgIDguNDg5 NzU3XVsgICAgVDBdIE9vcHM6IDAwMTAgWyMxXSBTTVAgS0FTQU4gUFRJClsgICAgOC40OTQ0NzZd WyAgICBUMF0gQ1BVOiAwIFBJRDogMCBDb21tOiBzd2FwcGVyLzAgVGFpbnRlZDogRyAgICAgICAg ICBJICAgICAgIDUuOS4wLXJjNi1uZXh0LTIwMjAwOTI1ICMyClsgICAgOC41MDM5ODddWyAgICBU MF0gSGFyZHdhcmUgbmFtZTogSFBFIFByb0xpYW50IERMNTYwIEdlbjEwL1Byb0xpYW50IERMNTYw IEdlbjEwLCBCSU9TIFUzNCAxMS8xMy8yMDE5ClsgICAgOC41MTMyMzhdWyAgICBUMF0gUklQOiAw MDEwOjB4MApbICAgIDguNTE2NTYyXVsgICAgVDBdIENvZGU6IEJhZCBSSVAgdgoKb3IKClsgICAg Mi45MDY3NDRdWyAgICBUMF0gQUNQSTogWDJBUEkzMiwgYWRkcmVzcyAweGZlYzY4MDAwLCBHU0kg MTI4LTEzNQpbICAgIDIuOTA3MDYzXVsgICAgVDBdIElPQVBJQ1sxNV06IGFwaWNfaWQgMjksIHZl cnNpb24gMzIsIGFkZHJlc3MgMHhmZWM3MDAwMCwgR1NJIDEzNi0xNDMKWyAgICAyLjkwNzA3MV1b ICAgIFQwXSBJT0FQSUNbMTZdOiBhcGljX2lkIDMwLCB2ZXJzaW9uIDMyLCBhZGRyZXNzIDB4ZmVj NzgwMDAsIEdTSSAxNDQtMTUxClsgICAgMi45MDcwNzldWyAgICBUMF0gQUNQSTogSU5UX1NSQ19P VlIgKGJ1cyAwIGJ1c19pcnEgMCBnbG9iYWxfaXJxIDIgZGZsIGRmbCkKWyAgICAyLjkwNzA4NF1b ICAgIFQwXSBBQ1BJOiBJTlRfU1JDX09WUiAoYnVzIDAgYnVzX2lycSA5IGdsb2JhbF9pcnEgOSBo aWdoIGxldmVsKQpbICAgIDIuOTA3MTAwXVsgICAgVDBdIFVzaW5nIEFDUEkgKE1BRFQpIGZvciBT TVAgY29uZmlndXJhdGlvbiBpbmZvcm1hdGlvbgpbICAgIDIuOTA3MTA1XVsgICAgVDBdIEFDUEk6 IEhQRVQgaWQ6IDB4ODA4NmE3MDEgYmFzZTogMHhmZWQwMDAwMApbICAgIDIuOTA3MTE2XVsgICAg VDBdIEFDUEk6IFNQQ1I6IGNvbnNvbGU6IHVhcnQsbW1pbywweDAsMTE1MjAwClsgICAgMi45MDcx MjFdWyAgICBUMF0gVFNDIGRlYWRsaW5lIHRpbWVyIGF2YWlsYWJsZQpbICAgIDIuOTA3MTI2XVsg ICAgVDBdIHNtcGJvb3Q6IEFsbG93aW5nIDE0NCBDUFVzLCAwIGhvdHBsdWcgQ1BVcwpbICAgIDIu OTA3MTYzXVsgICAgVDBdIFttZW0gMHhkMDAwMDAwMC0weGZkZmZmZmZmXSBhdmFpbGFibGUgZm9y IFBDSSBkZXZpY2VzClsgICAgMi45MDcxNzVdWyAgICBUMF0gY2xvY2tzb3VyY2U6IHJlZmluZWQt amlmZmllczogbWFzazogMHhmZmZmZmZmZiBtYXhfY3ljbGVzOiAweGZmZmZmZmZmLCBtYXhfaWRs ZV9uczogMTkxMTI2MDQ0NjI3NTAwMDAgbnMKWyAgICAyLjkxNDU0MV1bICAgIFQwXSBzZXR1cF9w ZXJjcHU6IE5SX0NQVVM6MjU2IG5yX2NwdW1hc2tfYml0czoxNDQgbnJfY3B1X2lkczoxNDQgbnJf bm9kZV9pZHM6NApbICAgIDIuOTI2MTA5XVsgICA0NjYgZWNhcCBmMDIwZGYKWyAgICA5LjEzNDcw OV1bICAgIFQwXSBETUFSOiBEUkhEIGJhc2U6IDB4MDAwMDAwZjVmZmMwMDAgZmxhZ3M6IDB4MApb ICAgIDkuMTQwODY3XVsgICAgVDBdIERNQVI6IGRtYXI4OiByZWdfYmFzZV9hZGRyIGY1ZmZjMDAw IHZlciAxOjAgY2FwIDhkMjA3OGMxMDZmMDQ2NiBlY2FwIGYwMjBkZgpbICAgIDkuMTQ5NjEwXVsg ICAgVDBdIERNQVI6IERSSEQgYmFzZTogMHgwMDAwMDBmN2ZmYzAwMCBmbGFnczogMHgwClsgICAg OS4xNTU3NjJdWyAgICBUMF0gRE1BUjogZG1hcjk6IHJlZ19iYXNlX2FkZHIgZjdmZmMwMDAgdmVy IDE6MCBjYXAgOGQyMDc4YzEwNmYwNDY2IGVjYXAgZjAyMGRmClsgICAgOS4xNjQ0OTFdWyAgICBU MF0gRE1BUjogRFJIRCBiYXNlOiAweDAwMDAwMGY5ZmZjMDAwIGZsYWdzOiAweDAKWyAgICA5LjE3 MDY0NV1bICAgIFQwXSBETUFSOiBkbWFyMTA6IHJlZ19iYXNlX2FkZHIgZjlmZmMwMDAgdmVyIDE6 MCBjYXAgOGQyMDc4YzEwNmYwNDY2IGVjYXAgZjAyMGRmClsgICAgOS4xNzk0NzZdWyAgICBUMF0g RE1BUjogRFJIRCBiYXNlOiAweDAwMDAwMGZiZmZjMDAwIGZsYWdzOiAweDAKWyAgICA5LjE4NTYy Nl1bICAgIFQwXSBETUFSOiBkbWFyMTE6IHJlZ19iYXNlX2FkZHIgZmJmZmMwMDAgdmVyIDE6MCBj YXAgOGQyMDc4YzEwNmYwNDY2IGVjYXAgZjAyMGRmClsgICAgOS4xOTQ0NDJdWyAgICBUMF0gRE1B UjogRFJIRCBiYXNlOiAweDAwMDAwMGRmZmZjMDAwIGZsYWdzOiAweDAKWyAgICA5LjIwMDU4N11b ICAgIFQwXSBETUFSOiBkbWFyMTI6IHJlZ19iYXNlX2FkZHIgZGZmZmMwMDAgdmVyIDE6MCBjYXAg OGQyMDc4YzEwNmYwNDY2IGVjYXAgZjAyMGRmClsgICAgOS4yMDk0MThdWyAgICBUMF0gRE1BUjog RFJIRCBiYXNlOiAweDAwMDAwMGUxZmZjMDAwIGZsYWdzOiAweDAKWyAgICA5LjIxNTU1MV1bICAg IFQwXSBETUFSOiBkbWFyMTM6IHJlZ19iYXNlX2FkZHIgZTFmZmMwMDAgdmVyIDE6MCBjYXAgOGQy MDc4YzEwNmYwNDY2IGVjYXAgZjAyMGRmClsgICAgOS4yMjQzNjddWyAgICBUMF0gRE1BUjogRFJI RCBiYXNlOiAweDAwMDAwMGUzZmZjODNdWyAgICBUMF0gIG1zaV9kb21haW5fYWxsb2MrMHg4ZS8w eDI4MApbICAgIDkuNjE1MDE1XVsgICAgVDBdICBfX2lycV9kb21haW5fYTg5OTJjZApbICAgIDku NzExOTA2XVsgICAgVDBdIFIxMDogZmZmZmZmZmY4NTQwN2Q3OCBSMTE6IGZmZmZmYmZmZjE4OTky Y2MgUjEyOiBmZmZmZmZmZjg1NDZmZmMwClsgICAgOS43MTk3NjFdWyAgICBUMF0gUjEzOiAwMDAw MDAwMDAwMDAwMDk4IFIxNDogZmZmZjg4ODEwNmU2M2E0MCBSMTU6IDAwMDAwMDAwMDAwMDAwMDEK WyAgICA5LjcyNzYxN11bICAgIFQwXSBGUzogIDAwMDAwMDAwMDAwMDAwMDAoMDAwMCkgR1M6ZmZm Zjg4ODdkZjgwMDAwMCgwMDAwKSBrbmxHUzowMDAwMDAwMDAwMDAwMDAwClsgICAgOS43MzY0MzFd WyAgICBUMF0gQ1M6ICAwMDEwIERTOiAwMDAwIEVTOiAwMDAwIENSMDogMDAwMDAwMDA4MDA1MDAz MwpbICAgIDkuNzQyODkyXVsgICAgVDBdIENSMjogZmZmZmZmZmZmZmZmZmZkNiBDUjM6IDAwMDAw MDFiYTc4MTQwMDEgQ1I0OiAwMDAwMDAwMDAwMDYwNmIwClsgICAgOS43NTA3NDddWyAgICBUMF0g RFIwOiAwMDAwMDAwMDAwMDAwMDAwIERSMTogMDAwMDAwMDAwMDAwMDAwMCBEUjI6IDAwMDAwMDAw MDAwMDAwMDAKWyAgICA5Ljc1ODYwMV1bICAgIFQwXSBEUjM6IDAwMDAwMDAwMDAwMDAwMDAgRFI2 OiAwMDAwMDAwMGZmZmUwZmYwIERSNzogMDAwMDAwMDAwMDAwMDQwMApbICAgIDkuNzY2NDU2XVsg ICAgVDBdIEtlcm5lbCBwYW5pYyAtIG5vdCBzeW5jaW5nOiBGYXRhbCBleGNlcHRpb24KWyAgICA5 Ljc3MjU0N11bICAgIFQwXSAtLS1bIGVuZCBLZXJuZWwgcGFuaWMgLSBub3Qgc3luY2luZzogRmF0 YWwgZXhjZXB0aW9uIF0tLS0KClRoZSB3b3JraW5nIGJvb3QgKHdpdGhvdXQgdGhvc2UgcGF0Y2hl cykgbG9va3MgbGlrZSB0aGlzOgoKWyAgICAxLjkxMzk2M11bICAgIFQwXSBBQ1BJOiBYMkFQSUNf Tk1JICh1aWRbMHhmNF0gaGlnaCBsZXZlbCBsaW50WzB4MV0pClsgICAgMS45MTM5NjddWyAgICBU MF0gQUNQSTogWDJBUElDX05NSSAodWlkWzB4ZjVdIGhpZ2ggbGV2ZWwgbGludFsweDFdKQpbICAg IDEuOTEzOTcwXVsgICAgVDBdIEFDUEk6IFgyQVBJQ19OTUkgKHVpZFsweGY2XSBoaWdoIGxldmVs IGxpbnRbMHgxXSkKWyAgICAxLjkxMzk3NF1bICAgIFQwXSBBQ1BJOiBYMkFQSUNfTk1JICh1aWRb MHhmN10gaGlnaCBsZXZlbCBsaW50WzB4MV0pClsgICAgMS45MTQwMTddWyAgICBUMF0gSU9BUElD WzBdOiBhcGljX2lkIDgsIHZlcnNpb24gMzIsIGFkZHJlc3MgMHhmZWMwMDAwMCwgR1NJIDAtMjMK WyAgICAxLjkxNDAzMl1bICAgIFQwXSBJT0FQSUNbMV06IGFwaWNfaWQgOSwgdmVyc2lvbiAzMiwg YWRkcmVzcyAweGZlYzAxMDAwLCBHU0kgMjQtMzEKWyAgICAxLjkxNDAzOV1bICAgIFQwXSBJT0FQ SUNbMl06IGFwaWNfaWQgMTAsIHZlcnNpb24gMzIsIGFkZHJlc3MgMHhmZWMwODAwMCwgR1NJIDMy LTM5ClsgICAgMS45MTQwNDddWyAgICBUMF0gSU9BUElDWzNdOiBhcGljX2lkIDExLCB2ZXJzaW9u IDMyLCBhZGRyZXNzIDB4ZmVjMTAwMDAsIEdTSSA0MC00NwpbICAgIDEuOTE0MDU0XVsgICAgVDBd IElPQVBJQ1s0XTogYXBpY19pZCAxMiwgdmVyc2lvbiAzMiwgYWRkcmVzcyAweGZlYzE4MDAwLCBH U0kgNDgtNTUKWyAgICAxLjkxNDA2Ml1bICAgIFQwXSBJT0FQSUNbNV06IGFwaWNfaWQgMTUsIHZl cnNpb24gMzIsIGFkZHJlc3MgMHhmZWMyMDAwMCwgR1NJIDU2LTYzClsgICAgMS5bICAgIDcuOTk0 NTY3XVsgICAgVDBdIG1lbXBvbGljeTogRW5hYmxpbmcgYXV0b21hdGljIE5VTUEgYmFsYW5jaW5n LiBDb25maWd1cmUgd2l0aCBudW1hX2JhbGFuY2luZz0gb3IgdGhlIGtlcm5lbC5udW1hX2JhbGFu Y2luZyBzeXNjdGwKWyAgICA4LjAwNjU0MV1bICAgIFQwXSBBQ1BJOiBDb3JlIHJldmlzaW9uIDIw MjAwNzE3ClsgICAgOC4wMTk3MTNdWyAgICBUMF0gY2xvY2tzb3VyY2U6IGhwZXQ6IG1hc2s6IDB4 ZmZmZmZmZmYgbWF4X2N5Y2xlczogMHhmZmZmZmZmZiwgbWF4X2lkbGVfbnM6IDc5NjM1ODU1MjQ1 IG5zClsgICAgOC4wMjk2NzJdWyAgICBUMF0gQVBJQzogU3dpdGNoIHRvIHN5bW1ldHJpYyBJL08g bW9kZSBzZXR1cApbICAgIDguMDM1MzU0XVsgICAgVDBdIERNQVI6IEhvc3QgYWRkcmVzcyB3aWR0 aCA0NgpbICAgIDguMDM5OTE1XVsgICAgVDBdIERNQVI6IERSSEQgYmFzZTogMHgwMDAwMDBlNWZm YzAwMCBmbGFnczogMHgwClsgICAgOC4wNDYwOTVdWyAgICBUMF0gRE1BUjogZG1hcjA6IHJlZ19i YXNlX2FkZHIgZTVmZmMwMDAgdmVyIDE6MCBjYXAgOGQyMDc4YzEwNmYwNDY2IGVjYXAgZjAyMGRm ClsgICAgOC4wNTQ4NDBdWyAgICBUMF0gRE1BUjogRFJIRCBiYXNlOiAweDAwMDAwMGU3ZmZjMDAw IGZsYWdzOiAweDAKWyAgICA4LjA2MDk5N11bICAgIFQwXSBETUFSOiBkbWFyMTogcmVnX2Jhc2Vf YWRkciBlN2ZmYzAwMCB2ZXIgMTowIGNhcCA4ZDIwNzhjMTA2ZjA0NjYgZWNhcCBmMDIwZGYKWyAg ICA4LjA2OTc0MF1bICAgIFQwXSBETUFSOiBEUkhEIGJhc2U6IDB4MDAwMDAwZTlmZmMwMDAgZmxh Z3M6IDB4MApbICAgIDguMDc1ODcyXVsgICAgVDBdIERNQVI6IGRtYXIyOiByZWdfYmFzZV9hZGRy IGU5ZmZjMDAwIHZlciAxOjAgY2FwIDhkMjA3OGMxMDZmMDQ2NiBlY2FwIGYwMjBkZgpbICAgIDgu MDg0NjE1XVsgICAgVDBdIERNQVI6IERSSEQgYmFzZTogMHgwMDAwMDBlYmZmYzAwMCBmbGFnczog MHgwClsgICAgOC4wOTA3NjFdWyAgICBUMF0gRE1BUjogZG1hcjM6IHJlZ19iYXNlX2FkZHIgZWJm ZmMwMDAgdmVyIDE6MCBjYXAgOGQyMDc4YzEwNmYwNDY2IGVjYXAgZk1BUi1JUjogRW5hYmxlZCBJ UlEgcmVtYXBwaW5nIGluIHgyYXBpYyBtb2RlClsgICAgOC41MTM0OTFdWyAgICBUMF0gLi5USU1F UjogdmVjdG9yPTB4MzAgYXBpYzE9MCBwaW4xPTIgYXBpYzI9LTEgcGluMj0tMQpbICAgIDguNTY4 Mjg5XVsgICAgVDBdIGNsb2Nrc291cmNlOiB0c2MtZWFybHk6IG1hc2s6IDB4ZmZmZmZmZmZmZmZm ZmZmZiBtYXhfY3ljbGVzOiAweDJiM2U0NTliZjRjLCBtYXhfaWRsZV9uczogNDQwNzk1Mjg5ODkw IG5zClsgICAgOC41Nzk1NzZdWyAgICBUMF0gQ2FsaWJyYXRpbmcgZGVsYXkgbG9vcCAoc2tpcHBl ZCksIHZhbHVlIGNhbGN1bGF0ZWQgdXNpbmcgdGltZXIgZnJlcXVlbmN5Li4gNjAwMC4wMCBCb2dv TUlQUyAobHBqPTMwMDAwMDAwKQpbICAgIDguNTg5NTc0XVsgICAgVDBdIHBpZF9tYXg6IGRlZmF1 bHQ6IDE0NzQ1NiBtaW5pbXVtOiAxMTUyClsgICAgOC43MTQwMjVdWyAgICBUMF0gZWZpOiBtZW1h dHRyOiBFbnRyeSBhdHRyaWJ1dGVzIGludmFsaWQ6IFJPIGFuZCBYUCBiaXRzIGJvdGggY2xlYXJl ZApbICAgIDguNzE5NTc3XVsgICAgVDBdIGVmaTogbWVtYXR0cjogISAweDAwMDBhMDU3YTAwMC0w eDAwMDBhMDViNGZmZiBbUnVudGltZSBDb2RlICAgICAgIHxSVU58ICB8ICB8ICB8ICB8ICB8ICB8 ICB8ICAgfCAgfCAgfCAgfCAgXQpbICAgIDguNzc1MzU1XVsgICAgVDBdIERlbnRyeSBjYWNoZSBo YXNoIHRhYmxlIGVudHJpZXM6IDgzODg2MDggKG9yZGVyOiAxNCwgNjcxMDg4NjQgYnl0ZXMsIHZt YWxsb2MpClsgICAgOC43OTg4NjhdWyAgICBUMF0gSW5vZGUtY2FjaGUgaGFzaCB0YWJsZSBlbnRy aWVzOiA0MTk0MzA0IChvcmRlcjogMTMsIDMzNTU0NDMyIGJ5dGVzLCB2bWFsbG9jKQpbICAgIDgu ODExNTUwXVsgICAgVDBdIE1vdW50LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogMTMxMDcyIChv cmRlcjogOCwgMTA0ODU3NiBieXRlcywgdm1hbGxvYykKWyAgICA4LjgyMDA3Nl1bICAgIFQwXSBN b3VudHBvaW50LWNhY2hlIGhhc2ggdGFibGUgZW50cmllczogMTMxMDcyIChvcmRlcjogOCwgMTA0 ODU3NiBieXRlcywgdm1hbGxvYykKWyAgICA4Ljg3OTMyN11bICAgIFQwXSBtY2U6IENQVTA6IFRo ZXJtYWwgbW9bICAgIDguOTk2OTE2XVsgICAgVDFdIFBlcmZvcm1hbmNlIEV2ZW50czogUEVCUyBm bXQzKywgU2t5bGFrZSBldmVudHMsIDMyLWRlZXAgTEJSLCBmdWxsLXdpZHRoIGNvdW50ZXJzLCBJ bnRlbCBQTVUgZHJpdmVyLgpbICAgIDguOTk5NTkxXVsgICAgVDFdIC4uLiB2ZXJzaW9uOiAgICAg ICAgICAgICAgICA0ClsgICAgOS4wMDQzMTBdWyAgICBUMV0gLi4uIGJpdCB3aWR0aDogICAgICAg ICAgICAgIDQ4ClsgICAgOS4wMDkxMThdWyAgICBUMV0gLi4uIGdlbmVyaWMgcmVnaXN0ZXJzOiAg ICAgIDQKWyAgICA5LjAwOTU3NF1bICAgIFQxXSAuLi4gdmFsdWUgbWFzazogICAgICAgICAgICAg MDAwMGZmZmZmZmZmZmZmZgpbICAgIDkuMDE1NjAxXVsgICAgVDFdIC4uLiBtYXggcGVyaW9kOiAg ICAgICAgICAgICAwMDAwN2ZmZmZmZmZmZmZmClsgICAgOS4wMTk1NzRdWyAgICBUMV0gLi4uIGZp eGVkLXB1cnBvc2UgZXZlbnRzOiAgIDMKWyAgICA5LjAyNDI5NF1bICAgIFQxXSAuLi4gZXZlbnQg bWFzazogICAgICAgICAgICAgMDAwMDAwMDcwMDAwMDAwZgpbICAgIDkuMDM0MzU3XVsgICAgVDFd IHJjdTogSGllcmFyY2hpY2FsIFNSQ1UgaW1wbGVtZW50YXRpb24uClsgICAgOS4wNjI1MTZdWyAg ICBUNV0gTk1JIHdhdGNoZG9nOiBFbmFibGVkLiBQZXJtYW5lbnRseSBjb25zdW1lcyBvbmUgaHct UE1VIGNvdW50ZXIuCgo+IAo+IFRoZXJlIGFyZSBxdWl0ZSBhIGJ1bmNoIG9mIGlzc3VlcyB0byBz b2x2ZToKPiAKPiAgIC0gWDg2IGRvZXMgbm90IHVzZSB0aGUgZGV2aWNlOjptc2lfZG9tYWluIHBv aW50ZXIgZm9yIGhpc3RvcmljYWwgcmVhc29ucwo+ICAgICBhbmQgZHVlIHRvIFhFTiwgd2hpY2gg bWFrZXMgaXQgaW1wb3NzaWJsZSB0byBjcmVhdGUgYW4gYXJjaGl0ZWN0dXJlCj4gICAgIGFnbm9z dGljIGRldmljZSBNU0kgaW5mcmFzdHJ1Y3R1cmUuCj4gCj4gICAtIFg4NiBoYXMgaXQncyBvd24g bXNpX2FsbG9jX2luZm8gZGF0YSB0eXBlIHdoaWNoIGlzIHBvaW50bGVzc2x5Cj4gICAgIGRpZmZl cmVudCBmcm9tIHRoZSBnZW5lcmljIHZlcnNpb24gYW5kIGRvZXMgbm90IGFsbG93IHRvIHNoYXJl IGNvZGUuCj4gCj4gICAtIFRoZSBsb2dpYyBvZiBjb21wb3NpbmcgTVNJIG1lc3NhZ2VzIGluIGFu IGhpZXJhcmNoeSBpcyBidXN0ZWQgYXQgdGhlCj4gICAgIGNvcmUgbGV2ZWwgYW5kIG9mIGNvdXJz ZSBzb21lICh4ODYpIGRyaXZlcnMgZGVwZW5kIG9uIHRoYXQuCj4gCj4gICAtIEEgZmV3IG1pbm9y IHNob3J0Y29taW5ncyBhcyB1c3VhbAo+IAo+IFRoaXMgc2VyaWVzIGFkZHJlc3NlcyB0aGF0IGlu IHNldmVyYWwgc3RlcHM6Cj4gCj4gIDEpIEFjY2lkZW50YWwgYnVnIGZpeGVzCj4gCj4gICAgICAg aW9tbXUvYW1kOiBQcmV2ZW50IE5VTEwgcG9pbnRlciBkZXJlZmVyZW5jZQo+IAo+ICAyKSBKYW5p dG9yaW5nCj4gCj4gICAgICAgeDg2L2luaXQ6IFJlbW92ZSB1bnVzZWQgaW5pdCBvcHMKPiAgICAg ICBQQ0k6IHZtZDogRG9udCBhYnVzZSB2ZWN0b3IgaXJxb21haW4gYXMgcGFyZW50Cj4gICAgICAg eDg2L21zaTogUmVtb3ZlIHBvaW50bGVzcyB2Y3B1X2FmZmluaXR5IGNhbGxiYWNrCj4gCj4gIDMp IFNhbml0aXppbmcgdGhlIGNvbXBvc2l0aW9uIG9mIE1TSSBtZXNzYWdlcyBpbiBhIGhpZXJhcmNo eQo+ICAKPiAgICAgICBnZW5pcnEvY2hpcDogVXNlIHRoZSBmaXJzdCBjaGlwIGluIGlycV9jaGlw X2NvbXBvc2VfbXNpX21zZygpCj4gICAgICAgeDg2L21zaTogTW92ZSBjb21wb3NlIG1lc3NhZ2Ug Y2FsbGJhY2sgd2hlcmUgaXQgYmVsb25ncwo+IAo+ICA0KSBTaW1wbGlmaWNhdGlvbiBvZiB0aGUg eDg2IHNwZWNpZmljIGludGVycnVwdCBhbGxvY2F0aW9uIG1lY2hhbmlzbQo+IAo+ICAgICAgIHg4 Ni9pcnE6IFJlbmFtZSBYODZfSVJRX0FMTE9DX1RZUEVfTVNJKiB0byByZWZsZWN0IFBDSSBkZXBl bmRlbmN5Cj4gICAgICAgeDg2L2lycTogQWRkIGFsbG9jYXRpb24gdHlwZSBmb3IgcGFyZW50IGRv bWFpbiByZXRyaWV2YWwKPiAgICAgICBpb21tdS92dC1kOiBDb25zb2xpZGF0ZSBpcnEgZG9tYWlu IGdldHRlcgo+ICAgICAgIGlvbW11L2FtZDogQ29uc29saWRhdGUgaXJxIGRvbWFpbiBnZXR0ZXIK PiAgICAgICBpb21tdS9pcnFfcmVtYXBwaW5nOiBDb25zb2xpZGF0ZSBpcnEgZG9tYWluIGxvb2t1 cAo+IAo+ICA1KSBDb25zb2xpZGF0aW9uIG9mIHRoZSBYODYgc3BlY2lmaWMgaW50ZXJydXB0IGFs bG9jYXRpb24gbWVjaGFuaXNtIHRvIGJlIGFzCj4gY2xvc2UKPiAgICAgYXMgcG9zc2libGUgdG8g dGhlIGdlbmVyaWMgTVNJIGFsbG9jYXRpb24gbWVjaGFuaXNtIHdoaWNoIGFsbG93cyB0byBnZXQK PiByaWQKPiAgICAgb2YgcXVpdGUgYSBidW5jaCBvZiB4ODYnaXNtcyB3aGljaCBhcmUgcG9pbnRs ZXNzCj4gCj4gICAgICAgeDg2L2lycTogUHJlcGFyZSBjb25zb2xpZGF0aW9uIG9mIGlycV9hbGxv Y19pbmZvCj4gICAgICAgeDg2L21zaTogQ29uc29saWRhdGUgSFBFVCBhbGxvY2F0aW9uCj4gICAg ICAgeDg2L2lvYXBpYzogQ29uc29saWRhdGUgSU9BUElDIGFsbG9jYXRpb24KPiAgICAgICB4ODYv aXJxOiBDb25zb2xpZGF0ZSBETUFSIGlycSBhbGxvY2F0aW9uCj4gICAgICAgeDg2L2lycTogQ29u c29saWRhdGUgVVYgZG9tYWluIGFsbG9jYXRpb24KPiAgICAgICBQQ0kvTVNJOiBSZXdvcmsgcGNp X21zaV9kb21haW5fY2FsY19od2lycSgpCj4gICAgICAgeDg2L21zaTogQ29uc29saWRhdGUgTVNJ IGFsbG9jYXRpb24KPiAgICAgICB4ODYvbXNpOiBVc2UgZ2VuZXJpYyBNU0kgZG9tYWluIG9wcwo+ IAo+ICAgNikgeDg2IHNwZWNpZmljIGNsZWFudXBzIHRvIHJlbW92ZSB0aGUgZGVwZW5kZW5jeSBv biBhcmNoXypfbXNpX2lycXMoKQo+IAo+ICAgICAgIHg4Ni9pcnE6IE1vdmUgYXBpY19wb3N0X2lu aXQoKSBpbnZvY2F0aW9uIHRvIG9uZSBwbGFjZQo+ICAgICAgIHg4Ni9wY2k6IFJlZHVjZGUgI2lm ZGVmZmVyeSBpbiBQQ0kgaW5pdCBjb2RlCj4gICAgICAgeDg2L2lycTogSW5pdGlhbGl6ZSBQQ0kv TVNJIGRvbWFpbiBhdCBQQ0kgaW5pdCB0aW1lCj4gICAgICAgaXJxZG9tYWluL21zaTogUHJvdmlk ZSBET01BSU5fQlVTX1ZNRF9NU0kKPiAgICAgICBQQ0k6IHZtZDogTWFyayBWTUQgaXJxZG9tYWlu IHdpdGggRE9NQUlOX0JVU19WTURfTVNJCj4gICAgICAgUENJL01TSTogUHJvdmlkZSBwY2lfZGV2 X2hhc19zcGVjaWFsX21zaV9kb21haW4oKSBoZWxwZXIKPiAgICAgICB4ODYveGVuOiBNYWtlIHhl bl9tc2lfaW5pdCgpIHN0YXRpYyBhbmQgcmVuYW1lIGl0IHRvIHhlbl9odm1fbXNpX2luaXQoKQo+ ICAgICAgIHg4Ni94ZW46IFJld29yayBNU0kgdGVhcmRvd24KPiAgICAgICB4ODYveGVuOiBDb25z b2xpZGF0ZSBYRU4tTVNJIGluaXQKPiAgICAgICBpcnFkb21haW4vbXNpOiBBbGxvdyB0byBvdmVy cmlkZSBtc2lfZG9tYWluX2FsbG9jL2ZyZWVfaXJxcygpCj4gICAgICAgeDg2L3hlbjogV3JhcCBY RU4gTVNJIG1hbmFnZW1lbnQgaW50byBpcnFkb21haW4KPiAgICAgICBpb21tbS92dC1kOiBTdG9y ZSBpcnEgZG9tYWluIGluIHN0cnVjdCBkZXZpY2UKPiAgICAgICBpb21tbS9hbWQ6IFN0b3JlIGly cSBkb21haW4gaW4gc3RydWN0IGRldmljZQo+ICAgICAgIHg4Ni9wY2k6IFNldCBkZWZhdWx0IGly cSBkb21haW4gaW4gcGNpYmlvc19hZGRfZGV2aWNlKCkKPiAgICAgICBQQ0kvTVNJOiBNYWtlIGFy Y2hfLipfbXNpX2lycVtzXSBmYWxsYmFja3Mgc2VsZWN0YWJsZQo+ICAgICAgIHg4Ni9pcnE6IENs ZWFudXAgdGhlIGFyY2hfKl9tc2lfaXJxcygpIGxlZnRvdmVycwo+ICAgICAgIHg4Ni9pcnE6IE1h a2UgbW9zdCBNU0kgb3BzIFhFTiBwcml2YXRlCj4gICAgICAgaW9tbXUvdnQtZDogUmVtb3ZlIGRv bWFpbiBzZWFyY2ggZm9yIFBDSS9NU0lbWF0KPiAgICAgICBpb21tdS9hbWQ6IFJlbW92ZSBkb21h aW4gc2VhcmNoIGZvciBQQ0kvTVNJCj4gCj4gICA3KSBYODYgc3BlY2lmaWMgcHJlcGFyYXRpb24g Zm9yIGRldmljZSBNU0kKPiAKPiAgICAgICB4ODYvaXJxOiBBZGQgREVWX01TSSBhbGxvY2F0aW9u IHR5cGUKPiAgICAgICB4ODYvbXNpOiBSZW5hbWUgYW5kIHJld29yayBwY2lfbXNpX3ByZXBhcmUo KSB0byBjb3ZlciBub24tUENJIE1TSQo+IAo+ICAgOCkgR2VuZXJpYyBkZXZpY2UgTVNJIGluZnJh c3RydWN0dXJlCj4gICAgICAgcGxhdGZvcm0tbXNpOiBQcm92aWRlIGRlZmF1bHQgaXJxX2NoaXA6 OiBBY2sKPiAgICAgICBnZW5pcnEvcHJvYzogVGFrZSBidXNsb2NrIG9uIGFmZmluaXR5IHdyaXRl Cj4gICAgICAgZ2VuaXJxL21zaTogUHJvdmlkZSBhbmQgdXNlIG1zaV9kb21haW5fc2V0X2RlZmF1 bHRfaW5mb19mbGFncygpCj4gICAgICAgcGxhdGZvcm0tbXNpOiBBZGQgZGV2aWNlIE1TSSBpbmZy YXN0cnVjdHVyZQo+ICAgICAgIGlycWRvbWFpbi9tc2k6IFByb3ZpZGUgbXNpX2FsbG9jL2ZyZWVf c3RvcmUoKSBjYWxsYmFja3MKPiAKPiAgIDkpIFBPQyBvZiBJTVMgKEludGVycnVwdCBNZXNzYWdl IFN0b3JtKSBpcnEgZG9tYWluIGFuZCBpcnFjaGlwCj4gICAgICBpbXBsZW1lbnRhdGlvbnMgZm9y IGJvdGggZGV2aWNlIGFycmF5IGFuZCBxdWV1ZSBzdG9yYWdlLgo+IAo+ICAgICAgIGlycWNoaXA6 IEFkZCBJTVMgKEludGVycnVwdCBNZXNzYWdlIFN0b3JtKSBkcml2ZXIgLSBOT1QgRk9SIE1FUkdJ TkcKPiAKPiBDaGFuZ2VzIHZzLiBWMToKPiAKPiAgICAtIEFkZHJlc3NlZCB2YXJpb3VzIHJldmll dyBjb21tZW50cyBhbmQgYWRkcmVzc2VkIHRoZSAwZGF5IGZhbGxvdXQuCj4gICAgICAtIENvcnJl Y3RlZCB0aGUgWEVOIGxvZ2ljIChKw7xyZ2VuKQo+ICAgICAgLSBNYWtlIHRoZSBhcmNoIGZhbGxi YWNrIGluIFBDSS9NU0kgb3B0LWluIG5vdCBvcHQtb3V0IChCam9ybikKPiAKPiAgICAtIEZpeGVk IHRoZSBjb21wb3NlIE1TSSBtZXNzYWdlIGluY29uc2lzdGVuY3kKPiAKPiAgICAtIEVuc3VyZSB0 aGF0IHRoZSBuZWNlc3NhcnkgZmxhZ3MgYXJlIHNldCBmb3IgZGV2aWNlIFNNSQo+IAo+ICAgIC0g TWFrZSB0aGUgaXJxIGJ1cyBsb2dpYyB3b3JrIGZvciBhZmZpbml0eSBzZXR0aW5nIHRvIHByZXBh cmUKPiAgICAgIHN1cHBvcnQgZm9yIElNUyBzdG9yYWdlIGluIHF1ZXVlIG1lbW9yeS4gSXQgdHVy bmVkIG91dCB0byBiZQo+ICAgICAgbGVzcyBzY2FyeSB0aGFuIEkgZmVhcmVkLgo+IAo+ICAgIC0g UmVtb3ZlIGxlZnRvdmVycyBpbiBpb21tdS9pbnRlbHxhbWQKPiAKPiAgICAtIFJld29ya2VkIHRo ZSBJTVMgUE9DIGRyaXZlciB0byBjb3ZlciBxdWV1ZSBzdG9yYWdlIHNvIEphc29uIGNhbiBoYXZl IGEKPiAgICAgIGxvb2sgd2hldGhlciB0aGF0IGZpdHMgdGhlIG5lZWRzIG9mIE1MWCBkZXZpY2Vz Lgo+IAo+IFRoZSB3aG9sZSBsb3QgaXMgYWxzbyBhdmFpbGFibGUgZnJvbSBnaXQ6Cj4gCj4gICAg Z2l0Oi8vZ2l0Lmtlcm5lbC5vcmcvcHViL3NjbS9saW51eC9rZXJuZWwvZ2l0L3RnbHgvZGV2ZWwu Z2l0IGRldmljZS1tc2kKPiAKPiBUaGlzIGhhcyBiZWVuIHRlc3RlZCBvbiBJbnRlbC9BTUQvS1ZN IGJ1dCBsYWNrcyB0ZXN0aW5nIG9uOgo+IAo+ICAgICAtIEhZUEVSViAoLUVOT0RFVikKPiAgICAg LSBWTUQgZW5hYmxlZCBzeXN0ZW1zICgtRU5PREVWKQo+ICAgICAtIFhFTiAoLUVOT0NMVUUpCj4g ICAgIC0gSU1TICgtRU5PREVWKQo+IAo+ICAgICAtIEFueSBub24tWDg2IGNvZGUgd2hpY2ggbWln aHQgZGVwZW5kIG9uIHRoZSBicm9rZW4gY29tcG9zZSBNU0kgbWVzc2FnZQo+ICAgICAgIGxvZ2lj LiBNYXJjIGV4Y3BlY3RzIG5vdCBtdWNoIGZhbGxvdXQsIGJ1dCBhZ3JlZXMgdGhhdCB3ZSBuZWVk IHRvIGZpeAo+ICAgICAgIGl0IGFueXdheS4KPiAKPiAjMSAtICMzIHNob3VsZCBiZSBhcHBsaWVk IHVuY29uZGl0aW9uYWxseSBmb3Igb2J2aW91cyByZWFzb25zCj4gIzQgLSAjNiBhcmUgd29ydHdo aWxlIGNsZWFudXBzIHdoaWNoIHNob3VsZCBiZSBkb25lIGluZGVwZW5kZW50IG9mIGRldmljZSBN U0kKPiAKPiAjNyAtICM4IGxvb2sgcHJvbWlzaW5nIHRvIGNsZWFudXAgdGhlIHBsYXRmb3JtIE1T SSBpbXBsZW1lbnRhdGlvbgo+ICAgICAgCWluZGVwZW5kZW50IG9mICM4LCBidXQgSSBuZWl0aGVy IGhhZCBjeWNsZXMgbm9yIHRoZSBzdG9tYWNoIHRvCj4gICAgICAJdGFja2xlIHRoYXQuCj4gCj4g IzkJaXMgb2J2aW91c2x5IGp1c3QgZm9yIHRoZSBmb2xrcyBpbnRlcmVzdGVkIGluIElNUwo+IAo+ IFRoYW5rcywKPiAKPiAJdGdseAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KaW9tbXUgbWFpbGluZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRp b24ub3JnCmh0dHBzOi8vbGlzdHMubGludXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2lvbW11 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00377C4363D for ; Fri, 25 Sep 2020 15:29:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8E1B5235F9 for ; Fri, 25 Sep 2020 15:29:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ExCmKxeP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E1B5235F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kLpfC-0001w6-Vp; Fri, 25 Sep 2020 15:29:34 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kLpfB-0001w1-CT for xen-devel@lists.xenproject.org; Fri, 25 Sep 2020 15:29:33 +0000 X-Inumbo-ID: 161d2f16-f914-4791-b4f5-845950cd9e6e Received: from us-smtp-delivery-124.mimecast.com (unknown [216.205.24.124]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id 161d2f16-f914-4791-b4f5-845950cd9e6e; Fri, 25 Sep 2020 15:29:31 +0000 (UTC) Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601047771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=p7Afe48pkEQZLqWPSXWRulCj196P+48pQyU4ztEKPbI=; b=ExCmKxePDxbZ5FJ4NYvz+U4pMum66vt1nGiAGB4Yr9GAYRs4ncKEwUuFBMFlNI4M3GKoHD yYAhSYAaf//wld4AjsRji1BD0gn+z2dHZw1pmczyHg0F+TsnPR0qzErhJdEXFBS95cqekK DwAQ7svzKraioPgeDvtY6Y0KqQcfOdc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-321-CloU21ZHNWSunAjJZOKQ-g-1; Fri, 25 Sep 2020 11:29:26 -0400 X-MC-Unique: CloU21ZHNWSunAjJZOKQ-g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 19D6B10082EB; Fri, 25 Sep 2020 15:29:22 +0000 (UTC) Received: from ovpn-66-87.rdu2.redhat.com (unknown [10.10.67.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4371D1002382; Fri, 25 Sep 2020 15:29:14 +0000 (UTC) Message-ID: <3c12bdec2c4ecdabcccd9ece3d495d792e9fc231.camel@redhat.com> Subject: Re: [patch V2 00/46] x86, PCI, XEN, genirq ...: Prepare for device MSI From: Qian Cai To: Thomas Gleixner , LKML , Stephen Rothwell , linux-next@vger.kernel.org Cc: x86@kernel.org, Joerg Roedel , iommu@lists.linux-foundation.org, linux-hyperv@vger.kernel.org, Haiyang Zhang , Jon Derrick , Lu Baolu , Wei Liu , "K. Y. Srinivasan" , Stephen Hemminger , Steve Wahl , Dimitri Sivanich , Russ Anderson , linux-pci@vger.kernel.org, Bjorn Helgaas , Lorenzo Pieralisi , Konrad Rzeszutek Wilk , xen-devel@lists.xenproject.org, Juergen Gross , Boris Ostrovsky , Stefano Stabellini , Marc Zyngier , Greg Kroah-Hartman , "Rafael J. Wysocki" , Megha Dey , Jason Gunthorpe , Dave Jiang , Alex Williamson , Jacob Pan , Baolu Lu , Kevin Tian , Dan Williams Date: Fri, 25 Sep 2020 11:29:13 -0400 In-Reply-To: <20200826111628.794979401@linutronix.de> References: <20200826111628.794979401@linutronix.de> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" On Wed, 2020-08-26 at 13:16 +0200, Thomas Gleixner wrote: > This is the second version of providing a base to support device MSI (non > PCI based) and on top of that support for IMS (Interrupt Message Storm) > based devices in a halfways architecture independent way. > > The first version can be found here: > > https://lore.kernel.org/r/20200821002424.119492231@linutronix.de > > It's still a mixed bag of bug fixes, cleanups and general improvements > which are worthwhile independent of device MSI. Reverting the part of this patchset on the top of today's linux-next fixed an boot issue on HPE ProLiant DL560 Gen10, i.e., $ git revert --no-edit 13b90cadfc29..bc95fd0d7c42 .config: https://gitlab.com/cailca/linux-mm/-/blob/master/x86.config It looks like the crashes happen in the interrupt remapping code where they are only able to to generate partial call traces. [ 1.912386][ T0] ACPI: X2APIC_NMI (uid[0xf5] high level 9983][ T0] ... MAX_LOCK_DEPTH: 48 [ 7.914876][ T0] ... MAX_LOCKDEP_KEYS: 8192 [ 7.919942][ T0] ... CLASSHASH_SIZE: 4096 [ 7.925009][ T0] ... MAX_LOCKDEP_ENTRIES: 32768 [ 7.930163][ T0] ... MAX_LOCKDEP_CHAINS: 65536 [ 7.935318][ T0] ... CHAINHASH_SIZE: 32768 [ 7.940473][ T0] memory used by lock dependency info: 6301 kB [ 7.946586][ T0] memory used for stack traces: 4224 kB [ 7.952088][ T0] per task-struct memory footprint: 1920 bytes [ 7.968312][ T0] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 7.980281][ T0] ACPI: Core revision 20200717 [ 7.993343][ T0] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns [ 8.003270][ T0] APIC: Switch to symmetric I/O mode setup [ 8.008951][ T0] DMAR: Host address width 46 [ 8.013512][ T0] DMAR: DRHD base: 0x000000e5ffc000 flags: 0x0 [ 8.019680][ T0] DMAR: dmar0: reg_base_addr e5ffc000 ver 1:0 cap 8d2078c106f0466 [ T0] DMAR-IR: IOAPIC id 15 under DRHD base 0xe5ffc000 IOMMU 0 [ 8.420990][ T0] DMAR-IR: IOAPIC id 8 under DRHD base 0xddffc000 IOMMU 15 [ 8.428166][ T0] DMAR-IR: IOAPIC id 9 under DRHD base 0xddffc000 IOMMU 15 [ 8.435341][ T0] DMAR-IR: HPET id 0 under DRHD base 0xddffc000 [ 8.441456][ T0] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. [ 8.457911][ T0] DMAR-IR: Enabled IRQ remapping in x2apic mode [ 8.466614][ T0] BUG: kernel NULL pointer dereference, address: 0000000000000000 [ 8.474295][ T0] #PF: supervisor instruction fetch in kernel mode [ 8.480669][ T0] #PF: error_code(0x0010) - not-present page [ 8.486518][ T0] PGD 0 P4D 0 [ 8.489757][ T0] Oops: 0010 [#1] SMP KASAN PTI [ 8.494476][ T0] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G I 5.9.0-rc6-next-20200925 #2 [ 8.503987][ T0] Hardware name: HPE ProLiant DL560 Gen10/ProLiant DL560 Gen10, BIOS U34 11/13/2019 [ 8.513238][ T0] RIP: 0010:0x0 [ 8.516562][ T0] Code: Bad RIP v or [ 2.906744][ T0] ACPI: X2API32, address 0xfec68000, GSI 128-135 [ 2.907063][ T0] IOAPIC[15]: apic_id 29, version 32, address 0xfec70000, GSI 136-143 [ 2.907071][ T0] IOAPIC[16]: apic_id 30, version 32, address 0xfec78000, GSI 144-151 [ 2.907079][ T0] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) [ 2.907084][ T0] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) [ 2.907100][ T0] Using ACPI (MADT) for SMP configuration information [ 2.907105][ T0] ACPI: HPET id: 0x8086a701 base: 0xfed00000 [ 2.907116][ T0] ACPI: SPCR: console: uart,mmio,0x0,115200 [ 2.907121][ T0] TSC deadline timer available [ 2.907126][ T0] smpboot: Allowing 144 CPUs, 0 hotplug CPUs [ 2.907163][ T0] [mem 0xd0000000-0xfdffffff] available for PCI devices [ 2.907175][ T0] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 2.914541][ T0] setup_percpu: NR_CPUS:256 nr_cpumask_bits:144 nr_cpu_ids:144 nr_node_ids:4 [ 2.926109][ 466 ecap f020df [ 9.134709][ T0] DMAR: DRHD base: 0x000000f5ffc000 flags: 0x0 [ 9.140867][ T0] DMAR: dmar8: reg_base_addr f5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.149610][ T0] DMAR: DRHD base: 0x000000f7ffc000 flags: 0x0 [ 9.155762][ T0] DMAR: dmar9: reg_base_addr f7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.164491][ T0] DMAR: DRHD base: 0x000000f9ffc000 flags: 0x0 [ 9.170645][ T0] DMAR: dmar10: reg_base_addr f9ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.179476][ T0] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0 [ 9.185626][ T0] DMAR: dmar11: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.194442][ T0] DMAR: DRHD base: 0x000000dfffc000 flags: 0x0 [ 9.200587][ T0] DMAR: dmar12: reg_base_addr dfffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.209418][ T0] DMAR: DRHD base: 0x000000e1ffc000 flags: 0x0 [ 9.215551][ T0] DMAR: dmar13: reg_base_addr e1ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 9.224367][ T0] DMAR: DRHD base: 0x000000e3ffc83][ T0] msi_domain_alloc+0x8e/0x280 [ 9.615015][ T0] __irq_domain_a8992cd [ 9.711906][ T0] R10: ffffffff85407d78 R11: fffffbfff18992cc R12: ffffffff8546ffc0 [ 9.719761][ T0] R13: 0000000000000098 R14: ffff888106e63a40 R15: 0000000000000001 [ 9.727617][ T0] FS: 0000000000000000(0000) GS:ffff8887df800000(0000) knlGS:0000000000000000 [ 9.736431][ T0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 9.742892][ T0] CR2: ffffffffffffffd6 CR3: 0000001ba7814001 CR4: 00000000000606b0 [ 9.750747][ T0] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 9.758601][ T0] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 9.766456][ T0] Kernel panic - not syncing: Fatal exception [ 9.772547][ T0] ---[ end Kernel panic - not syncing: Fatal exception ]--- The working boot (without those patches) looks like this: [ 1.913963][ T0] ACPI: X2APIC_NMI (uid[0xf4] high level lint[0x1]) [ 1.913967][ T0] ACPI: X2APIC_NMI (uid[0xf5] high level lint[0x1]) [ 1.913970][ T0] ACPI: X2APIC_NMI (uid[0xf6] high level lint[0x1]) [ 1.913974][ T0] ACPI: X2APIC_NMI (uid[0xf7] high level lint[0x1]) [ 1.914017][ T0] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23 [ 1.914032][ T0] IOAPIC[1]: apic_id 9, version 32, address 0xfec01000, GSI 24-31 [ 1.914039][ T0] IOAPIC[2]: apic_id 10, version 32, address 0xfec08000, GSI 32-39 [ 1.914047][ T0] IOAPIC[3]: apic_id 11, version 32, address 0xfec10000, GSI 40-47 [ 1.914054][ T0] IOAPIC[4]: apic_id 12, version 32, address 0xfec18000, GSI 48-55 [ 1.914062][ T0] IOAPIC[5]: apic_id 15, version 32, address 0xfec20000, GSI 56-63 [ 1.[ 7.994567][ T0] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl [ 8.006541][ T0] ACPI: Core revision 20200717 [ 8.019713][ T0] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 79635855245 ns [ 8.029672][ T0] APIC: Switch to symmetric I/O mode setup [ 8.035354][ T0] DMAR: Host address width 46 [ 8.039915][ T0] DMAR: DRHD base: 0x000000e5ffc000 flags: 0x0 [ 8.046095][ T0] DMAR: dmar0: reg_base_addr e5ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 8.054840][ T0] DMAR: DRHD base: 0x000000e7ffc000 flags: 0x0 [ 8.060997][ T0] DMAR: dmar1: reg_base_addr e7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 8.069740][ T0] DMAR: DRHD base: 0x000000e9ffc000 flags: 0x0 [ 8.075872][ T0] DMAR: dmar2: reg_base_addr e9ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df [ 8.084615][ T0] DMAR: DRHD base: 0x000000ebffc000 flags: 0x0 [ 8.090761][ T0] DMAR: dmar3: reg_base_addr ebffc000 ver 1:0 cap 8d2078c106f0466 ecap fMAR-IR: Enabled IRQ remapping in x2apic mode [ 8.513491][ T0] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 [ 8.568289][ T0] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x2b3e459bf4c, max_idle_ns: 440795289890 ns [ 8.579576][ T0] Calibrating delay loop (skipped), value calculated using timer frequency.. 6000.00 BogoMIPS (lpj=30000000) [ 8.589574][ T0] pid_max: default: 147456 minimum: 1152 [ 8.714025][ T0] efi: memattr: Entry attributes invalid: RO and XP bits both cleared [ 8.719577][ T0] efi: memattr: ! 0x0000a057a000-0x0000a05b4fff [Runtime Code |RUN| | | | | | | | | | | | ] [ 8.775355][ T0] Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes, vmalloc) [ 8.798868][ T0] Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes, vmalloc) [ 8.811550][ T0] Mount-cache hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 8.820076][ T0] Mountpoint-cache hash table entries: 131072 (order: 8, 1048576 bytes, vmalloc) [ 8.879327][ T0] mce: CPU0: Thermal mo[ 8.996916][ T1] Performance Events: PEBS fmt3+, Skylake events, 32-deep LBR, full-width counters, Intel PMU driver. [ 8.999591][ T1] ... version: 4 [ 9.004310][ T1] ... bit width: 48 [ 9.009118][ T1] ... generic registers: 4 [ 9.009574][ T1] ... value mask: 0000ffffffffffff [ 9.015601][ T1] ... max period: 00007fffffffffff [ 9.019574][ T1] ... fixed-purpose events: 3 [ 9.024294][ T1] ... event mask: 000000070000000f [ 9.034357][ T1] rcu: Hierarchical SRCU implementation. [ 9.062516][ T5] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter. > > There are quite a bunch of issues to solve: > > - X86 does not use the device::msi_domain pointer for historical reasons > and due to XEN, which makes it impossible to create an architecture > agnostic device MSI infrastructure. > > - X86 has it's own msi_alloc_info data type which is pointlessly > different from the generic version and does not allow to share code. > > - The logic of composing MSI messages in an hierarchy is busted at the > core level and of course some (x86) drivers depend on that. > > - A few minor shortcomings as usual > > This series addresses that in several steps: > > 1) Accidental bug fixes > > iommu/amd: Prevent NULL pointer dereference > > 2) Janitoring > > x86/init: Remove unused init ops > PCI: vmd: Dont abuse vector irqomain as parent > x86/msi: Remove pointless vcpu_affinity callback > > 3) Sanitizing the composition of MSI messages in a hierarchy > > genirq/chip: Use the first chip in irq_chip_compose_msi_msg() > x86/msi: Move compose message callback where it belongs > > 4) Simplification of the x86 specific interrupt allocation mechanism > > x86/irq: Rename X86_IRQ_ALLOC_TYPE_MSI* to reflect PCI dependency > x86/irq: Add allocation type for parent domain retrieval > iommu/vt-d: Consolidate irq domain getter > iommu/amd: Consolidate irq domain getter > iommu/irq_remapping: Consolidate irq domain lookup > > 5) Consolidation of the X86 specific interrupt allocation mechanism to be as > close > as possible to the generic MSI allocation mechanism which allows to get > rid > of quite a bunch of x86'isms which are pointless > > x86/irq: Prepare consolidation of irq_alloc_info > x86/msi: Consolidate HPET allocation > x86/ioapic: Consolidate IOAPIC allocation > x86/irq: Consolidate DMAR irq allocation > x86/irq: Consolidate UV domain allocation > PCI/MSI: Rework pci_msi_domain_calc_hwirq() > x86/msi: Consolidate MSI allocation > x86/msi: Use generic MSI domain ops > > 6) x86 specific cleanups to remove the dependency on arch_*_msi_irqs() > > x86/irq: Move apic_post_init() invocation to one place > x86/pci: Reducde #ifdeffery in PCI init code > x86/irq: Initialize PCI/MSI domain at PCI init time > irqdomain/msi: Provide DOMAIN_BUS_VMD_MSI > PCI: vmd: Mark VMD irqdomain with DOMAIN_BUS_VMD_MSI > PCI/MSI: Provide pci_dev_has_special_msi_domain() helper > x86/xen: Make xen_msi_init() static and rename it to xen_hvm_msi_init() > x86/xen: Rework MSI teardown > x86/xen: Consolidate XEN-MSI init > irqdomain/msi: Allow to override msi_domain_alloc/free_irqs() > x86/xen: Wrap XEN MSI management into irqdomain > iommm/vt-d: Store irq domain in struct device > iommm/amd: Store irq domain in struct device > x86/pci: Set default irq domain in pcibios_add_device() > PCI/MSI: Make arch_.*_msi_irq[s] fallbacks selectable > x86/irq: Cleanup the arch_*_msi_irqs() leftovers > x86/irq: Make most MSI ops XEN private > iommu/vt-d: Remove domain search for PCI/MSI[X] > iommu/amd: Remove domain search for PCI/MSI > > 7) X86 specific preparation for device MSI > > x86/irq: Add DEV_MSI allocation type > x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI > > 8) Generic device MSI infrastructure > platform-msi: Provide default irq_chip:: Ack > genirq/proc: Take buslock on affinity write > genirq/msi: Provide and use msi_domain_set_default_info_flags() > platform-msi: Add device MSI infrastructure > irqdomain/msi: Provide msi_alloc/free_store() callbacks > > 9) POC of IMS (Interrupt Message Storm) irq domain and irqchip > implementations for both device array and queue storage. > > irqchip: Add IMS (Interrupt Message Storm) driver - NOT FOR MERGING > > Changes vs. V1: > > - Addressed various review comments and addressed the 0day fallout. > - Corrected the XEN logic (Jürgen) > - Make the arch fallback in PCI/MSI opt-in not opt-out (Bjorn) > > - Fixed the compose MSI message inconsistency > > - Ensure that the necessary flags are set for device SMI > > - Make the irq bus logic work for affinity setting to prepare > support for IMS storage in queue memory. It turned out to be > less scary than I feared. > > - Remove leftovers in iommu/intel|amd > > - Reworked the IMS POC driver to cover queue storage so Jason can have a > look whether that fits the needs of MLX devices. > > The whole lot is also available from git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git device-msi > > This has been tested on Intel/AMD/KVM but lacks testing on: > > - HYPERV (-ENODEV) > - VMD enabled systems (-ENODEV) > - XEN (-ENOCLUE) > - IMS (-ENODEV) > > - Any non-X86 code which might depend on the broken compose MSI message > logic. Marc excpects not much fallout, but agrees that we need to fix > it anyway. > > #1 - #3 should be applied unconditionally for obvious reasons > #4 - #6 are wortwhile cleanups which should be done independent of device MSI > > #7 - #8 look promising to cleanup the platform MSI implementation > independent of #8, but I neither had cycles nor the stomach to > tackle that. > > #9 is obviously just for the folks interested in IMS > > Thanks, > > tglx