* Re: AW: AW: AW: AW: Xen data from meta-virtualization layer [not found] ` <HE1PR05MB4794569AC67109AF8B6517268BE20@HE1PR05MB4794.eurprd05.prod.outlook.com> @ 2020-11-17 23:53 ` Stefano Stabellini 2020-11-18 12:03 ` Rahul Singh 2020-11-18 12:23 ` AW: AW: AW: AW: " Julien Grall 0 siblings, 2 replies; 16+ messages in thread From: Stefano Stabellini @ 2020-11-17 23:53 UTC (permalink / raw) To: Leo Krueger Cc: Stefano Stabellini, Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand.Marquis, julien [-- Attachment #1: Type: text/plain, Size: 7792 bytes --] Adding Bertrand, Oleksandr, Julien, and others -- they have a more recent experience with GICv3 ITS than me and might be able to help. I am attaching the device tree Leo sent a few days ago for reference. Typically when you can set the ethernet link up and no packets are exchanged it is because of a missing interrupt. In this case a missing MSI. Bertrand, I believe you tried the GIC ITS driver with PCI devices recently. It is expected to work correctly with MSIs in Dom0, right? On Tue, 17 Nov 2020, Leo Krueger wrote: > Hi, > > I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it before...) but then had to add the following node to my device tree > > gic_lpi_base: syscon@0x80000000 { > compatible = "gic-lpi-base"; > reg = <0x0 0x80000000 0x0 0x100000>; > max-gic-redistributors = <2>; > }; > > to somehow change something in regard to the ITS and MSI/MSI-X > > (XEN) GICv3 initialization: > (XEN) gic_dist_addr=0x00000006000000 > (XEN) gic_maintenance_irq=25 > (XEN) gic_rdist_stride=0 > (XEN) gic_rdist_regions=1 > (XEN) redistributor regions: > (XEN) - region 0: 0x00000006040000 - 0x00000006080000 > (XEN) GICv3: using at most 57344 LPIs on the host. > (XEN) GICv3: 288 lines, (IID 0001143b). > (XEN) GICv3: Found ITS @0x6020000 > (XEN) using non-cacheable ITS command queue > (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000 > > [ 0.000000] GICv3: Distributor has no Range Selector support > [ 0.000000] GICv3: no VLPI support, no direct LPI support > [ 0.000000] ITS [mem 0x06020000-0x0603ffff] > [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 (flat, esz 8, psz 64K, shr 1) > [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt Collections @dc820000 (flat, esz 2, psz 64K, shr 1) > [ 0.000000] GIC: using LPI property table @0x00000000dc830000 > [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000006040000 > [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000 > ... > [ 0.040080] Platform MSI: gic-its domain created > [ 0.040136] PCI/MSI: /interrupt-controller/gic-its domain created > [ 0.040181] fsl-mc MSI: /interrupt-controller/gic-its domain created > > > Still I am ending up with the " Failed to add - passthrough or MSI/MSI-X might fail!" log messages for some PCI devices, but at least the on-board ethernet ports (fsl_enetc ) are initialized. > I can set the link up and a link is successfully established. > > But (!) I cannot receive or transmit anything (no error message...) and there seem to be no interrupts: > > 29: 0 ITS-MSI 1 Edge gbe0-rxtx0 > 32: 0 ITS-MSI 8192 Edge ptp_qoriq > > (from /proc/interrupts). > > Any idea on this one? I keep digging and checking whether my device tree needs some additional fixes. > > Kind regards, > Leo > > -- > Leo Krüger, M.Sc. > Senior Systems Engineer Distributed Systems > Intelligent Digital Cabin > > ZAL Zentrum für Angewandte Luftfahrtforschung GmbH > Hein-Saß-Weg 22 > 21129 Hamburg > > +49 (0) 40 248 595-154 > > zal.aero | twitter.com/ZALTechCenter | facebook.com/ZALTechCenter > > ZAL Zentrum für Angewandte Luftfahrtforschung GmbH > Sitz der Gesellschaft / Legal Domicile: Hamburg > Registergericht / Registration Court: Amtsgericht Hamburg HRB 110232 > Vorsitzender des Aufsichtsrates / Chairman of the Supervisory Board: StR Andreas Rieckhof > Geschäftsführung / Board of Management: Roland Gerhards > > Disclaimer: > This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have > received this mail in error), please notify the sender immediately and destroy this e-mail. Any unauthorised copying, > disclosure or distribution of the material in this e-mail is strictly forbidden. > > > -----Ursprüngliche Nachricht----- > > Von: Stefano Stabellini <stefano.stabellini@xilinx.com> > > Gesendet: Dienstag, 17. November 2020 01:59 > > An: Leo Krueger <leo.krueger@zal.aero> > > Cc: Peng Fan <peng.fan@nxp.com>; Stefano Stabellini > > <stefano.stabellini@xilinx.com>; brucea@xilinx.com; Cornelia Bruelhart > > <cornelia.bruelhart@zal.aero> > > Betreff: Re: AW: AW: AW: Xen data from meta-virtualization layer > > > > Replies inline below > > > > > > On Sun, 15 Nov 2020, Leo Krueger wrote: > > > Hi Peng, hi Stefano, > > > > > > > > > > > > sorry for the long silence… > > > > > > > > > > > > I tried the change suggested (and hope I didn’t do anything wrong > > > while doing so…) on top of XEN 4.13.2 (before, I always tried with > > > 4.12 but wanted to give 4.13.2 a try as well) but I do not see any difference, > > still the same “unhandled context fault” log entries pop up and I cannot > > access my sdcard. > > > > > > > > > > > > As it seems to work without respectively disabled iommu, that would be > > > fine for me for now. What I am worried about a bit more is PCIe or > > MSI/MSIX to be exact. > > > > > > > > > > > > Here is the gic-v3 and its node from my device tree: > > > > > > > > > > > > interrupt-controller@6000000 { > > > > > > compatible = "arm,gic-v3"; > > > > > > #address-cells = <0x2>; > > > > > > #size-cells = <0x2>; > > > > > > ranges; > > > > > > reg = <0x0 0x6000000 0x0 0x10000 0x0 0x6040000 0x0 0x40000>; > > > > > > #interrupt-cells = <0x3>; > > > > > > interrupt-controller; > > > > > > interrupts = <0x1 0x9 0xf08>; > > > > > > phandle = <0x1>; > > > > > > > > > > > > gic-its@6020000 { > > > > > > compatible = "arm,gic-v3-its"; > > > > > > msi-controller; > > > > > > reg = <0x0 0x6020000 0x0 0x20000>; > > > > > > phandle = <0xd>; > > > > > > }; > > > > > > }; > > > > > > > > > > > > And here are some kernel log excerpts related to GIC when booting > > without (!) XEN: > > > > > > > > > > > > [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode > > > > > > [ 0.000000] GICv3: Distributor has no Range Selector support > > > > > > [ 0.000000] GICv3: no VLPI support, no direct LPI support > > > > > > [ 0.000000] ITS [mem 0x06020000-0x0603ffff] > > > > > > [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices > > > @20fb880000 (flat, esz 8, psz 64K, shr 0) > > > > > > [ 0.000000] ITS: using cache flushing for cmd queue > > > > > > [ 0.000000] GIC: using LPI property table @0x00000020fb830000 > > > > > > [ 0.000000] GICv3: CPU0: found redistributor 0 region > > > 0:0x0000000006040000 > > > > > > [ 0.000000] CPU0: using LPI pending table @0x00000020fb840000 > > > > > > [ 0.000000] GIC: using cache flushing for LPI property table > > > > > > > > > > > > However, when booting with XEN, only the following three lines are > > contained in the kernel log: > > > > > > > > > > > > [ 0.000000] GICv3: Distributor has no Range Selector support > > > > > > [ 0.000000] GICv3: no VLPI support, no direct LPI support > > > > > > [ 0.000000] GICv3: CPU0: found redistributor 0 region > > > 0:0x0000000006040000 > > > > "no VLPI support" is very suspicious, it looks like Dom0 doesn't find any ITS > > support. > > > > Can you double check that you have the ITS driver in Xen built-in? That would > > be CONFIG_HAS_ITS. If you do "make menuconfig" and enable "Configure > > standard Xen features (expert users)" you should get a new option "GICv3 > > ITS MSI controller support" under "Architecture Features". > > Make sure to enable it. > > > > Let me know if that works! > [-- Attachment #2: Type: text/plain, Size: 66249 bytes --] / { compatible = "kontron,sl28", "fsl,ls1028a"; interrupt-parent = <0x00000001>; #address-cells = <0x00000002>; #size-cells = <0x00000002>; model = "Kontron SMARC-sAL28"; chosen { xen,dom0-bootargs = "root=/dev/mmcblk1p2 console=hvc0 earlycon=xen earlyprintk=xen clk_ignore_unused rootwait "; xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=512M dom0_max_vcpus=1 bootscrub=0 vwfi=native sched=null "; #size-cells = <0x00000002>; #address-cells = <0x00000002>; dom0 { reg = <0x00000000 0x81200000 0x00000000 0x01517008>; compatible = "xen,linux-zimage", "xen,multiboot-module"; }; }; aliases { rtc1 = "/soc/timer@2800000"; crypto = "/soc/crypto@8000000"; serial0 = "/soc/serial@21c0500"; serial1 = "/soc/serial@21c0600"; }; cpus { #address-cells = <0x00000001>; #size-cells = <0x00000000>; cpu@0 { device_type = "cpu"; compatible = "arm,cortex-a72"; reg = <0x00000000>; enable-method = "psci"; clocks = <0x00000002 0x00000001 0x00000000>; next-level-cache = <0x00000003>; cpu-idle-states = <0x00000004>; #cooling-cells = <0x00000002>; phandle = <0x00000013>; }; cpu@1 { device_type = "cpu"; compatible = "arm,cortex-a72"; reg = <0x00000001>; enable-method = "psci"; clocks = <0x00000002 0x00000001 0x00000000>; next-level-cache = <0x00000003>; cpu-idle-states = <0x00000004>; #cooling-cells = <0x00000002>; phandle = <0x00000014>; }; l2-cache { compatible = "cache"; phandle = <0x00000003>; }; }; idle-states { entry-method = "arm,psci"; cpu-pw20 { compatible = "arm,idle-state"; idle-state-name = "PW20"; arm,psci-suspend-param = <0x00000000>; entry-latency-us = <0x000007d0>; exit-latency-us = <0x000007d0>; min-residency-us = <0x00001770>; phandle = <0x00000004>; }; }; clock-sysclk { compatible = "fixed-clock"; #clock-cells = <0x00000000>; clock-frequency = <0x05f5e100>; clock-output-names = "sysclk"; phandle = <0x00000007>; }; clock-osc-27m { compatible = "fixed-clock"; #clock-cells = <0x00000000>; clock-frequency = <0x019bfcc0>; clock-output-names = "phy_27m"; phandle = <0x00000005>; }; clock-display@f1f0000 { compatible = "fsl,ls1028a-plldig"; reg = <0x00000000 0x0f1f0000 0x00000000 0x0000ffff>; #clock-cells = <0x00000000>; clocks = <0x00000005>; phandle = <0x00000017>; }; clock-axi { compatible = "fixed-clock"; #clock-cells = <0x00000000>; clock-frequency = <0x26be3680>; clock-output-names = "aclk"; phandle = <0x00000018>; }; clock-apb { compatible = "fixed-clock"; #clock-cells = <0x00000000>; clock-frequency = <0x26be3680>; clock-output-names = "pclk"; phandle = <0x00000019>; }; clock-hdpcore { compatible = "fixed-clock"; #clock-cells = <0x00000000>; clock-frequency = <0x09af8da0>; clock-output-names = "hdpclk"; phandle = <0x0000001b>; }; reboot { compatible = "syscon-reboot"; regmap = <0x00000006>; offset = <0x00000000>; mask = <0x00000002>; }; timer { compatible = "arm,armv8-timer"; interrupts = <0x00000001 0x0000000d 0x00000308 0x00000001 0x0000000e 0x00000308 0x00000001 0x0000000b 0x00000308 0x00000001 0x0000000a 0x00000308>; }; interrupt-controller@6000000 { compatible = "arm,gic-v3"; #address-cells = <0x00000002>; #size-cells = <0x00000002>; ranges; reg = <0x00000000 0x06000000 0x00000000 0x00010000 0x00000000 0x06040000 0x00000000 0x00040000>; #interrupt-cells = <0x00000003>; interrupt-controller; interrupts = <0x00000001 0x00000009 0x00000f08>; phandle = <0x00000001>; gic-its@6020000 { compatible = "arm,gic-v3-its"; msi-controller; reg = <0x00000000 0x06020000 0x00000000 0x00020000>; phandle = <0x0000000d>; }; }; soc { compatible = "simple-bus"; #address-cells = <0x00000002>; #size-cells = <0x00000002>; ranges; phandle = <0x0000001f>; memory-controller@1080000 { compatible = "fsl,qoriq-memory-controller"; reg = <0x00000000 0x01080000 0x00000000 0x00001000>; interrupts = <0x00000000 0x00000090 0x00000004>; big-endian; phandle = <0x00000020>; }; syscon@1e00000 { compatible = "fsl,ls1028a-dcfg", "syscon"; reg = <0x00000000 0x01e00000 0x00000000 0x00010000>; big-endian; phandle = <0x00000021>; }; syscon@1e60000 { compatible = "fsl,ls1028a-rst", "syscon"; reg = <0x00000000 0x01e60000 0x00000000 0x00010000>; little-endian; phandle = <0x00000006>; }; syscon@1fc0000 { compatible = "fsl,ls1028a-scfg", "syscon"; reg = <0x00000000 0x01fc0000 0x00000000 0x00010000>; big-endian; phandle = <0x00000022>; }; clock-controller@1300000 { compatible = "fsl,ls1028a-clockgen"; reg = <0x00000000 0x01300000 0x00000000 0x000a0000>; #clock-cells = <0x00000002>; clocks = <0x00000007>; phandle = <0x00000002>; }; usb@3100000 { xen,passthrough; compatible = "snps,dwc3"; reg = <0x00000000 0x03100000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000050 0x00000004>; dr_mode = "host"; snps,dis_rxdet_inp3_quirk; snps,quirk-frame-length-adjustment = <0x00000020>; snps,incr-burst-type-adjustment = <0x00000001 0x00000004 0x00000008 0x00000010>; phandle = <0x00000023>; }; usb@3110000 { xen,passthrough; compatible = "snps,dwc3"; reg = <0x00000000 0x03110000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000051 0x00000004>; dr_mode = "host"; snps,dis_rxdet_inp3_quirk; snps,quirk-frame-length-adjustment = <0x00000020>; snps,incr-burst-type-adjustment = <0x00000001 0x00000004 0x00000008 0x00000010>; phandle = <0x00000024>; }; spi@20c0000 { compatible = "nxp,lx2160a-fspi"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x020c0000 0x00000000 0x00010000 0x00000000 0x20000000 0x00000000 0x10000000>; reg-names = "fspi_base", "fspi_mmap"; interrupts = <0x00000000 0x00000019 0x00000004>; clocks = <0x00000002 0x00000004 0x00000003 0x00000002 0x00000004 0x00000003>; clock-names = "fspi_en", "fspi"; status = "okay"; nxp,fspi-has-second-chip; phandle = <0x00000025>; w25q32jw@0 { #address-cells = <0x00000001>; #size-cells = <0x00000001>; compatible = "w25q32jw", "jedec,spi-nor"; m25p,fast-read; spi-max-frequency = <0x07ed6b40>; reg = <0x00000000>; spi-rx-bus-width = <0x00000002>; spi-tx-bus-width = <0x00000001>; partition@0 { reg = <0x00000000 0x00010000>; label = "rcw"; read-only; }; partition@10000 { reg = <0x00010000 0x000f0000>; label = "failsafe bootloader"; read-only; }; partition@100000 { reg = <0x00100000 0x00040000>; label = "failsafe DP firmware"; read-only; }; partition@140000 { reg = <0x00140000 0x000a0000>; label = "failsafe trusted firmware"; read-only; }; partition@1e0000 { reg = <0x001e0000 0x00020000>; label = "reserved"; read-only; }; partition@200000 { reg = <0x00200000 0x00010000>; label = "configuration store"; }; partition@210000 { reg = <0x00210000 0x000f0000>; label = "bootloader"; }; partition@300000 { reg = <0x00300000 0x00040000>; label = "DP firmware"; }; partition@340000 { reg = <0x00340000 0x000a0000>; label = "trusted firmware"; }; partition@3e0000 { reg = <0x003e0000 0x00020000>; label = "bootloader environment"; }; }; }; i2c@2000000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02000000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000022 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names; dmas = <0x00000008 0x00000001 0x00000027 0x00000008 0x00000001 0x00000026>; status = "okay"; phandle = <0x00000026>; rtc@32 { compatible = "microcrystal,rv8803"; reg = <0x00000032>; interrupt-parent = <0x00000009>; interrupts = <0x00000000 0x00000000>; wakeup-source; }; sl28cpld@4a { #address-cells = <0x00000001>; #size-cells = <0x00000000>; compatible = "kontron,sl28cpld"; reg = <0x0000004a>; interrupt-parent = <0x0000000a>; interrupts = <0x00000006 0x00000002>; interrupt-controller; #interrupt-cells = <0x00000002>; phandle = <0x00000009>; watchdog@4 { compatible = "kontron,sl28cpld-wdt"; reg = <0x00000004>; }; fan@b { compatible = "kontron,sl28cpld-fan"; reg = <0x0000000b>; }; pwm0@c { #pwm-cells = <0x00000002>; compatible = "kontron,sl28cpld-pwm"; reg = <0x0000000c>; phandle = <0x00000027>; }; pwm1@e { #pwm-cells = <0x00000002>; compatible = "kontron,sl28cpld-pwm"; reg = <0x0000000e>; phandle = <0x00000028>; }; gpio0@10 { compatible = "kontron,sl28cpld-gpio"; reg = <0x00000010>; interrupt-parent = <0x0000000a>; interrupts = <0x00000006 0x00000002>; gpio-controller; #gpio-cells = <0x00000002>; gpio-line-names = "GPIO0_CAM0_PWR_N", "GPIO1_CAM1_PWR_N", "GPIO2_CAM0_RST_N", "GPIO3_CAM1_RST_N", "GPIO4_HDA_RST_N", "GPIO5_PWM_OUT", "GPIO6_TACHIN", "GPIO7"; interrupt-controller; #interrupt-cells = <0x00000002>; phandle = <0x00000029>; }; gpio1@15 { compatible = "kontron,sl28cpld-gpio"; reg = <0x00000015>; interrupt-parent = <0x0000000a>; interrupts = <0x00000006 0x00000002>; gpio-controller; #gpio-cells = <0x00000002>; gpio-line-names = [47 50 49 4f 38 00 47 50 49 4f 39 00 47 50 49 4f 31 30 00 47 50 49 4f 31 31 00 00 00 00 00]; interrupt-controller; #interrupt-cells = <0x00000002>; phandle = <0x0000002a>; }; gpo0@1a { compatible = "kontron,sl28cpld-gpo"; reg = <0x0000001a>; gpio-controller; #gpio-cells = <0x00000002>; gpio-line-names = * 0x0000000082801988 [0x00000072]; phandle = <0x0000001d>; }; gpi0@1b { compatible = "kontron,sl28cpld-gpi"; reg = <0x0000001b>; gpio-controller; #gpio-cells = <0x00000002>; gpio-line-names = * 0x0000000082801a78 [0x00000052]; phandle = <0x0000001e>; }; }; eeprom@50 { compatible = "atmel,24c32"; reg = <0x00000050>; pagesize = <0x00000020>; }; }; i2c@2010000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02010000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000022 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000025 0x00000008 0x00000001 0x00000024>; status = "disabled"; phandle = <0x0000002b>; }; i2c@2020000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02020000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000023 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000023 0x00000008 0x00000001 0x00000022>; status = "disabled"; phandle = <0x0000002c>; }; i2c@2030000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02030000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000023 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names; dmas = <0x00000008 0x00000001 0x00000029 0x00000008 0x00000001 0x00000028>; status = "okay"; phandle = <0x0000002d>; }; i2c@2040000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02040000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000004a 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names; dmas = <0x00000008 0x00000001 0x0000002b 0x00000008 0x00000001 0x0000002a>; status = "okay"; phandle = <0x0000002e>; eeprom@50 { compatible = "atmel,24c32"; reg = <0x00000050>; pagesize = <0x00000020>; }; }; i2c@2050000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02050000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000004a 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000002d 0x00000008 0x00000001 0x0000002c>; status = "disabled"; phandle = <0x0000002f>; }; i2c@2060000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02060000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000004b 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000002f 0x00000008 0x00000001 0x0000002e>; status = "disabled"; phandle = <0x00000030>; }; i2c@2070000 { compatible = "fsl,vf610-i2c"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02070000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000004b 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000011 0x00000008 0x00000001 0x00000010>; status = "disabled"; phandle = <0x00000031>; }; spi@2100000 { compatible = "fsl,ls1028a-dspi", "fsl,ls1021a-v1.0-dspi"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02100000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000001a 0x00000004>; clock-names = "dspi"; clocks = <0x00000002 0x00000004 0x00000001>; spi-num-chipselects = <0x00000005>; little-endian; status = "disabled"; phandle = <0x00000032>; }; spi@2110000 { compatible = "fsl,ls1028a-dspi", "fsl,ls1021a-v1.0-dspi"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02110000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000001a 0x00000004>; clock-names = "dspi"; clocks = <0x00000002 0x00000004 0x00000001>; spi-num-chipselects = <0x00000005>; little-endian; status = "disabled"; phandle = <0x00000033>; }; spi@2120000 { compatible = "fsl,ls1028a-dspi", "fsl,ls1021a-v1.0-dspi"; #address-cells = <0x00000001>; #size-cells = <0x00000000>; reg = <0x00000000 0x02120000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000001a 0x00000004>; clock-names = "dspi"; clocks = <0x00000002 0x00000004 0x00000001>; spi-num-chipselects = <0x00000005>; little-endian; status = "disabled"; phandle = <0x00000034>; }; can@2180000 { xen,passthrough; compatible = "fsl,ls1028ar1-flexcan", "fsl,lx2160ar1-flexcan"; reg = <0x00000000 0x02180000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000015 0x00000004>; clocks = <0x00000007 0x00000002 0x00000004 0x00000001>; clock-names = "ipg", "per"; status = "okay"; phandle = <0x00000035>; }; can@2190000 { xen,passthrough; compatible = "fsl,ls1028ar1-flexcan", "fsl,lx2160ar1-flexcan"; reg = <0x00000000 0x02190000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000016 0x00000004>; clocks = <0x00000007 0x00000002 0x00000004 0x00000001>; clock-names = "ipg", "per"; status = "disabled"; phandle = <0x00000036>; }; serial@21c0500 { compatible = "fsl,ns16550", "ns16550a"; reg = <0x00000000 0x021c0500 0x00000000 0x00000100>; interrupts = <0x00000000 0x00000020 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; status = "okay"; phandle = <0x00000037>; }; serial@21c0600 { xen,passthrough; compatible = "fsl,ns16550", "ns16550a"; reg = <0x00000000 0x021c0600 0x00000000 0x00000100>; interrupts = <0x00000000 0x00000020 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; status = "okay"; phandle = <0x00000038>; }; serial@2260000 { compatible = "fsl,ls1021a-lpuart"; reg = <0x00000000 0x02260000 0x00000000 0x00001000>; interrupts = <0x00000000 0x000000e8 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; clock-names = "ipg"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000021 0x00000008 0x00000001 0x00000020>; little-endian; status = "disabled"; phandle = <0x00000039>; }; serial@2270000 { compatible = "fsl,ls1021a-lpuart"; reg = <0x00000000 0x02270000 0x00000000 0x00001000>; interrupts = <0x00000000 0x000000e9 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; clock-names = "ipg"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000001f 0x00000008 0x00000001 0x0000001e>; little-endian; status = "okay"; phandle = <0x0000003a>; }; serial@2280000 { compatible = "fsl,ls1021a-lpuart"; reg = <0x00000000 0x02280000 0x00000000 0x00001000>; interrupts = <0x00000000 0x000000ea 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; clock-names = "ipg"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000001d 0x00000008 0x00000001 0x0000001c>; little-endian; status = "disabled"; phandle = <0x0000003b>; }; serial@2290000 { compatible = "fsl,ls1021a-lpuart"; reg = <0x00000000 0x02290000 0x00000000 0x00001000>; interrupts = <0x00000000 0x000000eb 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; clock-names = "ipg"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000001b 0x00000008 0x00000001 0x0000001a>; little-endian; status = "disabled"; phandle = <0x0000003c>; }; serial@22a0000 { compatible = "fsl,ls1021a-lpuart"; reg = <0x00000000 0x022a0000 0x00000000 0x00001000>; interrupts = <0x00000000 0x000000ec 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; clock-names = "ipg"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000019 0x00000008 0x00000001 0x00000018>; little-endian; status = "disabled"; phandle = <0x0000003d>; }; serial@22b0000 { compatible = "fsl,ls1021a-lpuart"; reg = <0x00000000 0x022b0000 0x00000000 0x00001000>; interrupts = <0x00000000 0x000000ed 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; clock-names = "ipg"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000017 0x00000008 0x00000001 0x00000016>; little-endian; status = "disabled"; phandle = <0x0000003e>; }; dma-controller@22c0000 { #stream-id-cells = <0x00000001>; #dma-cells = <0x00000002>; compatible = "fsl,vf610-edma"; reg = <0x00000000 0x022c0000 0x00000000 0x00010000 0x00000000 0x022d0000 0x00000000 0x00010000 0x00000000 0x022e0000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000038 0x00000004 0x00000000 0x00000038 0x00000004>; interrupt-names = "edma-tx", "edma-err"; dma-channels = <0x00000020>; clock-names = "dmamux0", "dmamux1"; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; phandle = <0x00000008>; }; gpio@2300000 { compatible = "fsl,ls1028a-gpio", "fsl,qoriq-gpio"; reg = <0x00000000 0x02300000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000024 0x00000004>; gpio-controller; #gpio-cells = <0x00000002>; interrupt-controller; #interrupt-cells = <0x00000002>; little-endian; gpio-line-names = [00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 54 44 4f 00 54 43 4b 00 00 00 00 00 00 00 00 00]; phandle = <0x0000003f>; }; gpio@2310000 { compatible = "fsl,ls1028a-gpio", "fsl,qoriq-gpio"; reg = <0x00000000 0x02310000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000024 0x00000004>; gpio-controller; #gpio-cells = <0x00000002>; interrupt-controller; #interrupt-cells = <0x00000002>; little-endian; gpio-line-names = [00 00 00 00 00 00 54 4d 53 00 54 44 49 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00]; phandle = <0x0000000a>; }; gpio@2320000 { compatible = "fsl,ls1028a-gpio", "fsl,qoriq-gpio"; reg = <0x00000000 0x02320000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000025 0x00000004>; gpio-controller; #gpio-cells = <0x00000002>; interrupt-controller; #interrupt-cells = <0x00000002>; little-endian; phandle = <0x00000040>; }; crypto@8000000 { xen,passthrough; compatible = "fsl,sec-v5.0", "fsl,sec-v4.0"; fsl,sec-era = <0x0000000a>; #address-cells = <0x00000001>; #size-cells = <0x00000001>; ranges = <0x00000000 0x00000000 0x08000000 0x00100000>; reg = <0x00000000 0x08000000 0x00000000 0x00100000>; interrupts = <0x00000000 0x0000008b 0x00000004>; dma-coherent; phandle = <0x00000041>; jr@10000 { compatible = "fsl,sec-v5.0-job-ring", "fsl,sec-v4.0-job-ring"; reg = <0x00010000 0x00010000>; interrupts = <0x00000000 0x0000008c 0x00000004>; phandle = <0x00000042>; }; jr@20000 { compatible = "fsl,sec-v5.0-job-ring", "fsl,sec-v4.0-job-ring"; reg = <0x00020000 0x00010000>; interrupts = <0x00000000 0x0000008d 0x00000004>; phandle = <0x00000043>; }; jr@30000 { compatible = "fsl,sec-v5.0-job-ring", "fsl,sec-v4.0-job-ring"; reg = <0x00030000 0x00010000>; interrupts = <0x00000000 0x0000008e 0x00000004>; phandle = <0x00000044>; }; jr@40000 { compatible = "fsl,sec-v5.0-job-ring", "fsl,sec-v4.0-job-ring"; reg = <0x00040000 0x00010000>; interrupts = <0x00000000 0x0000008f 0x00000004>; phandle = <0x00000045>; }; }; wdt@c000000 { compatible = "arm,sp805", "arm,primecell"; reg = <0x00000000 0x0c000000 0x00000000 0x00001000>; clocks = <0x00000002 0x00000004 0x0000000f 0x00000002 0x00000004 0x0000000f>; clock-names = "apb_pclk", "wdog_clk"; phandle = <0x00000046>; }; wdt@c010000 { compatible = "arm,sp805", "arm,primecell"; reg = <0x00000000 0x0c010000 0x00000000 0x00001000>; clocks = <0x00000002 0x00000004 0x0000000f 0x00000002 0x00000004 0x0000000f>; clock-names = "apb_pclk", "wdog_clk"; phandle = <0x00000047>; }; mmc@2140000 { #stream-id-cells = <0x00000001>; compatible = "fsl,ls1028a-esdhc", "fsl,esdhc"; reg = <0x00000000 0x02140000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000001c 0x00000004>; clock-frequency = <0x00000000>; clocks = <0x00000002 0x00000002 0x00000001>; voltage-ranges = <0x00000708 0x00000708 0x00000ce4 0x00000ce4>; sdhci,auto-cmd12; little-endian; bus-width = <0x00000004>; status = "okay"; sd-uhs-sdr104; sd-uhs-sdr50; sd-uhs-sdr25; sd-uhs-sdr12; phandle = <0x00000048>; }; mmc@2150000 { #stream-id-cells = <0x00000001>; compatible = "fsl,ls1028a-esdhc", "fsl,esdhc"; reg = <0x00000000 0x02150000 0x00000000 0x00010000>; interrupts = <0x00000000 0x0000003f 0x00000004>; clock-frequency = <0x00000000>; clocks = <0x00000002 0x00000002 0x00000001>; voltage-ranges = <0x00000708 0x00000708 0x00000ce4 0x00000ce4>; sdhci,auto-cmd12; broken-cd; little-endian; bus-width = <0x00000008>; status = "okay"; mmc-hs200-1_8v; mmc-pwrseq = <0x0000000b>; phandle = <0x00000049>; }; sata@3200000 { compatible = "fsl,ls1028a-ahci"; reg = <0x00000000 0x03200000 0x00000000 0x00010000 0x00000007 0x00100520 0x00000000 0x00000004>; reg-names = "ahci", "sata-ecc"; interrupts = <0x00000000 0x00000085 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001>; status = "disabled"; phandle = <0x0000004a>; }; pcie@3400000 { compatible = "fsl,ls1028a-pcie"; reg = <0x00000000 0x03400000 0x00000000 0x00100000 0x00000080 0x00000000 0x00000000 0x00002000>; reg-names = "regs", "config"; interrupts = <0x00000000 0x0000006c 0x00000004 0x00000000 0x0000006d 0x00000004>; interrupt-names = "pme", "aer"; #address-cells = <0x00000003>; #size-cells = <0x00000002>; device_type = "pci"; dma-coherent; iommu-map = <0x00000000 0x0000000c 0x00000000 0x00000001>; bus-range = <0x00000000 0x000000ff>; ranges = <0x81000000 0x00000000 0x00000000 0x00000080 0x00010000 0x00000000 0x00010000 0x82000000 0x00000000 0x40000000 0x00000080 0x40000000 0x00000000 0x40000000>; msi-parent = <0x0000000d>; #interrupt-cells = <0x00000001>; interrupt-map-mask = <0x00000000 0x00000000 0x00000000 0x00000007>; interrupt-map = * 0x0000000082803bec [0x000000a0]; status = "disabled"; }; pcie@3500000 { compatible = "fsl,ls1028a-pcie"; reg = <0x00000000 0x03500000 0x00000000 0x00100000 0x00000088 0x00000000 0x00000000 0x00002000>; reg-names = "regs", "config"; interrupts = <0x00000000 0x00000071 0x00000004 0x00000000 0x00000072 0x00000004>; interrupt-names = "pme", "aer"; #address-cells = <0x00000003>; #size-cells = <0x00000002>; device_type = "pci"; dma-coherent; iommu-map = <0x00000000 0x0000000c 0x00000000 0x00000001>; bus-range = <0x00000000 0x000000ff>; ranges = <0x81000000 0x00000000 0x00000000 0x00000088 0x00010000 0x00000000 0x00010000 0x82000000 0x00000000 0x40000000 0x00000088 0x40000000 0x00000000 0x40000000>; msi-parent = <0x0000000d>; #interrupt-cells = <0x00000001>; interrupt-map-mask = <0x00000000 0x00000000 0x00000000 0x00000007>; interrupt-map = * 0x0000000082803e50 [0x000000a0]; status = "disabled"; }; pcie@1f0000000 { compatible = "pci-host-ecam-generic"; reg = <0x00000001 0xf0000000 0x00000000 0x00100000>; #address-cells = <0x00000003>; #size-cells = <0x00000002>; msi-parent = <0x0000000d>; device_type = "pci"; bus-range = <0x00000000 0x00000000>; dma-coherent; msi-map = <0x00000000 0x0000000d 0x00000017 0x0000000e>; iommu-map = <0x00000000 0x0000000c 0x00000017 0x0000000e>; ranges = * 0x0000000082804004 [0x000000c4]; ethernet@0,0 { #stream-id-cells = <0x00000001>; compatible = "fsl,enetc"; reg = <0x00000000 0x00000000 0x00000000 0x00000000 0x00000000>; phy-handle = <0x0000000e>; phy-connection-type = "sgmii"; status = "okay"; phandle = <0x0000004b>; }; ethernet@0,1 { #stream-id-cells = <0x00000001>; compatible = "fsl,enetc"; reg = <0x00000100 0x00000000 0x00000000 0x00000000 0x00000000>; phy-handle = <0x0000000f>; phy-connection-type = "rgmii"; status = "okay"; phandle = <0x0000004c>; }; ethernet@0,2 { compatible = "fsl,enetc"; reg = <0x00000200 0x00000000 0x00000000 0x00000000 0x00000000>; status = "disabled"; phandle = <0x0000004d>; fixed-link { speed = <0x000003e8>; full-duplex; }; }; mdio@0,3 { compatible = "fsl,enetc-mdio"; reg = <0x00000300 0x00000000 0x00000000 0x00000000 0x00000000>; #address-cells = <0x00000001>; #size-cells = <0x00000000>; phandle = <0x0000004e>; ethernet-phy@5 { reg = <0x00000005>; phandle = <0x0000000e>; }; ethernet-phy@4 { reg = <0x00000004>; status = "okay"; phandle = <0x0000000f>; }; }; ethernet@0,4 { compatible = "fsl,enetc-ptp"; reg = <0x00000400 0x00000000 0x00000000 0x00000000 0x00000000>; clocks = <0x00000002 0x00000004 0x00000000>; little-endian; }; switch@0,5 { compatible = "mscc,felix-switch"; reg = <0x00000500 0x00000000 0x00000000 0x00000000 0x00000000>; interrupts = <0x00000000 0x0000005f 0x00000004>; ports { #address-cells = <0x00000001>; #size-cells = <0x00000000>; port@0 { reg = <0x00000000>; status = "disabled"; phandle = <0x0000004f>; }; port@1 { reg = <0x00000001>; status = "disabled"; phandle = <0x00000050>; }; port@2 { reg = <0x00000002>; status = "disabled"; phandle = <0x00000051>; }; port@3 { reg = <0x00000003>; status = "disabled"; phandle = <0x00000052>; }; port@4 { reg = <0x00000004>; status = "disabled"; phandle = <0x00000053>; fixed-link { speed = <0x000003e8>; full-duplex; }; }; port@5 { reg = <0x00000005>; ethernet = <0x00000010>; status = "disabled"; phandle = <0x00000054>; fixed-link { speed = <0x000003e8>; full-duplex; }; }; }; }; ethernet@0,6 { compatible = "fsl,enetc"; reg = <0x00000600 0x00000000 0x00000000 0x00000000 0x00000000>; status = "disabled"; phandle = <0x00000010>; fixed-link { speed = <0x000003e8>; full-duplex; }; }; }; iommu@5000000 { mmu-masters = <0x00000049 0x00000800 0x00000057 0x00000012 0x0000004b 0x00000417 0x0000004c 0x00000418 0x00000008 0x00000020>; compatible = "arm,mmu-500"; reg = <0x00000000 0x05000000 0x00000000 0x00800000>; #global-interrupts = <0x00000008>; #iommu-cells = <0x00000000>; stream-match-mask = <0x00007c00>; interrupts = * 0x0000000082804864 [0x00000360]; phandle = <0x0000000c>; }; dma-controller@8380000 { #stream-id-cells = <0x00000001>; compatible = "fsl,ls1028a-qdma", "fsl,ls1021a-qdma"; reg = <0x00000000 0x08380000 0x00000000 0x00001000 0x00000000 0x08390000 0x00000000 0x00010000 0x00000000 0x083a0000 0x00000000 0x00040000>; interrupts = <0x00000000 0x0000002b 0x00000004 0x00000000 0x000000fb 0x00000004 0x00000000 0x000000fc 0x00000004 0x00000000 0x000000fd 0x00000004 0x00000000 0x000000fe 0x00000004>; interrupt-names = "qdma-error", "qdma-queue0", "qdma-queue1", "qdma-queue2", "qdma-queue3"; channels = <0x00000008>; block-number = <0x00000001>; block-offset = <0x00010000>; queues = <0x00000002>; status-sizes = <0x00000040>; queue-sizes = <0x00000040 0x00000040>; phandle = <0x00000057>; }; gpu@f0c0000 { xen,passthrough; compatible = "fsl,ls1028a-gpu"; reg = <0x00000000 0x0f0c0000 0x00000000 0x00010000 0x00000000 0x80000000 0x00000000 0x80000000 0x00000000 0x00000000 0x00000000 0x03000000>; reg-names = "base", "phys_baseaddr", "contiguous_mem"; interrupts = <0x00000000 0x000000dc 0x00000004>; }; audio-controller@f100000 { #sound-dai-cells = <0x00000000>; compatible = "fsl,vf610-sai"; reg = <0x00000000 0x0f100000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000052 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; clock-names = "bus", "mclk1", "mclk2", "mclk3"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000004 0x00000008 0x00000001 0x00000003>; status = "disabled"; phandle = <0x00000058>; }; audio-controller@f110000 { #sound-dai-cells = <0x00000000>; compatible = "fsl,vf610-sai"; reg = <0x00000000 0x0f110000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000052 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; clock-names = "bus", "mclk1", "mclk2", "mclk3"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000006 0x00000008 0x00000001 0x00000005>; status = "disabled"; phandle = <0x00000059>; }; audio-controller@f120000 { #sound-dai-cells = <0x00000000>; compatible = "fsl,vf610-sai"; reg = <0x00000000 0x0f120000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000053 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; clock-names = "bus", "mclk1", "mclk2", "mclk3"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x00000008 0x00000008 0x00000001 0x00000007>; status = "disabled"; phandle = <0x0000005a>; }; audio-controller@f130000 { #sound-dai-cells = <0x00000000>; compatible = "fsl,vf610-sai"; reg = <0x00000000 0x0f130000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000053 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; clock-names = "bus", "mclk1", "mclk2", "mclk3"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000000a 0x00000008 0x00000001 0x00000009>; status = "disabled"; phandle = <0x0000005b>; }; audio-controller@f140000 { #sound-dai-cells = <0x00000000>; compatible = "fsl,vf610-sai"; reg = <0x00000000 0x0f140000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000054 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; clock-names = "bus", "mclk1", "mclk2", "mclk3"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000000c 0x00000008 0x00000001 0x0000000b>; status = "disabled"; phandle = <0x0000005c>; }; audio-controller@f150000 { #sound-dai-cells = <0x00000000>; compatible = "fsl,vf610-sai"; reg = <0x00000000 0x0f150000 0x00000000 0x00010000>; interrupts = <0x00000000 0x00000054 0x00000004>; clocks = <0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001 0x00000002 0x00000004 0x00000001>; clock-names = "bus", "mclk1", "mclk2", "mclk3"; dma-names = "tx", "rx"; dmas = <0x00000008 0x00000001 0x0000000e 0x00000008 0x00000001 0x0000000d>; status = "disabled"; phandle = <0x0000005d>; }; rcpm@1e34040 { compatible = "fsl,ls1028a-rcpm", "fsl,qoriq-rcpm-2.1+"; reg = <0x00000000 0x01e34040 0x00000000 0x0000001c>; #fsl,rcpm-wakeup-cells = <0x00000007>; little-endian; phandle = <0x00000016>; }; timer@2800000 { compatible = "fsl,ls1028a-ftm-alarm"; reg = <0x00000000 0x02800000 0x00000000 0x00010000>; fsl,rcpm-wakeup = <0x00000016 0x00000000 0x00000000 0x00000000 0x00000000 0x00004000 0x00000000 0x00000000>; interrupts = <0x00000000 0x0000002c 0x00000004>; phandle = <0x0000005e>; }; }; malidp@f080000 { xen,passthrough; compatible = "arm,mali-dp500"; reg = <0x00000000 0x0f080000 0x00000000 0x00010000>; interrupts = <0x00000000 0x000000de 0x00000004 0x00000000 0x000000df 0x00000004>; interrupt-names = "DE", "SE"; clocks = <0x00000017 0x00000018 0x00000018 0x00000019>; clock-names = "pxlclk", "mclk", "aclk", "pclk"; arm,malidp-output-port-lines = [08 08 08]; phandle = <0x0000005f>; port { endpoint { remote-endpoint = <0x0000001a>; phandle = <0x0000001c>; }; }; }; hdp@f200000 { xen,passthrough; compatible = "fsl,ls1028a-dp"; reg = <0x00000000 0x0f200000 0x00000000 0x000fffff>; interrupts = <0x00000000 0x000000dd 0x00000004>; clocks = <0x00000007 0x00000002 0x00000002 0x00000002 0x0000001b 0x0000001b 0x0000001b 0x0000001b 0x0000001b>; clock-names = "clk_ipg", "clk_core", "clk_pxl", "clk_pxl_mux", "clk_pxl_link", "clk_apb", "clk_vif"; fsl,no_edid; resolution = "3840x2160@60", "1920x1080@60", "1280x720@60", "720x480@60"; lane_mapping = <0x0000004e>; edp_link_rate = <0x00000006>; edp_num_lanes = <0x00000004>; status = "okay"; phandle = <0x00000060>; port { endpoint { remote-endpoint = <0x0000001c>; phandle = <0x0000001a>; }; }; }; emmc-pwrseq { compatible = "mmc-pwrseq-simple"; reset-gpios = <0x0000001d 0x00000002 0x00000000>; phandle = <0x0000000b>; }; buttons0 { compatible = "gpio-keys"; power-button { interrupt-parent = <0x00000009>; interrupts = <0x00000004 0x00000000>; linux,code = <0x00000074>; label = "Power"; wakeup-source; }; sleep-button { interrupt-parent = <0x00000009>; interrupts = <0x00000005 0x00000000>; linux,code = <0x0000008e>; label = "Sleep"; wakeup-source; }; }; buttons1 { compatible = "gpio-keys-polled"; poll-interval = <0x000000c8>; lid_switch { linux,input-type = <0x00000005>; linux,code = <0x00000000>; gpios = <0x0000001e 0x00000004 0x00000001>; label = "Lid"; phandle = <0x00000061>; }; }; charger { compatible = "gpio-charger"; charger-type = "battery"; gpios = <0x0000001e 0x00000006 0x00000001>; charging-gpio = <0x0000001e 0x00000005 0x00000001>; bat-low-gpio = <0x0000001e 0x00000003 0x00000001>; }; __symbols__ { cpu0 = "/cpus/cpu@0"; cpu1 = "/cpus/cpu@1"; l2 = "/cpus/l2-cache"; CPU_PW20 = "/idle-states/cpu-pw20"; sysclk = "/clock-sysclk"; osc_27m = "/clock-osc-27m"; dpclk = "/clock-display@f1f0000"; aclk = "/clock-axi"; pclk = "/clock-apb"; hdpclk = "/clock-hdpcore"; gic = "/interrupt-controller@6000000"; its = "/interrupt-controller@6000000/gic-its@6020000"; soc = "/soc"; ddr = "/soc/memory-controller@1080000"; dcfg = "/soc/syscon@1e00000"; rst = "/soc/syscon@1e60000"; scfg = "/soc/syscon@1fc0000"; clockgen = "/soc/clock-controller@1300000"; usb0 = "/soc/usb@3100000"; usb1 = "/soc/usb@3110000"; fspi = "/soc/spi@20c0000"; i2c0 = "/soc/i2c@2000000"; sl28cpld = "/soc/i2c@2000000/sl28cpld@4a"; pwm0 = "/soc/i2c@2000000/sl28cpld@4a/pwm0@c"; pwm1 = "/soc/i2c@2000000/sl28cpld@4a/pwm1@e"; cpld_gpio0 = "/soc/i2c@2000000/sl28cpld@4a/gpio0@10"; cpld_gpio1 = "/soc/i2c@2000000/sl28cpld@4a/gpio1@15"; cpld_gpo0 = "/soc/i2c@2000000/sl28cpld@4a/gpo0@1a"; cpld_gpi0 = "/soc/i2c@2000000/sl28cpld@4a/gpi0@1b"; i2c1 = "/soc/i2c@2010000"; i2c2 = "/soc/i2c@2020000"; i2c3 = "/soc/i2c@2030000"; i2c4 = "/soc/i2c@2040000"; i2c5 = "/soc/i2c@2050000"; i2c6 = "/soc/i2c@2060000"; i2c7 = "/soc/i2c@2070000"; dspi0 = "/soc/spi@2100000"; dspi1 = "/soc/spi@2110000"; dspi2 = "/soc/spi@2120000"; can0 = "/soc/can@2180000"; can1 = "/soc/can@2190000"; duart0 = "/soc/serial@21c0500"; duart1 = "/soc/serial@21c0600"; lpuart0 = "/soc/serial@2260000"; lpuart1 = "/soc/serial@2270000"; lpuart2 = "/soc/serial@2280000"; lpuart3 = "/soc/serial@2290000"; lpuart4 = "/soc/serial@22a0000"; lpuart5 = "/soc/serial@22b0000"; edma0 = "/soc/dma-controller@22c0000"; gpio1 = "/soc/gpio@2300000"; gpio2 = "/soc/gpio@2310000"; gpio3 = "/soc/gpio@2320000"; crypto = "/soc/crypto@8000000"; sec_jr0 = "/soc/crypto@8000000/jr@10000"; sec_jr1 = "/soc/crypto@8000000/jr@20000"; sec_jr2 = "/soc/crypto@8000000/jr@30000"; sec_jr3 = "/soc/crypto@8000000/jr@40000"; cluster1_core0_watchdog = "/soc/wdt@c000000"; cluster1_core1_watchdog = "/soc/wdt@c010000"; esdhc = "/soc/mmc@2140000"; esdhc1 = "/soc/mmc@2150000"; sata = "/soc/sata@3200000"; enetc_port0 = "/soc/pcie@1f0000000/ethernet@0,0"; enetc_port1 = "/soc/pcie@1f0000000/ethernet@0,1"; enetc_port2 = "/soc/pcie@1f0000000/ethernet@0,2"; enetc_mdio_pf3 = "/soc/pcie@1f0000000/mdio@0,3"; phy0 = "/soc/pcie@1f0000000/mdio@0,3/ethernet-phy@5"; phy1 = "/soc/pcie@1f0000000/mdio@0,3/ethernet-phy@4"; switch_port0 = "/soc/pcie@1f0000000/switch@0,5/ports/port@0"; switch_port1 = "/soc/pcie@1f0000000/switch@0,5/ports/port@1"; switch_port2 = "/soc/pcie@1f0000000/switch@0,5/ports/port@2"; switch_port3 = "/soc/pcie@1f0000000/switch@0,5/ports/port@3"; switch_port4 = "/soc/pcie@1f0000000/switch@0,5/ports/port@4"; switch_port5 = "/soc/pcie@1f0000000/switch@0,5/ports/port@5"; enetc_port3 = "/soc/pcie@1f0000000/ethernet@0,6"; tmu = "/soc/tmu@1f00000"; core_cluster_alert = "/soc/thermal-zones/core-cluster/trips/core-cluster-alert"; core_cluster_crit = "/soc/thermal-zones/core-cluster/trips/core-cluster-crit"; ddr_controller_alert = "/soc/thermal-zones/ddr-controller/trips/ddr-controller-alert"; ddr_controller_crit = "/soc/thermal-zones/ddr-controller/trips/ddr-controller-crit"; smmu = "/soc/iommu@5000000"; qdma = "/soc/dma-controller@8380000"; sai1 = "/soc/audio-controller@f100000"; sai2 = "/soc/audio-controller@f110000"; sai3 = "/soc/audio-controller@f120000"; sai4 = "/soc/audio-controller@f130000"; sai5 = "/soc/audio-controller@f140000"; sai6 = "/soc/audio-controller@f150000"; rcpm = "/soc/rcpm@1e34040"; ftm_alarm0 = "/soc/timer@2800000"; display0 = "/malidp@f080000"; dp0_out = "/malidp@f080000/port/endpoint"; display1 = "/hdp@f200000"; dp1_out = "/hdp@f200000/port/endpoint"; emmc_pwrseq = "/emmc-pwrseq"; lid_sw = "/buttons1/lid_switch"; }; }; ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2020-11-17 23:53 ` AW: AW: AW: AW: Xen data from meta-virtualization layer Stefano Stabellini @ 2020-11-18 12:03 ` Rahul Singh 2020-11-18 12:23 ` AW: AW: AW: AW: " Julien Grall 1 sibling, 0 replies; 16+ messages in thread From: Rahul Singh @ 2020-11-18 12:03 UTC (permalink / raw) To: Stefano Stabellini Cc: Leo Krueger, Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand Marquis, julien Hello Stefano, > On 17 Nov 2020, at 11:53 pm, Stefano Stabellini <stefano.stabellini@xilinx.com> wrote: > > Adding Bertrand, Oleksandr, Julien, and others -- they have a more > recent experience with GICv3 ITS than me and might be able to help. > I am attaching the device tree Leo sent a few days ago for reference. > > > Typically when you can set the ethernet link up and no packets are > exchanged it is because of a missing interrupt. In this case a missing > MSI. > > Bertrand, I believe you tried the GIC ITS driver with PCI devices > recently. It is expected to work correctly with MSIs in Dom0, right? > Yes we are using the XEN GIC ITS driver and MSI interrupts is working fine in DOM0. 20: 112 0 0 0 ITS-MSI 1572864 Edge eth0 21: 441 0 0 0 ITS-MSI 3670016 Edge eth1 22: 4286 0 0 0 ITS-MSI 4194304 Edge xhci_hcd Regards, Rahul > > On Tue, 17 Nov 2020, Leo Krueger wrote: >> Hi, >> >> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it before...) but then had to add the following node to my device tree >> >> gic_lpi_base: syscon@0x80000000 { >> compatible = "gic-lpi-base"; >> reg = <0x0 0x80000000 0x0 0x100000>; >> max-gic-redistributors = <2>; >> }; >> >> to somehow change something in regard to the ITS and MSI/MSI-X >> >> (XEN) GICv3 initialization: >> (XEN) gic_dist_addr=0x00000006000000 >> (XEN) gic_maintenance_irq=25 >> (XEN) gic_rdist_stride=0 >> (XEN) gic_rdist_regions=1 >> (XEN) redistributor regions: >> (XEN) - region 0: 0x00000006040000 - 0x00000006080000 >> (XEN) GICv3: using at most 57344 LPIs on the host. >> (XEN) GICv3: 288 lines, (IID 0001143b). >> (XEN) GICv3: Found ITS @0x6020000 >> (XEN) using non-cacheable ITS command queue >> (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000 >> >> [ 0.000000] GICv3: Distributor has no Range Selector support >> [ 0.000000] GICv3: no VLPI support, no direct LPI support >> [ 0.000000] ITS [mem 0x06020000-0x0603ffff] >> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 (flat, esz 8, psz 64K, shr 1) >> [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt Collections @dc820000 (flat, esz 2, psz 64K, shr 1) >> [ 0.000000] GIC: using LPI property table @0x00000000dc830000 >> [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000006040000 >> [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000 >> ... >> [ 0.040080] Platform MSI: gic-its domain created >> [ 0.040136] PCI/MSI: /interrupt-controller/gic-its domain created >> [ 0.040181] fsl-mc MSI: /interrupt-controller/gic-its domain created >> >> >> Still I am ending up with the " Failed to add - passthrough or MSI/MSI-X might fail!" log messages for some PCI devices, but at least the on-board ethernet ports (fsl_enetc ) are initialized. >> I can set the link up and a link is successfully established. >> >> But (!) I cannot receive or transmit anything (no error message...) and there seem to be no interrupts: >> >> 29: 0 ITS-MSI 1 Edge gbe0-rxtx0 >> 32: 0 ITS-MSI 8192 Edge ptp_qoriq >> >> (from /proc/interrupts). >> >> Any idea on this one? I keep digging and checking whether my device tree needs some additional fixes. >> >> Kind regards, >> Leo >> >> -- >> Leo Krüger, M.Sc. >> Senior Systems Engineer Distributed Systems >> Intelligent Digital Cabin >> >> ZAL Zentrum für Angewandte Luftfahrtforschung GmbH >> Hein-Saß-Weg 22 >> 21129 Hamburg >> >> +49 (0) 40 248 595-154 >> >> zal.aero | twitter.com/ZALTechCenter | facebook.com/ZALTechCenter >> >> ZAL Zentrum für Angewandte Luftfahrtforschung GmbH >> Sitz der Gesellschaft / Legal Domicile: Hamburg >> Registergericht / Registration Court: Amtsgericht Hamburg HRB 110232 >> Vorsitzender des Aufsichtsrates / Chairman of the Supervisory Board: StR Andreas Rieckhof >> Geschäftsführung / Board of Management: Roland Gerhards >> >> Disclaimer: >> This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have >> received this mail in error), please notify the sender immediately and destroy this e-mail. Any unauthorised copying, >> disclosure or distribution of the material in this e-mail is strictly forbidden. >> >>> -----Ursprüngliche Nachricht----- >>> Von: Stefano Stabellini <stefano.stabellini@xilinx.com> >>> Gesendet: Dienstag, 17. November 2020 01:59 >>> An: Leo Krueger <leo.krueger@zal.aero> >>> Cc: Peng Fan <peng.fan@nxp.com>; Stefano Stabellini >>> <stefano.stabellini@xilinx.com>; brucea@xilinx.com; Cornelia Bruelhart >>> <cornelia.bruelhart@zal.aero> >>> Betreff: Re: AW: AW: AW: Xen data from meta-virtualization layer >>> >>> Replies inline below >>> >>> >>> On Sun, 15 Nov 2020, Leo Krueger wrote: >>>> Hi Peng, hi Stefano, >>>> >>>> >>>> >>>> sorry for the long silence… >>>> >>>> >>>> >>>> I tried the change suggested (and hope I didn’t do anything wrong >>>> while doing so…) on top of XEN 4.13.2 (before, I always tried with >>>> 4.12 but wanted to give 4.13.2 a try as well) but I do not see any difference, >>> still the same “unhandled context fault” log entries pop up and I cannot >>> access my sdcard. >>>> >>>> >>>> >>>> As it seems to work without respectively disabled iommu, that would be >>>> fine for me for now. What I am worried about a bit more is PCIe or >>> MSI/MSIX to be exact. >>>> >>>> >>>> >>>> Here is the gic-v3 and its node from my device tree: >>>> >>>> >>>> >>>> interrupt-controller@6000000 { >>>> >>>> compatible = "arm,gic-v3"; >>>> >>>> #address-cells = <0x2>; >>>> >>>> #size-cells = <0x2>; >>>> >>>> ranges; >>>> >>>> reg = <0x0 0x6000000 0x0 0x10000 0x0 0x6040000 0x0 0x40000>; >>>> >>>> #interrupt-cells = <0x3>; >>>> >>>> interrupt-controller; >>>> >>>> interrupts = <0x1 0x9 0xf08>; >>>> >>>> phandle = <0x1>; >>>> >>>> >>>> >>>> gic-its@6020000 { >>>> >>>> compatible = "arm,gic-v3-its"; >>>> >>>> msi-controller; >>>> >>>> reg = <0x0 0x6020000 0x0 0x20000>; >>>> >>>> phandle = <0xd>; >>>> >>>> }; >>>> >>>> }; >>>> >>>> >>>> >>>> And here are some kernel log excerpts related to GIC when booting >>> without (!) XEN: >>>> >>>> >>>> >>>> [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode >>>> >>>> [ 0.000000] GICv3: Distributor has no Range Selector support >>>> >>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support >>>> >>>> [ 0.000000] ITS [mem 0x06020000-0x0603ffff] >>>> >>>> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices >>>> @20fb880000 (flat, esz 8, psz 64K, shr 0) >>>> >>>> [ 0.000000] ITS: using cache flushing for cmd queue >>>> >>>> [ 0.000000] GIC: using LPI property table @0x00000020fb830000 >>>> >>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region >>>> 0:0x0000000006040000 >>>> >>>> [ 0.000000] CPU0: using LPI pending table @0x00000020fb840000 >>>> >>>> [ 0.000000] GIC: using cache flushing for LPI property table >>>> >>>> >>>> >>>> However, when booting with XEN, only the following three lines are >>> contained in the kernel log: >>>> >>>> >>>> >>>> [ 0.000000] GICv3: Distributor has no Range Selector support >>>> >>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support >>>> >>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region >>>> 0:0x0000000006040000 >>> >>> "no VLPI support" is very suspicious, it looks like Dom0 doesn't find any ITS >>> support. >>> >>> Can you double check that you have the ITS driver in Xen built-in? That would >>> be CONFIG_HAS_ITS. If you do "make menuconfig" and enable "Configure >>> standard Xen features (expert users)" you should get a new option "GICv3 >>> ITS MSI controller support" under "Architecture Features". >>> Make sure to enable it. >>> >>> Let me know if that works! > <devicetree.dts> Regards, Rahul ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: AW: AW: AW: AW: Xen data from meta-virtualization layer 2020-11-17 23:53 ` AW: AW: AW: AW: Xen data from meta-virtualization layer Stefano Stabellini 2020-11-18 12:03 ` Rahul Singh @ 2020-11-18 12:23 ` Julien Grall 2020-11-22 22:55 ` AW: " Leo Krueger 1 sibling, 1 reply; 16+ messages in thread From: Julien Grall @ 2020-11-18 12:23 UTC (permalink / raw) To: Stefano Stabellini, Leo Krueger Cc: Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand.Marquis Hi, On 17/11/2020 23:53, Stefano Stabellini wrote: > Adding Bertrand, Oleksandr, Julien, and others -- they have a more > recent experience with GICv3 ITS than me and might be able to help. > I am attaching the device tree Leo sent a few days ago for reference. > > > Typically when you can set the ethernet link up and no packets are > exchanged it is because of a missing interrupt. In this case a missing > MSI. > > Bertrand, I believe you tried the GIC ITS driver with PCI devices > recently. It is expected to work correctly with MSIs in Dom0, right? OSSTest has some hardware (e.g. Thunder-X) where ITS is required to boot Dom0. I haven't seen any failure on recent Xen. We are testing 4.11 and onwards on Thunder-X. However, it may be possible that some more work is necessary for other hardware (e.g. workaround, missing code...). See more below. > > > > On Tue, 17 Nov 2020, Leo Krueger wrote: >> Hi, >> >> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it before...) but then had to add the following node to my device tree >> >> gic_lpi_base: syscon@0x80000000 { >> compatible = "gic-lpi-base"; I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, could you clarify which flavor/version of Linux you are using? >> reg = <0x0 0x80000000 0x0 0x100000>; >> max-gic-redistributors = <2>; >> }; >> >> to somehow change something in regard to the ITS and MSI/MSI-X >> >> (XEN) GICv3 initialization: >> (XEN) gic_dist_addr=0x00000006000000 >> (XEN) gic_maintenance_irq=25 >> (XEN) gic_rdist_stride=0 >> (XEN) gic_rdist_regions=1 >> (XEN) redistributor regions: >> (XEN) - region 0: 0x00000006040000 - 0x00000006080000 >> (XEN) GICv3: using at most 57344 LPIs on the host. >> (XEN) GICv3: 288 lines, (IID 0001143b). >> (XEN) GICv3: Found ITS @0x6020000 >> (XEN) using non-cacheable ITS command queue >> (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000 >> >> [ 0.000000] GICv3: Distributor has no Range Selector support >> [ 0.000000] GICv3: no VLPI support, no direct LPI support >> [ 0.000000] ITS [mem 0x06020000-0x0603ffff] >> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 (flat, esz 8, psz 64K, shr 1) >> [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt Collections @dc820000 (flat, esz 2, psz 64K, shr 1) >> [ 0.000000] GIC: using LPI property table @0x00000000dc830000 >> [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000006040000 >> [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000 >> ... >> [ 0.040080] Platform MSI: gic-its domain created >> [ 0.040136] PCI/MSI: /interrupt-controller/gic-its domain created >> [ 0.040181] fsl-mc MSI: /interrupt-controller/gic-its domain created >> >> >> Still I am ending up with the " Failed to add - passthrough or MSI/MSI-X might fail!" log messages for some PCI devices, but at least the on-board ethernet ports (fsl_enetc ) are initialized. >> I can set the link up and a link is successfully established. This message is normal. Xen on Arm is not yet aware of PCI devices and therefore the hypercalls to add PCI devices will return -EOPNOTSUPP. However, this is not an issue because the virtual ITS implementation will allow dom0 to configure any devices. >> >> But (!) I cannot receive or transmit anything (no error message...) and there seem to be no interrupts: >> >> 29: 0 ITS-MSI 1 Edge gbe0-rxtx0 >> 32: 0 ITS-MSI 8192 Edge ptp_qoriq >> >> (from /proc/interrupts). >> >> Any idea on this one? I keep digging and checking whether my device tree needs some additional fixes. Can you apply patch [1] and provide the logs? This will dump more information about the command received by the vITS and the one send to the host ITS. Note that Xen will need to be build with CONFIG_DEBUG=y in order to get some of the messages. [...] >>>> [ 0.000000] GICv3: Distributor has no Range Selector support >>>> >>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support >>>> >>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region >>>> 0:0x0000000006040000 >>> >>> "no VLPI support" is very suspicious, it looks like Dom0 doesn't find any ITS >>> support. VLPI is a feature that was introduced in GICv4 to directly inject LPI in the guest. So this is normal to see this message when running on Xen because we are going to only expose a GICv3 to a domain until at least we support nested virt. However, you were right about that Xen didn't expose the ITS because the following lines were missing: [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 (flat, esz 8, psz 64K, shr 1) Cheers, [1] diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 9558bad96ac3..8a0a02308e74 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -87,6 +87,10 @@ static int its_send_command(struct host_its *hw_its, const void *its_cmd) /* No ITS commands from an interrupt handler (at the moment). */ ASSERT(!in_irq()); + printk(XENLOG_WARNING, "pITS cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", + its_cmd_get_command(command), + command[0], command[1], command[2], command[3]); + spin_lock(&hw_its->cmd_lock); do { diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index 869bc97fa1aa..e7c5bcd8d423 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -183,7 +183,10 @@ void gicv3_do_LPI(unsigned int lpi) /* Find out if a guest mapped something to this physical LPI. */ hlpip = gic_get_host_lpi(lpi); if ( !hlpip ) + { + printk("%s: Received LPI %u but it is not mapped?\n", __func__, lpi); goto out; + } hlpi.data = read_u64_atomic(&hlpip->data); @@ -222,6 +225,9 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id, { union host_lpi *hlpip, hlpi; + printk("%s: host_lpi %u domain %d virq_lpi %u\n", + __func__, host_lpi, domain_id, virq_lpi); + ASSERT(host_lpi >= LPI_OFFSET); host_lpi -= LPI_OFFSET; diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c index 58d939b85f92..89ef137b3e6b 100644 --- a/xen/arch/arm/vgic-v3-its.c +++ b/xen/arch/arm/vgic-v3-its.c @@ -897,7 +897,7 @@ out_unlock: static void dump_its_command(uint64_t *command) { - gdprintk(XENLOG_WARNING, " cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", + gdprintk(XENLOG_WARNING, "vITS cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", its_cmd_get_command(command), command[0], command[1], command[2], command[3]); } @@ -926,6 +926,8 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its) if ( ret ) return ret; + dump_its_command(command); + switch ( its_cmd_get_command(command) ) { case GITS_CMD_CLEAR: -- Julien Grall ^ permalink raw reply related [flat|nested] 16+ messages in thread
* AW: AW: AW: AW: AW: Xen data from meta-virtualization layer 2020-11-18 12:23 ` AW: AW: AW: AW: " Julien Grall @ 2020-11-22 22:55 ` Leo Krueger 2020-11-23 11:41 ` Rahul Singh 2020-11-23 18:27 ` AW: AW: AW: AW: " Julien Grall 0 siblings, 2 replies; 16+ messages in thread From: Leo Krueger @ 2020-11-22 22:55 UTC (permalink / raw) To: Julien Grall, Stefano Stabellini Cc: Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand.Marquis [-- Attachment #1: Type: text/plain, Size: 12047 bytes --] Hi Julien, finally I could try out what you suggested, please find my answers inline. > -----Ursprüngliche Nachricht----- > Von: Julien Grall <julien@xen.org> > Gesendet: Mittwoch, 18. November 2020 13:24 > An: Stefano Stabellini <stefano.stabellini@xilinx.com>; Leo Krueger > <leo.krueger@zal.aero> > Cc: Peng Fan <peng.fan@nxp.com>; brucea@xilinx.com; Cornelia Bruelhart > <cornelia.bruelhart@zal.aero>; oleksandr_andrushchenko@epam.com; xen- > devel@lists.xenproject.org; Bertrand.Marquis@arm.com > Betreff: Re: AW: AW: AW: AW: Xen data from meta-virtualization layer > > Hi, > > On 17/11/2020 23:53, Stefano Stabellini wrote: > > Adding Bertrand, Oleksandr, Julien, and others -- they have a more > > recent experience with GICv3 ITS than me and might be able to help. > > I am attaching the device tree Leo sent a few days ago for reference. > > > > > > Typically when you can set the ethernet link up and no packets are > > exchanged it is because of a missing interrupt. In this case a missing > > MSI. > > > > Bertrand, I believe you tried the GIC ITS driver with PCI devices > > recently. It is expected to work correctly with MSIs in Dom0, right? > > OSSTest has some hardware (e.g. Thunder-X) where ITS is required to boot > Dom0. I haven't seen any failure on recent Xen. We are testing 4.11 and > onwards on Thunder-X. > > However, it may be possible that some more work is necessary for other > hardware (e.g. workaround, missing code...). See more below. > > > > > > > > > On Tue, 17 Nov 2020, Leo Krueger wrote: > >> Hi, > >> > >> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it > >> before...) but then had to add the following node to my device tree > >> > >> gic_lpi_base: syscon@0x80000000 { > >> compatible = "gic-lpi-base"; > > I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, could > you clarify which flavor/version of Linux you are using? It is Linux 4.19 from Yocto (Warror release). XEN 4.13.2. While searching around the Internet for any solution, I came across [0] which contained the gic-lpi-base node. So I just tried adding it (quite desperate I know) and voila, it at least brought me one step further (XEN exposing the ITS)... > > >> reg = <0x0 0x80000000 0x0 0x100000>; > >> max-gic-redistributors = <2>; > >> }; > >> > >> to somehow change something in regard to the ITS and MSI/MSI-X > >> > >> (XEN) GICv3 initialization: > >> (XEN) gic_dist_addr=0x00000006000000 > >> (XEN) gic_maintenance_irq=25 > >> (XEN) gic_rdist_stride=0 > >> (XEN) gic_rdist_regions=1 > >> (XEN) redistributor regions: > >> (XEN) - region 0: 0x00000006040000 - 0x00000006080000 > >> (XEN) GICv3: using at most 57344 LPIs on the host. > >> (XEN) GICv3: 288 lines, (IID 0001143b). > >> (XEN) GICv3: Found ITS @0x6020000 > >> (XEN) using non-cacheable ITS command queue > >> (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000 > >> > >> [ 0.000000] GICv3: Distributor has no Range Selector support > >> [ 0.000000] GICv3: no VLPI support, no direct LPI support > >> [ 0.000000] ITS [mem 0x06020000-0x0603ffff] > >> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices > @dc880000 (flat, esz 8, psz 64K, shr 1) > >> [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt > Collections @dc820000 (flat, esz 2, psz 64K, shr 1) > >> [ 0.000000] GIC: using LPI property table @0x00000000dc830000 > >> [ 0.000000] GICv3: CPU0: found redistributor 0 region > 0:0x0000000006040000 > >> [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000 > >> ... > >> [ 0.040080] Platform MSI: gic-its domain created > >> [ 0.040136] PCI/MSI: /interrupt-controller/gic-its domain created > >> [ 0.040181] fsl-mc MSI: /interrupt-controller/gic-its domain created > >> > >> > >> Still I am ending up with the " Failed to add - passthrough or MSI/MSI-X > might fail!" log messages for some PCI devices, but at least the on-board > ethernet ports (fsl_enetc ) are initialized. > >> I can set the link up and a link is successfully established. > > This message is normal. Xen on Arm is not yet aware of PCI devices and > therefore the hypercalls to add PCI devices will return -EOPNOTSUPP. > > However, this is not an issue because the virtual ITS implementation will > allow dom0 to configure any devices. > > >> > >> But (!) I cannot receive or transmit anything (no error message...) and > there seem to be no interrupts: > >> > >> 29: 0 ITS-MSI 1 Edge gbe0-rxtx0 > >> 32: 0 ITS-MSI 8192 Edge ptp_qoriq > >> > >> (from /proc/interrupts). > >> > >> Any idea on this one? I keep digging and checking whether my device tree > needs some additional fixes. > > Can you apply patch [1] and provide the logs? This will dump more > information about the command received by the vITS and the one send to > the host ITS. For XEN 4.13.2 I had to adapt your patch slightly [1], see below (yes I know, quite ugly in parts). Find attached the boot log and an output of "xl dmesg" which is truncated due to the large amount of messages. When enabling the network interface (gbe0), the following output is visible: root@kontron-sal28:~# ip link set up dev gbe0 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c 0000000000000001 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 [ 34.034598] Atheros 8031 ethernet 0000:00:00.3:05: attached PHY driver [Atheros 8031 ethernet] (mii_bus:phy_addr=0000:00:00.3:05, irq=POLL) [ 34.041111] 8021q: adding VLAN 0 to HW filter on device gbe0 [ 34.041209] IPv6: ADDRCONF(NETDEV_UP): gbe0: link is not ready root@kontron-sal28:~# [ 35.041951] fsl_enetc 0000:00:00.0 gbe0: Link is Down [ 38.114426] fsl_enetc 0000:00:00.0 gbe0: Link is Up - 1Gbps/Full - flow control off [ 38.114508] IPv6: ADDRCONF(NETDEV_CHANGE): gbe0: link becomes ready Does that tell you anything? > > Note that Xen will need to be build with CONFIG_DEBUG=y in order to get > some of the messages. > > [...] > > >>>> [ 0.000000] GICv3: Distributor has no Range Selector support > >>>> > >>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support > >>>> > >>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region > >>>> 0:0x0000000006040000 > >>> > >>> "no VLPI support" is very suspicious, it looks like Dom0 doesn't > >>> find any ITS support. > VLPI is a feature that was introduced in GICv4 to directly inject LPI in the > guest. So this is normal to see this message when running on Xen because > we are going to only expose a GICv3 to a domain until at least we support > nested virt. > > However, you were right about that Xen didn't expose the ITS because the > following lines were missing: > > [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 > (flat, esz 8, psz 64K, shr 1) > > Cheers, Best regards, Leo > > [1] > diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index > 9558bad96ac3..8a0a02308e74 100644 > --- a/xen/arch/arm/gic-v3-its.c > +++ b/xen/arch/arm/gic-v3-its.c > @@ -87,6 +87,10 @@ static int its_send_command(struct host_its *hw_its, > const void *its_cmd) > /* No ITS commands from an interrupt handler (at the moment). */ > ASSERT(!in_irq()); > > + printk(XENLOG_WARNING, "pITS cmd 0x%02lx: %016lx %016lx %016lx > %016lx\n", > + its_cmd_get_command(command), > + command[0], command[1], command[2], command[3]); > + > spin_lock(&hw_its->cmd_lock); > > do { > diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index > 869bc97fa1aa..e7c5bcd8d423 100644 > --- a/xen/arch/arm/gic-v3-lpi.c > +++ b/xen/arch/arm/gic-v3-lpi.c > @@ -183,7 +183,10 @@ void gicv3_do_LPI(unsigned int lpi) > /* Find out if a guest mapped something to this physical LPI. */ > hlpip = gic_get_host_lpi(lpi); > if ( !hlpip ) > + { > + printk("%s: Received LPI %u but it is not mapped?\n", __func__, > lpi); > goto out; > + } > > hlpi.data = read_u64_atomic(&hlpip->data); > > @@ -222,6 +225,9 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, > int domain_id, > { > union host_lpi *hlpip, hlpi; > > + printk("%s: host_lpi %u domain %d virq_lpi %u\n", > + __func__, host_lpi, domain_id, virq_lpi); > + > ASSERT(host_lpi >= LPI_OFFSET); > > host_lpi -= LPI_OFFSET; > diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c index > 58d939b85f92..89ef137b3e6b 100644 > --- a/xen/arch/arm/vgic-v3-its.c > +++ b/xen/arch/arm/vgic-v3-its.c > @@ -897,7 +897,7 @@ out_unlock: > > static void dump_its_command(uint64_t *command) > { > - gdprintk(XENLOG_WARNING, " cmd 0x%02lx: %016lx %016lx %016lx > %016lx\n", > + gdprintk(XENLOG_WARNING, "vITS cmd 0x%02lx: %016lx %016lx %016lx > %016lx\n", > its_cmd_get_command(command), > command[0], command[1], command[2], command[3]); > } > @@ -926,6 +926,8 @@ static int vgic_its_handle_cmds(struct domain *d, > struct virt_its *its) > if ( ret ) > return ret; > > + dump_its_command(command); > + > switch ( its_cmd_get_command(command) ) > { > case GITS_CMD_CLEAR: > > > -- > Julien Grall [0] https://www.mail-archive.com/u-boot@lists.denx.de/msg379708.html [1] diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 9558bad96a..d175ba52b0 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -87,6 +87,10 @@ static int its_send_command(struct host_its *hw_its, const void *its_cmd) /* No ITS commands from an interrupt handler (at the moment). */ ASSERT(!in_irq()); + printk(XENLOG_WARNING "pITS cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", + (((uint64_t *) its_cmd)[0] >> 0) & GENMASK(8 - 1, 0), + ((uint64_t *) its_cmd)[0], ((uint64_t *) its_cmd)[1], ((uint64_t *) its_cmd)[2], ((uint64_t *) its_cmd)[3]); + spin_lock(&hw_its->cmd_lock); do { diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index 78b9521b21..2c3b0fc9e5 100644 --- a/xen/arch/arm/gic-v3-lpi.c +++ b/xen/arch/arm/gic-v3-lpi.c @@ -181,8 +181,10 @@ void gicv3_do_LPI(unsigned int lpi) /* Find out if a guest mapped something to this physical LPI. */ hlpip = gic_get_host_lpi(lpi); - if ( !hlpip ) + if ( !hlpip ) { + printk("%s: Received LPI %u but it is not mapped?\n", __func__, lpi); goto out; + } hlpi.data = read_u64_atomic(&hlpip->data); @@ -221,6 +223,9 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id, { union host_lpi *hlpip, hlpi; + printk("%s: host_lpi %u domain %d virt_lpi %u\n", + __func__, host_lpi, domain_id, virt_lpi); + ASSERT(host_lpi >= LPI_OFFSET); host_lpi -= LPI_OFFSET; diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c index 6e153c698d..dd5081ef80 100644 --- a/xen/arch/arm/vgic-v3-its.c +++ b/xen/arch/arm/vgic-v3-its.c @@ -897,7 +897,7 @@ out_unlock: static void dump_its_command(uint64_t *command) { - gdprintk(XENLOG_WARNING, " cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", + gdprintk(XENLOG_WARNING, "vITS cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", its_cmd_get_command(command), command[0], command[1], command[2], command[3]); } @@ -926,6 +926,8 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its) if ( ret ) return ret; + dump_its_command(command); + switch ( its_cmd_get_command(command) ) { case GITS_CMD_CLEAR: [-- Attachment #2: boot-xendebug.log --] [-- Type: application/octet-stream, Size: 26771 bytes --] Xen 4.13.2 (XEN) Xen version 4.13.2 (@) (aarch64-poky-linux-gcc (GCC) 8.3.0) debug=y Sun Nov 22 21:56:06 UTC 2020 (XEN) Latest ChangeSet: Fri Oct 30 12:24:39 2020 +0100 git:0060ac29bc-dirty (XEN) build-id: 1f6c38235b06eb99a3e73b3846f910a4df9dc028 (XEN) Processor: 410fd083: "ARM Limited", variant: 0x0, part 0xd08, rev 0x3 (XEN) 64-bit Execution: (XEN) Processor Features: 0000000001002222 0000000000000000 (XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32 (XEN) Extensions: FloatingPoint AdvancedSIMD GICv3-SysReg (XEN) Debug Features: 0000000010305106 0000000000000000 (XEN) Auxiliary Features: 0000000000000000 0000000000000000 (XEN) Memory Model Features: 0000000000001124 0000000000000000 (XEN) ISA Features: 0000000000010000 0000000000000000 (XEN) 32-bit Execution: (XEN) Processor Features: 00000131:10011011 (XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle (XEN) Extensions: GenericTimer Security (XEN) Debug Features: 03010066 (XEN) Auxiliary Features: 00000000 (XEN) Memory Model Features: 10201105 40000000 01260000 02102211 (XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00010001 (XEN) Using SMC Calling Convention v1.0 (XEN) Using PSCI v0.2 (XEN) SMP: Allowing 2 CPUs (XEN) enabled workaround for: ARM erratum 1319537 (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 25000 KHz (XEN) GICv3 initialization: (XEN) gic_dist_addr=0x00000006000000 (XEN) gic_maintenance_irq=25 (XEN) gic_rdist_stride=0 (XEN) gic_rdist_regions=1 (XEN) redistributor regions: (XEN) - region 0: 0x00000006040000 - 0x00000006080000 (XEN) GICv3: using at most 57344 LPIs on the host. (XEN) GICv3: 288 lines, (IID 0001143b). (XEN) GICv3: Found ITS @0x6020000 (XEN) using non-cacheable ITS command queue (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000 (XEN) pITS cmd 0x09: 0000000000000009 0000000000000000 8000000000000000 0000000000000000 (XEN) pITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) XSM Framework v1.0.0 initialized (XEN) Initialising XSM SILO mode (XEN) Using scheduler: null Scheduler (null) (XEN) Initializing null scheduler (XEN) WARNING: This is experimental software in development. (XEN) Use at your own risk. (XEN) Allocated console ring of 16 KiB. (XEN) CPU0: Guest atomics will try 13 times before pausing the domain (XEN) Bringing up CPU1 (XEN) GICv3: CPU1: Found redistributor in region 0 @000000004003c000 (XEN) pITS cmd 0x09: 0000000000000009 0000000000000000 8000000000010001 0000000000000000 (XEN) pITS cmd 0x05: 0000000000000005 0000000000000000 0000000000010000 0000000000000000 (XEN) CPU1: Guest atomics will try 12 times before pausing the domain (XEN) CPU 1 booted. (XEN) Brought up 2 CPUs (XEN) I/O virtualisation disabled (XEN) P2M: 44-bit IPA with 44-bit PA and 8-bit VMID (XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594 (XEN) alternatives: Patching with alt table 00000000002d40e8 -> 00000000002d48bc (XEN) *** LOADING DOMAIN 0 *** (XEN) Loading d0 kernel from boot module @ 0000000081200000 (XEN) Allocating 1:1 mappings totalling 512MB for dom0: (XEN) BANK[0] 0x000000c0000000-0x000000e0000000 (512MB) (XEN) Grant table range: 0x00000081000000-0x00000081040000 (XEN) Allocating PPI 16 for event channel interrupt (XEN) Loading zImage from 0000000081200000 to 00000000c0080000-00000000c1597008 (XEN) Loading d0 DTB to 0x00000000c8000000-0x00000000c8006cb6 (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *************************************************** (XEN) No support for ARM_SMCCC_ARCH_WORKAROUND_1. (XEN) Please update your firmware. (XEN) *************************************************** (XEN) No support for ARM_SMCCC_ARCH_WORKAROUND_1. (XEN) Please update your firmware. (XEN) *************************************************** (XEN) 3... 2... 1... (XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input) (XEN) sched_null.c:344: 0 <-- d0v0 (XEN) Freed 344kB init memory. (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER4 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER8 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER12 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER16 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER20 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER24 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER28 (XEN) d0v0: vGICD: unhandled word write 0x000000ffffffff to ICACTIVER32 (XEN) d0v0: vGICR: SGI: unhandled word write 0x000000ffffffff to ICACTIVER0 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x09: 0000000000000009 0000000000000000 8000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0d: 000000000000000d 0000000000000000 0000000000000000 0000000000000000 [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083] [ 0.000000] Linux version 4.19.68+g4f0ecdbd4f33 (oe-user@oe-host) (gcc version 8.3.0 (GCC)) #1 SMP PREEMPT Thu Oct 29 22:05:56 UTC 2020 [ 0.000000] Machine model: Kontron SMARC-sAL28 [ 0.000000] Xen 4.13 support found [ 0.000000] cma: Reserved 32 MiB at 0x00000000de000000 [ 0.000000] NUMA: No NUMA configuration found [ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000dfffffff] [ 0.000000] NUMA: NODE_DATA [mem 0xddfd5b40-0xddfd72ff] [ 0.000000] Zone ranges: [ 0.000000] DMA32 [mem 0x00000000c0000000-0x00000000dfffffff] [ 0.000000] Normal empty [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00000000c0000000-0x00000000dfffffff] [ 0.000000] Initmem setup node 0 [mem 0x00000000c0000000-0x00000000dfffffff] [ 0.000000] psci: probing for conduit method from DT. [ 0.000000] psci: PSCIv1.1 detected in firmware. [ 0.000000] psci: Using standard PSCI v0.2 function IDs [ 0.000000] psci: Trusted OS migration not required [ 0.000000] psci: SMC Calling Convention v1.1 [ 0.000000] random: get_random_bytes called from start_kernel+0x94/0x3ec with crng_init=0 [ 0.000000] percpu: Embedded 25 pages/cpu s62296 r8192 d31912 u102400 [ 0.000000] Detected PIPT I-cache on CPU0 [ 0.000000] CPU features: enabling workaround for EL2 vector hardening [ 0.000000] CPU features: enabling workaround for Speculative Store Bypass Disable [ 0.000000] CPU features: detected: Kernel page table isolation (KPTI) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 129024 [ 0.000000] Policy zone: DMA32 [ 0.000000] Kernel command line: root=/dev/mmcblk1p2 console=hvc0 earlycon=xen earlyprintk=xen clk_ignore_unused rootwait [ 0.000000] Memory: 452116K/524288K available (12862K kernel code, 1628K rwdata, 5708K rodata, 1344K init, 899K bss, 39404K reserved, 32768K cma-reserved) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] rcu: Preemptible hierarchical RCU implementation. [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1. [ 0.000000] Tasks RCU enabled. [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 [ 0.000000] GICv3: Distributor has no Range Selector support [ 0.000000] GICv3: no VLPI support, no direct LPI support [ 0.000000] ITS [mem 0x06020000-0x0603ffff] [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 (flat, esz 8, psz 64K, shr 1) [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt Collections @dc820000 (flat, esz 2, psz 64K, shr 1) [ 0.000000] GIC: using LPI property table @0x00000000dc830000 [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000006040000 [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000 [ 0.000000] arch_timer: cp15 timer(s) running at 25.00MHz (virt). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x5c40939b5, max_idle_ns: 440795202646 ns [ 0.000003] sched_clock: 56 bits at 25MHz, resolution 40ns, wraps every 4398046511100ns [ 0.000230] Console: colour dummy device 80x25 [ 0.000812] console [hvc0] enabled [ 0.000857] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=100000) [ 0.000882] pid_max: default: 32768 minimum: 301 [ 0.000927] Security Framework initialized [ 0.001178] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes) [ 0.001293] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes) [ 0.001323] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes) [ 0.001342] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes) [ 0.024034] ASID allocator initialised with 32768 entries [ 0.024549] xen:grant_table: Grant tables using version 1 layout [ 0.024575] Grant table initialized [ 0.024606] xen:events: Using FIFO-based ABI [ 0.024646] Xen: initializing cpu0 [ 0.032041] rcu: Hierarchical SRCU implementation. [ 0.040081] Platform MSI: gic-its domain created [ 0.040140] PCI/MSI: /interrupt-controller/gic-its domain created [ 0.040187] fsl-mc MSI: /interrupt-controller/gic-its domain created [ 0.048072] smp: Bringing up secondary CPUs ... [ 0.048086] smp: Brought up 1 node, 1 CPU [ 0.048097] SMP: Total of 1 processors activated. [ 0.048113] CPU features: detected: GIC system register CPU interface [ 0.048129] CPU features: detected: 32-bit EL0 Support [ 0.051116] CPU: All CPU(s) started at EL1 [ 0.051133] alternatives: patching kernel code [ 0.051728] devtmpfs: initialized [ 0.056337] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns [ 0.056374] futex hash table entries: 256 (order: 2, 16384 bytes) [ 0.056912] pinctrl core: initialized pinctrl subsystem [ 0.057704] NET: Registered protocol family 16 [ 0.057908] audit: initializing netlink subsys (disabled) [ 0.058469] audit: type=2000 audit(0.056:1): state=initialized audit_enabled=0 res=1 [ 0.058925] vdso: 2 pages (1 code @ (____ptrval____), 1 data @ (____ptrval____)) [ 0.058947] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers. [ 0.062295] DMA: preallocated 256 KiB pool for atomic allocations [ 0.062394] xen:swiotlb_xen: Warning: only able to allocate 4 MB for software IO TLB [ 0.063142] software IO TLB: mapped [mem 0xdbc00000-0xdc000000] (4MB) [ 0.064269] Serial: AMBA PL011 UART driver [ 0.104947] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages [ 0.108728] cryptd: max_cpu_qlen set to 1000 [ 0.117877] xen:balloon: Initialising balloon driver [ 0.119767] vgaarb: loaded [ 0.120078] SCSI subsystem initialized [ 0.121057] usbcore: registered new interface driver usbfs [ 0.121130] usbcore: registered new interface driver hub [ 0.121204] usbcore: registered new device driver usb [ 0.121736] imx-i2c 2000000.i2c: scl-gpios not found [ 0.122155] of_dma_request_slave_channel: dma-names property of node '/soc/i2c@2000000' missing or empty [ 0.122179] i2c i2c-0: IMX I2C adapter registered [ 0.122328] imx-i2c 2030000.i2c: scl-gpios not found [ 0.122452] of_dma_request_slave_channel: dma-names property of node '/soc/i2c@2030000' missing or empty [ 0.122475] i2c i2c-1: IMX I2C adapter registered [ 0.122616] imx-i2c 2040000.i2c: scl-gpios not found [ 0.122802] of_dma_request_slave_channel: dma-names property of node '/soc/i2c@2040000' missing or empty [ 0.122823] i2c i2c-2: IMX I2C adapter registered [ 0.123273] pps_core: LinuxPPS API ver. 1 registered [ 0.123288] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> [ 0.123346] PTP clock support registered [ 0.123546] EDAC MC: Ver: 3.0.0 [ 0.126576] No BMan portals available! [ 0.126954] QMan: Allocated lookup table at (____ptrval____), entry count 65537 [ 0.127075] No QMan portals available! [ 0.127223] No USDPAA memory, no 'fsl,usdpaa-mem' in device-tree [ 0.128688] Advanced Linux Sound Architecture Driver Initialized. [ 0.129396] clocksource: Switched to clocksource arch_sys_counter [ 0.129484] VFS: Disk quotas dquot_6.6.0 [ 0.129525] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) [ 0.138006] NET: Registered protocol family 2 [ 0.138292] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes) [ 0.138406] TCP established hash table entries: 4096 (order: 3, 32768 bytes) [ 0.138440] TCP bind hash table entries: 4096 (order: 4, 65536 bytes) [ 0.138498] TCP: Hash tables configured (established 4096 bind 4096) [ 0.138567] UDP hash table entries: 256 (order: 1, 8192 bytes) [ 0.138589] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes) [ 0.138654] NET: Registered protocol family 1 [ 0.148017] RPC: Registered named UNIX socket transport module. [ 0.148040] RPC: Registered udp transport module. [ 0.148052] RPC: Registered tcp transport module. [ 0.148064] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.148406] kvm [1]: HYP mode not available [ 0.149251] Initialise system trusted keyrings [ 0.149762] workingset: timestamp_bits=44 max_order=17 bucket_order=0 [ 0.152943] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 0.156914] NFS: Registering the id_resolver key type [ 0.156948] Key type id_resolver registered [ 0.156959] Key type id_legacy registered [ 0.156973] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 0.157009] jffs2: version 2.2. (NAND) © 2001-2006 Red Hat, Inc. [ 0.157486] 9p: Installing v9fs 9p2000 file system support [ 0.161737] Key type asymmetric registered [ 0.161758] Asymmetric key parser 'x509' registered [ 0.161808] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 244) [ 0.161826] io scheduler noop registered [ 0.161837] io scheduler deadline registered [ 0.161931] io scheduler cfq registered (default) [ 0.161944] io scheduler mq-deadline registered [ 0.161957] io scheduler kyber registered [ 0.164994] pci-host-generic 1f0000000.pcie: host bridge /soc/pcie@1f0000000 ranges: [ 0.165029] pci-host-generic 1f0000000.pcie: MEM 0x1f8000000..0x1f815ffff -> 0x00000000 [ 0.165054] pci-host-generic 1f0000000.pcie: MEM 0x1f8160000..0x1f81cffff -> 0x00000000 [ 0.165077] pci-host-generic 1f0000000.pcie: MEM 0x1f81d0000..0x1f81effff -> 0x00000000 [ 0.165100] pci-host-generic 1f0000000.pcie: MEM 0x1f81f0000..0x1f820ffff -> 0x00000000 [ 0.165123] pci-host-generic 1f0000000.pcie: MEM 0x1f8210000..0x1f822ffff -> 0x00000000 [ 0.165146] pci-host-generic 1f0000000.pcie: MEM 0x1f8230000..0x1f824ffff -> 0x00000000 [ 0.165167] pci-host-generic 1f0000000.pcie: MEM 0x1fc000000..0x1fc3fffff -> 0x00000000 [ 0.165218] pci-host-generic 1f0000000.pcie: ECAM at [mem 0x1f0000000-0x1f00fffff] for [bus 00] [ 0.165305] pci-host-generic 1f0000000.pcie: PCI host bridge to bus 0000:00 [ 0.165322] pci_bus 0000:00: root bus resource [bus 00] [ 0.165337] pci_bus 0000:00: root bus resource [mem 0x1f8000000-0x1f815ffff] (bus address [0x00000000-0x0015ffff]) [ 0.165368] pci_bus 0000:00: root bus resource [mem 0x1f8160000-0x1f81cffff pref] (bus address [0x00000000-0x0006ffff]) [ 0.168237] pci_bus 0000:00: root bus resource [mem 0x1f81d0000-0x1f81effff] (bus address [0x00000000-0x0001ffff]) [ 0.168270] pci_bus 0000:00: root bus resource [mem 0x1f81f0000-0x1f820ffff pref] (bus address [0x00000000-0x0001ffff]) [ 0.168293] pci_bus 0000:00: root bus resource [mem 0x1f8210000-0x1f822ffff] (bus address [0x00000000-0x0001ffff]) [ 0.168316] pci_bus 0000:00: root bus resource [mem 0x1f8230000-0x1f824ffff pref] (bus address [0x00000000-0x0001ffff]) [ 0.168339] pci_bus 0000:00: root bus resource [mem 0x1fc000000-0x1fc3fffff] (bus address [0x00000000-0x003fffff]) [ 0.168463] pci 0000:00:00.0: VF(n) BAR0 space: [mem 0x1f81d0000-0x1f81effff 64bit] (contains BAR0 for 2 VFs) [ 0.168486] pci 0000:00:00.0: VF(n) BAR2 space: [mem 0x1f81f0000-0x1f820ffff 64bit pref] (contains BAR2 for 2 VFs) (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=25: not implemented (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.168639] pci 0000:00:00.0: Failed to add - passthrough or MSI/MSI-X might fail! [ 0.168805] pci 0000:00:00.1: VF(n) BAR0 space: [mem 0x1f8210000-0x1f822ffff 64bit] (contains BAR0 for 2 VFs) [ 0.168829] pci 0000:00:00.1: VF(n) BAR2 space: [mem 0x1f8230000-0x1f824ffff 64bit pref] (contains BAR2 for 2 VFs) (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.168931] pci 0000:00:00.1: Failed to add - passthrough or MSI/MSI-X might fail! (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.169138] pci 0000:00:00.2: Failed to add - passthrough or MSI/MSI-X might fail! (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.169332] pci 0000:00:00.3: Failed to add - passthrough or MSI/MSI-X might fail! (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.169613] pci 0000:00:00.4: Failed to add - passthrough or MSI/MSI-X might fail! (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.172781] pci 0000:00:00.5: Failed to add - passthrough or MSI/MSI-X might fail! (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.172995] pci 0000:00:00.6: Failed to add - passthrough or MSI/MSI-X might fail! [ 0.173964] OF: /soc/pcie@1f0000000: Invalid msi-map translation - no match for rid 0xf8 on (null) (XEN) physdev.c:16:d0v0 PHYSDEVOP cmd=15: not implemented [ 0.174044] pci 0000:00:1f.0: Failed to add - passthrough or MSI/MSI-X might fail! [ 0.174538] layerscape-pcie 3400000.pcie: host bridge /soc/pcie@3400000 ranges: [ 0.174570] layerscape-pcie 3400000.pcie: IO 0x8000010000..0x800001ffff -> 0x00000000 [ 0.174594] layerscape-pcie 3400000.pcie: MEM 0x8040000000..0x807fffffff -> 0x40000000 [ 0.174837] layerscape-pcie 3400000.pcie: PCI host bridge to bus 0001:00 [ 0.174856] pci_bus 0001:00: root bus resource [bus 00-ff] [ 0.174870] pci_bus 0001:00: root bus resource [io 0x0000-0xffff] [ 0.174887] pci_bus 0001:00: root bus resource [mem 0x8040000000-0x807fffffff] (bus address [0x40000000-0x7fffffff]) [ 0.175070] pci 0001:00:00.0: Failed to add - passthrough or MSI/MSI-X might fail! [ 0.176593] pci 0001:00:00.0: BAR 6: assigned [mem 0x8040000000-0x80400007ff pref] [ 0.176613] pci 0001:00:00.0: PCI bridge to [bus 01] [ 0.177125] pcieport 0001:00:00.0: Signaling PME with IRQ 16 [ 0.177281] pcieport 0001:00:00.0: AER enabled with IRQ 17 [ 0.177874] layerscape-pcie 3500000.pcie: host bridge /soc/pcie@3500000 ranges: [ 0.177908] layerscape-pcie 3500000.pcie: IO 0x8800010000..0x880001ffff -> 0x00000000 [ 0.177939] layerscape-pcie 3500000.pcie: MEM 0x8840000000..0x887fffffff -> 0x40000000 [ 0.178174] layerscape-pcie 3500000.pcie: PCI host bridge to bus 0002:00 [ 0.178193] pci_bus 0002:00: root bus resource [bus 00-ff] [ 0.178208] pci_bus 0002:00: root bus resource [io 0x10000-0x1ffff] (bus address [0x0000-0xffff]) [ 0.178229] pci_bus 0002:00: root bus resource [mem 0x8840000000-0x887fffffff] (bus address [0x40000000-0x7fffffff]) [ 0.178416] pci 0002:00:00.0: Failed to add - passthrough or MSI/MSI-X might fail! [ 0.179940] pci 0002:00:00.0: BAR 6: assigned [mem 0x8840000000-0x88400007ff pref] [ 0.179961] pci 0002:00:00.0: PCI bridge to [bus 01] [ 0.180274] pcieport 0002:00:00.0: Signaling PME with IRQ 18 [ 0.180458] pcieport 0002:00:00.0: AER enabled with IRQ 19 [ 0.186395] Freescale LS2 console driver [ 0.186531] fsl-ls2-console: device fsl_mc_console registered [ 0.186618] fsl-ls2-console: device fsl_aiop_console registered [ 0.189325] xen:xen_evtchn: Event-channel device installed [ 0.196223] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.198204] SuperH (H)SCI(F) driver initialized [ 0.198499] msm_serial: driver initialized [ 0.198726] 2270000.serial: ttyLP0 at MMIO 0x2270000 (irq = 10, base_baud = 12500000) is a FSL_LPUART [ 0.203664] cacheinfo: Unable to detect cache hierarchy for CPU 0 [ 0.209143] loop: module loaded [ 0.209237] Invalid max_queues (4), will use default max: 1. [ 0.211203] at24 0-0050: 4096 byte 24c32 EEPROM, writable, 32 bytes/write [ 0.212609] at24 2-0050: 4096 byte 24c32 EEPROM, writable, 32 bytes/write [ 0.216406] sl28cpld 0-004a: registered IRQ 27 [ 0.216426] sl28cpld 0-004a: successfully probed. CPLD version 18. [ 0.239712] m25p80 spi0.0: w25q32jw (4096 Kbytes) [ 0.240316] 10 fixed-partitions partitions found on MTD device 20c0000.spi [ 0.240335] Creating 10 MTD partitions on "20c0000.spi": [ 0.240351] 0x000000000000-0x000000010000 : "rcw" [ 0.240834] 0x000000010000-0x000000100000 : "failsafe bootloader" [ 0.241282] 0x000000100000-0x000000140000 : "failsafe DP firmware" [ 0.241792] 0x000000140000-0x0000001e0000 : "failsafe trusted firmware" [ 0.242252] 0x0000001e0000-0x000000200000 : "reserved" [ 0.242708] 0x000000200000-0x000000210000 : "configuration store" [ 0.243159] 0x000000210000-0x000000300000 : "bootloader" [ 0.243613] 0x000000300000-0x000000340000 : "DP firmware" [ 0.244069] 0x000000340000-0x0000003e0000 : "trusted firmware" [ 0.244511] 0x0000003e0000-0x000000400000 : "bootloader environment" [ 0.250047] libphy: Fixed MDIO Bus: probed [ 0.252046] tun: Universal TUN/TAP device driver, 1.6 [ 0.256135] thunder_xcv, ver 1.0 [ 0.256220] thunder_bgx, ver 1.0 [ 0.256286] nicpf, ver 1.0 [ 0.256506] Freescale FM module, FMD API version 21.1.0 [ 0.256626] Freescale FM Ports module [ 0.256637] fsl_mac: fsl_mac: FSL FMan MAC API based driver [ 0.256737] fsl_dpa: FSL DPAA Ethernet driver [ 0.256825] fsl_advanced: FSL DPAA Advanced drivers: [ 0.256838] fsl_proxy: FSL DPAA Proxy initialization driver [ 0.256922] fsl_oh: FSL FMan Offline Parsing port driver [ 0.361406] fsl_enetc 0000:00:00.0: enabling device (0400 -> 0402) [ 0.361469] fsl_enetc 0000:00:00.0: no MAC address specified for SI1, using 42:14:fd:42:55:1d [ 0.361491] fsl_enetc 0000:00:00.0: no MAC address specified for SI2, using 1e:22:15:9a:c9:e9 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x08: 0000001700000008 0000000000000000 80000000db471200 0000000000000000 (XEN) pITS cmd 0x08: 0000001700000008 0000000000000004 80000020f9de4600 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200000000000 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000000 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200100000001 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000001 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200200000002 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000002 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200300000003 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000003 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200400000004 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000004 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200500000005 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000005 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200600000006 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000006 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000200700000007 0000000000000000 00000000000000[ 1.656243] fsl_enetc 0000:00:00.1 gbe1: renamed from eth1 [ 1.721814] fsl_enetc 0000:00:00.0 gbe0: renamed from eth0 [ 3.011356] urandom_read: 3 callbacks suppressed [ 3.011360] random: udevd: uninitialized urandom read (16 bytes read) [ 3.011490] random: udevd: uninitialized urandom read (16 bytes read) [ 3.011542] random: udevd: uninitialized urandom read (16 bytes read) [ 3.012782] udevd[1812]: specified group 'kvm' unknown [ 4.010698] FAT-fs (mmcblk1p1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck. [ 6.043586] oak_pci: loading out-of-tree module taints kernel. [ 6.049496] Marvell PCIe Switch Driver - (oak) version 0.0.0 [ 6.049523] Copyright (c) Marvell - 2018 [ 6.073648] EXT4-fs (mmcblk1p2): re-mounted. Opts: (null) [ 6.516987] random: dd: uninitialized urandom read (512 bytes read) ALSA: Restoring mixer settings... /usr/sbin/alsactl: load_state:1735: No soundcards found... INIT: Entering runlevel: 5 Configuring network interfaces... done. Starting system message bus: dbus. Mounting cgroups...Done Starting Dropbear SSH server: [ 6.764603] NET: Registered protocol family 10 [ 6.769738] Segment Routing with IPv6 dropbear. Starting rpcbind daemon...done. Starting bluetooth: bluetoothd. Starting dockerd: [ 6.922993] Bluetooth: Core ver 2.22 [ 6.923057] NET: Registered protocol family 31 [ 6.923070] Bluetooth: HCI device and connection manager initialized [ 6.923090] Bluetooth: HCI socket layer initialized [ 6.923113] Bluetooth: L2CAP socket layer initialized [ 6.923137] Bluetooth: SCO socket layer initialized Starting ntpd: done Starting syslogd/klogd: done Starting internet superserver: xinetd. * Starting Avahi mDNS/DNS-SD Daemon: avahi-daemon [ ok ] Starting Telephony daemon Starting /usr/sbin/xenstored... Setting domain 0 name, domid and JSON config... Done setting up Dom0 Starting xenconsoled... Starting QEMU as disk backend for dom0 /etc/rc5.d/S80xencommons: line 73: /usr/bin/qemu-system-i386: No such file or directory Starting domain watchdog daemon: xenwatchdogd startup [done] Poky (Yocto Project Reference Distro) 2.7.1 kontron-sal28 hvc0 [-- Attachment #3: xen-debug-output.txt --] [-- Type: text/plain, Size: 16003 bytes --] ) pITS cmd 0x0c: 000000170000000c 0000000000000013 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201400000014 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000014 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201500000015 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000015 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201600000016 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000016 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201700000017 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000017 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201800000018 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000018 0000000000000000 0000000000000000 000000000 (XEN) pITS cmd 0x0c: 000000170000000c 0000000000000019 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201a0000001a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 000000000000001a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201b0000001b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 000000000000001b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201c0000001c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 000000000000001c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201d0000001d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 000000000000001d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201e0000001e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 000000000000001e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000170000000a 0000201f0000001f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000170000000c 000000000000001f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0a: 000000170000000a 0000200000000000 0000000000000000 0000000000000000 (XEN) gicv3_lpi_update_host_entry: host_lpi 8192 domain 0 virt_lpi 8192 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0a: 000000170000000a 0000200100000001 0000000000000000 0000000000000000 (XEN) gicv3_lpi_update_host_entry: host_lpi 8193 domain 0 virt_lpi 8193 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x08: 0000001800000008 0000000000000000 80000000db471a00 0000000000000000 (XEN) pITS cmd 0x08: 0000001800000008 0000000000000004 80000020f9de2700 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202000000000 0000000000000000 0000000000000000 000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202100000001 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000001 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202200000002 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000002 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202300000003 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000003 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202400000004 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000004 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202500000005 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000005 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202600000006 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000006 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202700000007 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000007 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202800000008 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000008 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202900000009 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000009 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202a0000000a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000000a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202b0000000b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000000b 0000000000000000 0000000000000000 000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000000c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202d0000000d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000000d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202e0000000e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000000e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000202f0000000f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000000f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203000000010 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000010 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203100000011 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000011 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203200000012 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000012 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203300000013 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000013 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203400000014 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000014 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203500000015 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000015 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203600000016 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000016 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203700000017 0000000000000000 0000000000000000 000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203800000018 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000018 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203900000019 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 0000000000000019 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203a0000001a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000001a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203b0000001b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000001b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203c0000001c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000001c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203d0000001d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000001d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203e0000001e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000001e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 000000180000000a 0000203f0000001f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 000000180000000c 000000000000001f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0a: 000000180000000a 0000200200000000 0000000000000000 0000000000000000 (XEN) gicv3_lpi_update_host_entry: host_lpi 8224 domain 0 virt_lpi 8194 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0a: 000000180000000a 0000200300000001 0000000000000000 0000000000000000 (XEN) gicv3_lpi_update_host_entry: host_lpi 8225 domain 0 virt_lpi 8195 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 0000000dbb69200 0000000000000000 (XEN) pITS cmd 0x08: 0000001b00000008 0000000000000004 80000020f9de1100 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204000000000 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000000 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204100000001 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000001 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204200000002 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000002 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204300000003 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000003 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204400000004 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000004 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204500000005 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000005 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204600000006 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000006 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204700000007 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000007 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204800000008 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000008 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204900000009 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000009 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204a0000000a 0000000000000000 0000000000000000 000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204b0000000b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000000b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204c0000000c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000000c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204d0000000d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000000d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204e0000000e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000000e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000204f0000000f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000000f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205000000010 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000010 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205100000011 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000011 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205200000012 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000012 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205300000013 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000013 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205400000014 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000014 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205500000015 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000015 0000000000000000 0000000000000000 000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000016 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205700000017 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000017 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205800000018 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000018 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205900000019 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 0000000000000019 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205a0000001a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000001a 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205b0000001b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000001b 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205c0000001c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000001c 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205d0000001d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000001d 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205e0000001e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000001e 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0a: 0000001b0000000a 0000205f0000001f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x0c: 0000001b0000000c 000000000000001f 0000000000000000 0000000000000000 (XEN) pITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0a: 0000001b0000000a 0000200400000000 0000000000000000 0000000000000000 (XEN) gicv3_lpi_update_host_entry: host_lpi 8256 domain 0 virt_lpi 8196 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 0000001b0000000c 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c 0000000000000001 0000000000000000 0000000000000000 (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2020-11-22 22:55 ` AW: " Leo Krueger @ 2020-11-23 11:41 ` Rahul Singh 2020-11-23 18:41 ` Julien Grall 2020-11-23 18:27 ` AW: AW: AW: AW: " Julien Grall 1 sibling, 1 reply; 16+ messages in thread From: Rahul Singh @ 2020-11-23 11:41 UTC (permalink / raw) To: Leo Krueger Cc: Julien Grall, Stefano Stabellini, Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand Marquis Hello , > On 22 Nov 2020, at 10:55 pm, Leo Krueger <leo.krueger@zal.aero> wrote: > > Hi Julien, > > finally I could try out what you suggested, please find my answers inline. > >> -----Ursprüngliche Nachricht----- >> Von: Julien Grall <julien@xen.org> >> Gesendet: Mittwoch, 18. November 2020 13:24 >> An: Stefano Stabellini <stefano.stabellini@xilinx.com>; Leo Krueger >> <leo.krueger@zal.aero> >> Cc: Peng Fan <peng.fan@nxp.com>; brucea@xilinx.com; Cornelia Bruelhart >> <cornelia.bruelhart@zal.aero>; oleksandr_andrushchenko@epam.com; xen- >> devel@lists.xenproject.org; Bertrand.Marquis@arm.com >> Betreff: Re: AW: AW: AW: AW: Xen data from meta-virtualization layer >> >> Hi, >> >> On 17/11/2020 23:53, Stefano Stabellini wrote: >>> Adding Bertrand, Oleksandr, Julien, and others -- they have a more >>> recent experience with GICv3 ITS than me and might be able to help. >>> I am attaching the device tree Leo sent a few days ago for reference. >>> >>> >>> Typically when you can set the ethernet link up and no packets are >>> exchanged it is because of a missing interrupt. In this case a missing >>> MSI. >>> >>> Bertrand, I believe you tried the GIC ITS driver with PCI devices >>> recently. It is expected to work correctly with MSIs in Dom0, right? >> >> OSSTest has some hardware (e.g. Thunder-X) where ITS is required to boot >> Dom0. I haven't seen any failure on recent Xen. We are testing 4.11 and >> onwards on Thunder-X. >> >> However, it may be possible that some more work is necessary for other >> hardware (e.g. workaround, missing code...). See more below. >> >>> >>> >>> >>> On Tue, 17 Nov 2020, Leo Krueger wrote: >>>> Hi, >>>> >>>> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it >>>> before...) but then had to add the following node to my device tree >>>> >>>> gic_lpi_base: syscon@0x80000000 { >>>> compatible = "gic-lpi-base"; >> >> I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, could >> you clarify which flavor/version of Linux you are using? > > It is Linux 4.19 from Yocto (Warror release). XEN 4.13.2. > While searching around the Internet for any solution, I came across [0] which contained the gic-lpi-base node. > So I just tried adding it (quite desperate I know) and voila, it at least brought me one step further (XEN exposing the ITS)... > >> >>>> reg = <0x0 0x80000000 0x0 0x100000>; >>>> max-gic-redistributors = <2>; >>>> }; >>>> >>>> to somehow change something in regard to the ITS and MSI/MSI-X >>>> >>>> (XEN) GICv3 initialization: >>>> (XEN) gic_dist_addr=0x00000006000000 >>>> (XEN) gic_maintenance_irq=25 >>>> (XEN) gic_rdist_stride=0 >>>> (XEN) gic_rdist_regions=1 >>>> (XEN) redistributor regions: >>>> (XEN) - region 0: 0x00000006040000 - 0x00000006080000 >>>> (XEN) GICv3: using at most 57344 LPIs on the host. >>>> (XEN) GICv3: 288 lines, (IID 0001143b). >>>> (XEN) GICv3: Found ITS @0x6020000 >>>> (XEN) using non-cacheable ITS command queue >>>> (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000 >>>> >>>> [ 0.000000] GICv3: Distributor has no Range Selector support >>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support >>>> [ 0.000000] ITS [mem 0x06020000-0x0603ffff] >>>> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices >> @dc880000 (flat, esz 8, psz 64K, shr 1) >>>> [ 0.000000] ITS@0x0000000006020000: allocated 32768 Interrupt >> Collections @dc820000 (flat, esz 2, psz 64K, shr 1) >>>> [ 0.000000] GIC: using LPI property table @0x00000000dc830000 >>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region >> 0:0x0000000006040000 >>>> [ 0.000000] CPU0: using LPI pending table @0x00000000dc840000 >>>> ... >>>> [ 0.040080] Platform MSI: gic-its domain created >>>> [ 0.040136] PCI/MSI: /interrupt-controller/gic-its domain created >>>> [ 0.040181] fsl-mc MSI: /interrupt-controller/gic-its domain created >>>> >>>> >>>> Still I am ending up with the " Failed to add - passthrough or MSI/MSI-X >> might fail!" log messages for some PCI devices, but at least the on-board >> ethernet ports (fsl_enetc ) are initialized. >>>> I can set the link up and a link is successfully established. >> >> This message is normal. Xen on Arm is not yet aware of PCI devices and >> therefore the hypercalls to add PCI devices will return -EOPNOTSUPP. >> >> However, this is not an issue because the virtual ITS implementation will >> allow dom0 to configure any devices. >> >>>> >>>> But (!) I cannot receive or transmit anything (no error message...) and >> there seem to be no interrupts: >>>> >>>> 29: 0 ITS-MSI 1 Edge gbe0-rxtx0 >>>> 32: 0 ITS-MSI 8192 Edge ptp_qoriq >>>> >>>> (from /proc/interrupts). >>>> >>>> Any idea on this one? I keep digging and checking whether my device tree >> needs some additional fixes. >> >> Can you apply patch [1] and provide the logs? This will dump more >> information about the command received by the vITS and the one send to >> the host ITS. > > For XEN 4.13.2 I had to adapt your patch slightly [1], see below (yes I know, quite ugly in parts). > Find attached the boot log and an output of "xl dmesg" which is truncated due to the large amount of messages. > > When enabling the network interface (gbe0), the following output is visible: > > root@kontron-sal28:~# ip link set up dev gbe0 > (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c 0000000000000001 0000000000000000 0000000000000000 > (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 > [ 34.034598] Atheros 8031 ethernet 0000:00:00.3:05: attached PHY driver [Atheros 8031 ethernet] (mii_bus:phy_addr=0000:00:00.3:05, irq=POLL) > [ 34.041111] 8021q: adding VLAN 0 to HW filter on device gbe0 > [ 34.041209] IPv6: ADDRCONF(NETDEV_UP): gbe0: link is not ready > root@kontron-sal28:~# [ 35.041951] fsl_enetc 0000:00:00.0 gbe0: Link is Down > [ 38.114426] fsl_enetc 0000:00:00.0 gbe0: Link is Up - 1Gbps/Full - flow control off > [ 38.114508] IPv6: ADDRCONF(NETDEV_CHANGE): gbe0: link becomes ready > > Does that tell you anything? > I just checked the logs shared, what I found out that there’s is an error while booting to configure the MSI for the PCI device because of that there will be case that Device Id generate out-of-band is not mapped correctly to ITS device table created while initialising the MSI for the device. I might be wrong let someone else also comments on this. [ 0.173964] OF: /soc/pcie@1f0000000: Invalid msi-map translation - no match for rid 0xf8 on (null) Regards, Rahul >> >> Note that Xen will need to be build with CONFIG_DEBUG=y in order to get >> some of the messages. >> >> [...] >> >>>>>> [ 0.000000] GICv3: Distributor has no Range Selector support >>>>>> >>>>>> [ 0.000000] GICv3: no VLPI support, no direct LPI support >>>>>> >>>>>> [ 0.000000] GICv3: CPU0: found redistributor 0 region >>>>>> 0:0x0000000006040000 >>>>> >>>>> "no VLPI support" is very suspicious, it looks like Dom0 doesn't >>>>> find any ITS support. >> VLPI is a feature that was introduced in GICv4 to directly inject LPI in the >> guest. So this is normal to see this message when running on Xen because >> we are going to only expose a GICv3 to a domain until at least we support >> nested virt. >> >> However, you were right about that Xen didn't expose the ITS because the >> following lines were missing: >> >> [ 0.000000] ITS@0x0000000006020000: allocated 65536 Devices @dc880000 >> (flat, esz 8, psz 64K, shr 1) >> >> Cheers, > > Best regards, > Leo > >> >> [1] >> diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index >> 9558bad96ac3..8a0a02308e74 100644 >> --- a/xen/arch/arm/gic-v3-its.c >> +++ b/xen/arch/arm/gic-v3-its.c >> @@ -87,6 +87,10 @@ static int its_send_command(struct host_its *hw_its, >> const void *its_cmd) >> /* No ITS commands from an interrupt handler (at the moment). */ >> ASSERT(!in_irq()); >> >> + printk(XENLOG_WARNING, "pITS cmd 0x%02lx: %016lx %016lx %016lx >> %016lx\n", >> + its_cmd_get_command(command), >> + command[0], command[1], command[2], command[3]); >> + >> spin_lock(&hw_its->cmd_lock); >> >> do { >> diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c index >> 869bc97fa1aa..e7c5bcd8d423 100644 >> --- a/xen/arch/arm/gic-v3-lpi.c >> +++ b/xen/arch/arm/gic-v3-lpi.c >> @@ -183,7 +183,10 @@ void gicv3_do_LPI(unsigned int lpi) >> /* Find out if a guest mapped something to this physical LPI. */ >> hlpip = gic_get_host_lpi(lpi); >> if ( !hlpip ) >> + { >> + printk("%s: Received LPI %u but it is not mapped?\n", __func__, >> lpi); >> goto out; >> + } >> >> hlpi.data = read_u64_atomic(&hlpip->data); >> >> @@ -222,6 +225,9 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, >> int domain_id, >> { >> union host_lpi *hlpip, hlpi; >> >> + printk("%s: host_lpi %u domain %d virq_lpi %u\n", >> + __func__, host_lpi, domain_id, virq_lpi); >> + >> ASSERT(host_lpi >= LPI_OFFSET); >> >> host_lpi -= LPI_OFFSET; >> diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c index >> 58d939b85f92..89ef137b3e6b 100644 >> --- a/xen/arch/arm/vgic-v3-its.c >> +++ b/xen/arch/arm/vgic-v3-its.c >> @@ -897,7 +897,7 @@ out_unlock: >> >> static void dump_its_command(uint64_t *command) >> { >> - gdprintk(XENLOG_WARNING, " cmd 0x%02lx: %016lx %016lx %016lx >> %016lx\n", >> + gdprintk(XENLOG_WARNING, "vITS cmd 0x%02lx: %016lx %016lx %016lx >> %016lx\n", >> its_cmd_get_command(command), >> command[0], command[1], command[2], command[3]); >> } >> @@ -926,6 +926,8 @@ static int vgic_its_handle_cmds(struct domain *d, >> struct virt_its *its) >> if ( ret ) >> return ret; >> >> + dump_its_command(command); >> + >> switch ( its_cmd_get_command(command) ) >> { >> case GITS_CMD_CLEAR: >> >> >> -- >> Julien Grall > > [0] https://www.mail-archive.com/u-boot@lists.denx.de/msg379708.html > [1] > diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c > index 9558bad96a..d175ba52b0 100644 > --- a/xen/arch/arm/gic-v3-its.c > +++ b/xen/arch/arm/gic-v3-its.c > @@ -87,6 +87,10 @@ static int its_send_command(struct host_its *hw_its, const void *its_cmd) > /* No ITS commands from an interrupt handler (at the moment). */ > ASSERT(!in_irq()); > > + printk(XENLOG_WARNING "pITS cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", > + (((uint64_t *) its_cmd)[0] >> 0) & GENMASK(8 - 1, 0), > + ((uint64_t *) its_cmd)[0], ((uint64_t *) its_cmd)[1], ((uint64_t *) its_cmd)[2], ((uint64_t *) its_cmd)[3]); > + > spin_lock(&hw_its->cmd_lock); > > do { > diff --git a/xen/arch/arm/gic-v3-lpi.c b/xen/arch/arm/gic-v3-lpi.c > index 78b9521b21..2c3b0fc9e5 100644 > --- a/xen/arch/arm/gic-v3-lpi.c > +++ b/xen/arch/arm/gic-v3-lpi.c > @@ -181,8 +181,10 @@ void gicv3_do_LPI(unsigned int lpi) > > /* Find out if a guest mapped something to this physical LPI. */ > hlpip = gic_get_host_lpi(lpi); > - if ( !hlpip ) > + if ( !hlpip ) { > + printk("%s: Received LPI %u but it is not mapped?\n", __func__, lpi); > goto out; > + } > > hlpi.data = read_u64_atomic(&hlpip->data); > > @@ -221,6 +223,9 @@ void gicv3_lpi_update_host_entry(uint32_t host_lpi, int domain_id, > { > union host_lpi *hlpip, hlpi; > > + printk("%s: host_lpi %u domain %d virt_lpi %u\n", > + __func__, host_lpi, domain_id, virt_lpi); > + > ASSERT(host_lpi >= LPI_OFFSET); > > host_lpi -= LPI_OFFSET; > diff --git a/xen/arch/arm/vgic-v3-its.c b/xen/arch/arm/vgic-v3-its.c > index 6e153c698d..dd5081ef80 100644 > --- a/xen/arch/arm/vgic-v3-its.c > +++ b/xen/arch/arm/vgic-v3-its.c > @@ -897,7 +897,7 @@ out_unlock: > > static void dump_its_command(uint64_t *command) > { > - gdprintk(XENLOG_WARNING, " cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", > + gdprintk(XENLOG_WARNING, "vITS cmd 0x%02lx: %016lx %016lx %016lx %016lx\n", > its_cmd_get_command(command), > command[0], command[1], command[2], command[3]); > } > @@ -926,6 +926,8 @@ static int vgic_its_handle_cmds(struct domain *d, struct virt_its *its) > if ( ret ) > return ret; > > + dump_its_command(command); > + > switch ( its_cmd_get_command(command) ) > { > case GITS_CMD_CLEAR: > <boot-xendebug.log><xen-debug-output.txt> ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2020-11-23 11:41 ` Rahul Singh @ 2020-11-23 18:41 ` Julien Grall 2020-11-23 22:31 ` AW: " Leo Krueger 0 siblings, 1 reply; 16+ messages in thread From: Julien Grall @ 2020-11-23 18:41 UTC (permalink / raw) To: Rahul Singh, Leo Krueger Cc: Stefano Stabellini, Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand Marquis On 23/11/2020 11:41, Rahul Singh wrote: > Hello , Hi Rahul, >> On 22 Nov 2020, at 10:55 pm, Leo Krueger <leo.krueger@zal.aero> wrote: >> root@kontron-sal28:~# ip link set up dev gbe0 >> (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c 0000000000000001 0000000000000000 0000000000000000 >> (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 >> [ 34.034598] Atheros 8031 ethernet 0000:00:00.3:05: attached PHY driver [Atheros 8031 ethernet] (mii_bus:phy_addr=0000:00:00.3:05, irq=POLL) >> [ 34.041111] 8021q: adding VLAN 0 to HW filter on device gbe0 >> [ 34.041209] IPv6: ADDRCONF(NETDEV_UP): gbe0: link is not ready >> root@kontron-sal28:~# [ 35.041951] fsl_enetc 0000:00:00.0 gbe0: Link is Down >> [ 38.114426] fsl_enetc 0000:00:00.0 gbe0: Link is Up - 1Gbps/Full - flow control off >> [ 38.114508] IPv6: ADDRCONF(NETDEV_CHANGE): gbe0: link becomes ready >> >> Does that tell you anything? >> > > I just checked the logs shared, what I found out that there’s is an error while booting to configure the MSI for the PCI device because of that there will be case that Device Id generate out-of-band is not mapped correctly to ITS device table created while initialising the MSI for the device. > I might be wrong let someone else also comments on this. I think there might be multiple issues. You spotted one below :). > [ 0.173964] OF: /soc/pcie@1f0000000: Invalid msi-map translation - no match for rid 0xf8 on (null) Leo, just to confirm, this error message is not spotted when booting Linux on baremetal? Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 16+ messages in thread
* AW: Xen data from meta-virtualization layer 2020-11-23 18:41 ` Julien Grall @ 2020-11-23 22:31 ` Leo Krueger 0 siblings, 0 replies; 16+ messages in thread From: Leo Krueger @ 2020-11-23 22:31 UTC (permalink / raw) To: Julien Grall, Rahul Singh Cc: Stefano Stabellini, Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand Marquis Hi, Thanks for your effort! > -----Ursprüngliche Nachricht----- > Von: Julien Grall <julien@xen.org> > Gesendet: Montag, 23. November 2020 19:42 > An: Rahul Singh <Rahul.Singh@arm.com>; Leo Krueger > <leo.krueger@zal.aero> > Cc: Stefano Stabellini <stefano.stabellini@xilinx.com>; Peng Fan > <peng.fan@nxp.com>; brucea@xilinx.com; Cornelia Bruelhart > <cornelia.bruelhart@zal.aero>; oleksandr_andrushchenko@epam.com; xen- > devel@lists.xenproject.org; Bertrand Marquis > <Bertrand.Marquis@arm.com> > Betreff: Re: Xen data from meta-virtualization layer > > > > On 23/11/2020 11:41, Rahul Singh wrote: > > Hello , > > Hi Rahul, > > >> On 22 Nov 2020, at 10:55 pm, Leo Krueger <leo.krueger@zal.aero> wrote: > >> root@kontron-sal28:~# ip link set up dev gbe0 > >> (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c > >> 0000000000000001 0000000000000000 0000000000000000 > >> (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 > 0000000000000000 0000000000000000 0000000000000000 > >> [ 34.034598] Atheros 8031 ethernet 0000:00:00.3:05: attached PHY driver > [Atheros 8031 ethernet] (mii_bus:phy_addr=0000:00:00.3:05, irq=POLL) > >> [ 34.041111] 8021q: adding VLAN 0 to HW filter on device gbe0 > >> [ 34.041209] IPv6: ADDRCONF(NETDEV_UP): gbe0: link is not ready > >> root@kontron-sal28:~# [ 35.041951] fsl_enetc 0000:00:00.0 gbe0: Link is > Down > >> [ 38.114426] fsl_enetc 0000:00:00.0 gbe0: Link is Up - 1Gbps/Full - flow > control off > >> [ 38.114508] IPv6: ADDRCONF(NETDEV_CHANGE): gbe0: link becomes > ready > >> > >> Does that tell you anything? > >> > > > > I just checked the logs shared, what I found out that there’s is an error > while booting to configure the MSI for the PCI device because of that there > will be case that Device Id generate out-of-band is not mapped correctly to > ITS device table created while initialising the MSI for the device. > > I might be wrong let someone else also comments on this. > > I think there might be multiple issues. You spotted one below :). > > > [ 0.173964] OF: /soc/pcie@1f0000000: Invalid msi-map translation - no > match for rid 0xf8 on (null) > > Leo, just to confirm, this error message is not spotted when booting Linux on > baremetal? In fact it is: [ 0.160077] OF: /soc/pcie@1f0000000: Invalid msi-map translation - no match for rid 0xf8 on (null) But everything works as expected here: 110: 34288 0 ITS-MSI 1 Edge gbe0-rxtx0 111: 0 6196 ITS-MSI 2 Edge gbe0-rxtx1 > > Cheers, Best wishes > > -- > Julien Grall ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: AW: AW: AW: AW: AW: Xen data from meta-virtualization layer 2020-11-22 22:55 ` AW: " Leo Krueger 2020-11-23 11:41 ` Rahul Singh @ 2020-11-23 18:27 ` Julien Grall 2020-11-24 23:11 ` AW: " Leo Krueger 1 sibling, 1 reply; 16+ messages in thread From: Julien Grall @ 2020-11-23 18:27 UTC (permalink / raw) To: Leo Krueger, Stefano Stabellini Cc: Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand.Marquis On 22/11/2020 22:55, Leo Krueger wrote: > Hi Julien, Hi Leo, > > finally I could try out what you suggested, please find my answers inline. Thank you for sending the logs! > >> -----Ursprüngliche Nachricht----- >> Von: Julien Grall <julien@xen.org> >> Gesendet: Mittwoch, 18. November 2020 13:24 >> An: Stefano Stabellini <stefano.stabellini@xilinx.com>; Leo Krueger >> <leo.krueger@zal.aero> >> Cc: Peng Fan <peng.fan@nxp.com>; brucea@xilinx.com; Cornelia Bruelhart >> <cornelia.bruelhart@zal.aero>; oleksandr_andrushchenko@epam.com; xen- >> devel@lists.xenproject.org; Bertrand.Marquis@arm.com >> Betreff: Re: AW: AW: AW: AW: Xen data from meta-virtualization layer >> >> Hi, >> >> On 17/11/2020 23:53, Stefano Stabellini wrote: >>> Adding Bertrand, Oleksandr, Julien, and others -- they have a more >>> recent experience with GICv3 ITS than me and might be able to help. >>> I am attaching the device tree Leo sent a few days ago for reference. >>> >>> >>> Typically when you can set the ethernet link up and no packets are >>> exchanged it is because of a missing interrupt. In this case a missing >>> MSI. >>> >>> Bertrand, I believe you tried the GIC ITS driver with PCI devices >>> recently. It is expected to work correctly with MSIs in Dom0, right? >> >> OSSTest has some hardware (e.g. Thunder-X) where ITS is required to boot >> Dom0. I haven't seen any failure on recent Xen. We are testing 4.11 and >> onwards on Thunder-X. >> >> However, it may be possible that some more work is necessary for other >> hardware (e.g. workaround, missing code...). See more below. >> >>> >>> >>> >>> On Tue, 17 Nov 2020, Leo Krueger wrote: >>>> Hi, >>>> >>>> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it >>>> before...) but then had to add the following node to my device tree >>>> >>>> gic_lpi_base: syscon@0x80000000 { >>>> compatible = "gic-lpi-base"; >> >> I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, could >> you clarify which flavor/version of Linux you are using? > > It is Linux 4.19 from Yocto (Warror release). XEN 4.13.2. Do you have a link to the Linux tree? Is there any additional patches on top of vanilla? > While searching around the Internet for any solution, I came across [0] which contained the gic-lpi-base node. > So I just tried adding it (quite desperate I know) and voila, it at least brought me one step further (XEN exposing the ITS)... I am slightly confused to how this would help. Xen and, AFAICT, Linux don't understand gic-lpi-base. Do you have modification in your Linux to use it? Looking at the DT changes in [0], it looks like the node is not a child of gic@. So I think Xen will map the region to Dom0. There are two things that I can notice: 1) This region is RAM, but I can't find any reserve node. Is there any specific code in Linux to reserve it? 2) The implementation in U-boot seems to suggest that the firmware will configure the LPIs and then enable it. If that's the case, then Xen needs to re-use the table in the DT rather than allocating a new one. However, I would have expected an error message in the log: "GICv3: CPUx: Cannot initialize LPIs" At least Xen should not expose gic-lpi-base to the kernel, but I will wait on more details about the Linux kernel used before commenting more. I would also be interested to know more details about the failure when gic-lpi-base is not added in your DT. In particular, I am interested to understand why Xen would not expose the ITS as we don't parse that node. [...] > For XEN 4.13.2 I had to adapt your patch slightly [1], see below (yes I know, quite ugly in parts). No worries, debug patches are not meant to be nice to read ;). > Find attached the boot log and an output of "xl dmesg" which is truncated due to the large amount of messages. > > When enabling the network interface (gbe0), the following output is visible: > > root@kontron-sal28:~# ip link set up dev gbe0 > (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c 0000000000000001 0000000000000000 0000000000000000 > (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 0000000000000000 0000000000000000 0000000000000000 0xc is INV and 0x5 is SYNC. Most likely the driver unmask the interrupt by writing in the property table (access are not trapped to Xen) and then requested to invalidate the cache state. > [ 34.034598] Atheros 8031 ethernet 0000:00:00.3:05: attached PHY driver [Atheros 8031 ethernet] (mii_bus:phy_addr=0000:00:00.3:05, irq=POLL) > [ 34.041111] 8021q: adding VLAN 0 to HW filter on device gbe0 > [ 34.041209] IPv6: ADDRCONF(NETDEV_UP): gbe0: link is not ready > root@kontron-sal28:~# [ 35.041951] fsl_enetc 0000:00:00.0 gbe0: Link is Down > [ 38.114426] fsl_enetc 0000:00:00.0 gbe0: Link is Up - 1Gbps/Full - flow control off > [ 38.114508] IPv6: ADDRCONF(NETDEV_CHANGE): gbe0: link becomes ready > > Does that tell you anything? It is at least a good sign because it means Linux is able to initialize/talk to the vITS. I would lean towards one (or multiple) issue with pITS and/or the device-tree exposed to Linux. I am not entirely what exactly... I think having more details about the Linux setup would be helpful. I will reply on Rahul's e-mail separately. Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 16+ messages in thread
* AW: AW: AW: AW: AW: AW: Xen data from meta-virtualization layer 2020-11-23 18:27 ` AW: AW: AW: AW: " Julien Grall @ 2020-11-24 23:11 ` Leo Krueger 2020-11-25 2:14 ` Stefano Stabellini 0 siblings, 1 reply; 16+ messages in thread From: Leo Krueger @ 2020-11-24 23:11 UTC (permalink / raw) To: Julien Grall, Stefano Stabellini Cc: Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand.Marquis Hi, > -----Ursprüngliche Nachricht----- > Von: Julien Grall <julien@xen.org> > Gesendet: Montag, 23. November 2020 19:27 > An: Leo Krueger <leo.krueger@zal.aero>; Stefano Stabellini > <stefano.stabellini@xilinx.com> > Cc: Peng Fan <peng.fan@nxp.com>; brucea@xilinx.com; Cornelia Bruelhart > <cornelia.bruelhart@zal.aero>; oleksandr_andrushchenko@epam.com; xen- > devel@lists.xenproject.org; Bertrand.Marquis@arm.com > Betreff: Re: AW: AW: AW: AW: AW: Xen data from meta-virtualization layer > > > > On 22/11/2020 22:55, Leo Krueger wrote: > > Hi Julien, > > Hi Leo, > > > > > finally I could try out what you suggested, please find my answers inline. > > Thank you for sending the logs! > > > > >> -----Ursprüngliche Nachricht----- > >> Von: Julien Grall <julien@xen.org> > >> Gesendet: Mittwoch, 18. November 2020 13:24 > >> An: Stefano Stabellini <stefano.stabellini@xilinx.com>; Leo Krueger > >> <leo.krueger@zal.aero> > >> Cc: Peng Fan <peng.fan@nxp.com>; brucea@xilinx.com; Cornelia > >> Bruelhart <cornelia.bruelhart@zal.aero>; > >> oleksandr_andrushchenko@epam.com; xen- devel@lists.xenproject.org; > >> Bertrand.Marquis@arm.com > >> Betreff: Re: AW: AW: AW: AW: Xen data from meta-virtualization layer > >> > >> Hi, > >> > >> On 17/11/2020 23:53, Stefano Stabellini wrote: > >>> Adding Bertrand, Oleksandr, Julien, and others -- they have a more > >>> recent experience with GICv3 ITS than me and might be able to help. > >>> I am attaching the device tree Leo sent a few days ago for reference. > >>> > >>> > >>> Typically when you can set the ethernet link up and no packets are > >>> exchanged it is because of a missing interrupt. In this case a > >>> missing MSI. > >>> > >>> Bertrand, I believe you tried the GIC ITS driver with PCI devices > >>> recently. It is expected to work correctly with MSIs in Dom0, right? > >> > >> OSSTest has some hardware (e.g. Thunder-X) where ITS is required to > >> boot Dom0. I haven't seen any failure on recent Xen. We are testing > >> 4.11 and onwards on Thunder-X. > >> > >> However, it may be possible that some more work is necessary for > >> other hardware (e.g. workaround, missing code...). See more below. > >> > >>> > >>> > >>> > >>> On Tue, 17 Nov 2020, Leo Krueger wrote: > >>>> Hi, > >>>> > >>>> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it > >>>> before...) but then had to add the following node to my device tree > >>>> > >>>> gic_lpi_base: syscon@0x80000000 { > >>>> compatible = "gic-lpi-base"; > >> > >> I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, > >> could you clarify which flavor/version of Linux you are using? > > > > It is Linux 4.19 from Yocto (Warror release). XEN 4.13.2. > > Do you have a link to the Linux tree? Is there any additional patches on top of > vanilla? Linux tree is found here: https://github.com/kontron/linux-smarc-sal28/commits/master-LSDK-19.09 (up to the latest commit in that branch) > > > While searching around the Internet for any solution, I came across [0] > which contained the gic-lpi-base node. > > So I just tried adding it (quite desperate I know) and voila, it at least > brought me one step further (XEN exposing the ITS)... > > I am slightly confused to how this would help. Xen and, AFAICT, Linux don't > understand gic-lpi-base. Do you have modification in your Linux to use it? I have no idea, to be honest. Maybe it is about the memory being reserved due to that node or something like that? > > Looking at the DT changes in [0], it looks like the node is not a child of gic@. > So I think Xen will map the region to Dom0. > > There are two things that I can notice: > 1) This region is RAM, but I can't find any reserve node. Is there any specific > code in Linux to reserve it? > 2) The implementation in U-boot seems to suggest that the firmware will > configure the LPIs and then enable it. If that's the case, then Xen needs to > re-use the table in the DT rather than allocating a new one. > However, I would have expected an error message in the log: > > "GICv3: CPUx: Cannot initialize LPIs" > > At least Xen should not expose gic-lpi-base to the kernel, but I will wait on > more details about the Linux kernel used before commenting more. > > I would also be interested to know more details about the failure when gic- > lpi-base is not added in your DT. In particular, I am interested to understand > why Xen would not expose the ITS as we don't parse that node. How can I supply you with more information in regard to that? Without that node, ITS was not exposed at all. > > [...] > > > For XEN 4.13.2 I had to adapt your patch slightly [1], see below (yes I know, > quite ugly in parts). > > No worries, debug patches are not meant to be nice to read ;). > > > Find attached the boot log and an output of "xl dmesg" which is truncated > due to the large amount of messages. > > > > When enabling the network interface (gbe0), the following output is > visible: > > > > root@kontron-sal28:~# ip link set up dev gbe0 > > (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x0c: 000000170000000c > > 0000000000000001 0000000000000000 0000000000000000 > > (XEN) vgic-v3-its.c:902:d0v0 vITS cmd 0x05: 0000000000000005 > > 0000000000000000 0000000000000000 0000000000000000 > > 0xc is INV and 0x5 is SYNC. Most likely the driver unmask the interrupt by > writing in the property table (access are not trapped to Xen) and then > requested to invalidate the cache state. > > > [ 34.034598] Atheros 8031 ethernet 0000:00:00.3:05: attached PHY driver > [Atheros 8031 ethernet] (mii_bus:phy_addr=0000:00:00.3:05, irq=POLL) > > [ 34.041111] 8021q: adding VLAN 0 to HW filter on device gbe0 > > [ 34.041209] IPv6: ADDRCONF(NETDEV_UP): gbe0: link is not ready > > root@kontron-sal28:~# [ 35.041951] fsl_enetc 0000:00:00.0 gbe0: Link is > Down > > [ 38.114426] fsl_enetc 0000:00:00.0 gbe0: Link is Up - 1Gbps/Full - flow > control off > > [ 38.114508] IPv6: ADDRCONF(NETDEV_CHANGE): gbe0: link becomes > ready > > > > Does that tell you anything? > > It is at least a good sign because it means Linux is able to initialize/talk to the > vITS. > > I would lean towards one (or multiple) issue with pITS and/or the device-tree > exposed to Linux. I am not entirely what exactly... I think having more details > about the Linux setup would be helpful. Ok let me know what you need from my side. I was considering the following things to try out next: - a more recent u-boot version as this might fix problems with the msi-map (at least that is what I think, I am not an expert here) - a different device tree (a more recent one, ...) - ... > > I will reply on Rahul's e-mail separately. > > Cheers, Best wishes! > > -- > Julien Grall ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: AW: AW: AW: AW: AW: AW: Xen data from meta-virtualization layer 2020-11-24 23:11 ` AW: " Leo Krueger @ 2020-11-25 2:14 ` Stefano Stabellini 2022-02-04 13:58 ` Michael Walle 0 siblings, 1 reply; 16+ messages in thread From: Stefano Stabellini @ 2020-11-25 2:14 UTC (permalink / raw) To: Leo Krueger, Zhiqiang.Hou Cc: Julien Grall, Stefano Stabellini, Peng Fan, brucea, Cornelia Bruelhart, oleksandr_andrushchenko, xen-devel, Bertrand.Marquis + Zhiqiang Hou On Tue, 24 Nov 2020, Leo Krueger wrote: > > >>> On Tue, 17 Nov 2020, Leo Krueger wrote: > > >>>> Hi, > > >>>> > > >>>> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it > > >>>> before...) but then had to add the following node to my device tree > > >>>> > > >>>> gic_lpi_base: syscon@0x80000000 { > > >>>> compatible = "gic-lpi-base"; > > >> > > >> I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, > > >> could you clarify which flavor/version of Linux you are using? > > > > > > It is Linux 4.19 from Yocto (Warror release). XEN 4.13.2. > > > > Do you have a link to the Linux tree? Is there any additional patches on top of > > vanilla? > > Linux tree is found here: https://github.com/kontron/linux-smarc-sal28/commits/master-LSDK-19.09 > (up to the latest commit in that branch) [...] > > Looking at the DT changes in [0], it looks like the node is not a child of gic@. > > So I think Xen will map the region to Dom0. > > > > There are two things that I can notice: > > 1) This region is RAM, but I can't find any reserve node. Is there any specific > > code in Linux to reserve it? > > 2) The implementation in U-boot seems to suggest that the firmware will > > configure the LPIs and then enable it. If that's the case, then Xen needs to > > re-use the table in the DT rather than allocating a new one. That Linux tree has no mentions of gic-lpi-base. That means that gic-lpi-base is only used in u-boot, not in Linux. In particular the most relevant commit is af288cb291da3abef6be0875527729296f7de7a0. In regards to the reserved-memory regions, maybe we are not seeing them because Leo posted the host device tree, not the one passed at runtime from u-boot to Linux? If so, Leo, could you please boot Linux on native (no Xen) and get the device tree from there at runtime using dtc -I fs -O dts /proc/device-tree ? However, the name of the reserved-memory region created by u-boot seems to be "lpi_rd_table". I cannot find any mentions of lpi_rd_table in the Linux kernel tree either. Zhiqiang, Leo is trying to boot Xen on sAL28. Linux booting on Xen throws errors in regards to GIC/ITS initialization. On other hardware Xen can use and virtualize GICv3 and ITS just fine. Could you please explain what is different about sAL28 and how Xen/Linux is expected to use the lpi_rd_table reserved-memory region? ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2020-11-25 2:14 ` Stefano Stabellini @ 2022-02-04 13:58 ` Michael Walle 2022-02-04 21:11 ` Stefano Stabellini 0 siblings, 1 reply; 16+ messages in thread From: Michael Walle @ 2022-02-04 13:58 UTC (permalink / raw) To: stefano.stabellini Cc: Bertrand.Marquis, Zhiqiang.Hou, brucea, cornelia.bruelhart, julien, leo.krueger, oleksandr_andrushchenko, peng.fan, xen-devel, Michael Walle Hi all, > + Zhiqiang Hou > > On Tue, 24 Nov 2020, Leo Krueger wrote: > > > >>> On Tue, 17 Nov 2020, Leo Krueger wrote: > > > >>>> Hi, > > > >>>> > > > >>>> I enabled CONFIG_HAS_ITS (what a stupid mistake by me to not set it > > > >>>> before...) but then had to add the following node to my device tree > > > >>>> > > > >>>> gic_lpi_base: syscon@0x80000000 { > > > >>>> compatible = "gic-lpi-base"; > > > >> > > > >> I couldn't find this compatible defined/used in Linux 5.10-rc4. @Leo, > > > >> could you clarify which flavor/version of Linux you are using? > > > > > > > > It is Linux 4.19 from Yocto (Warror release). XEN 4.13.2. > > > > > > Do you have a link to the Linux tree? Is there any additional patches on top of > > > vanilla? > > > > Linux tree is found here: https://github.com/kontron/linux-smarc-sal28/commits/master-LSDK-19.09 > > (up to the latest commit in that branch) FWIW, I'm the developer of the support for this board both in the mentioned branch as well as upstream. If you don't need graphics you shouldn't really be using the branch above, but the latest vanilla kernel release. > [...] > > > > Looking at the DT changes in [0], it looks like the node is not a child of gic@. > > > So I think Xen will map the region to Dom0. > > > > > > There are two things that I can notice: > > > 1) This region is RAM, but I can't find any reserve node. Is there any specific > > > code in Linux to reserve it? > > > 2) The implementation in U-boot seems to suggest that the firmware will > > > configure the LPIs and then enable it. If that's the case, then Xen needs to > > > re-use the table in the DT rather than allocating a new one. > > That Linux tree has no mentions of gic-lpi-base. That means that > gic-lpi-base is only used in u-boot, not in Linux. In particular the > most relevant commit is af288cb291da3abef6be0875527729296f7de7a0. That node was horrible hack from NXP for u-boot and was removed in a84cea06bb8fff69810a890ac0e4b47ea5726512. > In regards to the reserved-memory regions, maybe we are not seeing them > because Leo posted the host device tree, not the one passed at runtime > from u-boot to Linux? > > If so, Leo, could you please boot Linux on native (no Xen) and get the > device tree from there at runtime using dtc -I fs -O dts > /proc/device-tree ? > > > However, the name of the reserved-memory region created by u-boot seems > to be "lpi_rd_table". I cannot find any mentions of lpi_rd_table in the > Linux kernel tree either. > > Zhiqiang, Leo is trying to boot Xen on sAL28. Linux booting on Xen > throws errors in regards to GIC/ITS initialization. On other hardware > Xen can use and virtualize GICv3 and ITS just fine. Could you please > explain what is different about sAL28 and how Xen/Linux is expected to > use the lpi_rd_table reserved-memory region? I actually stumbled across this thread after trying out Xen myself. I'm using lastest vanilla u-boot (with pending PSCI patches), vanilla kernel and vanilla Xen. So far I've discovered, that xen complains that it cannot route IRQ64 to dom0. That is because on the LS1028A there is a dual UART (two 16550 with one shared interrupt) and xen takes the first UART and then tries to map the interrupt of the second UART to linux. For now, I don't know how this is solved correctly. As a quick hack, I removed the second uart node from the device tree. But what is more severe is that the iommu isn't set up correctly. I'm getting the following faults: (XEN) smmu: /soc/iommu@5000000: Unexpected global fault, this could be serious (XEN) smmu: /soc/iommu@5000000: GFSR 0x80000002, GFSYNR0 0x00000000, GFSYNR1 0x0000042a, GFSYNR2 0x00000000 If I decode it correctly, the streamid should be 0x2a which would be one of the PCI devices on the internal root complex. Probably the network card. This is the first developer experience with Xen, so please bear with me :) It seems that Xen doesn't add the master to the IOMMU. To me it seems that only devices with a 'iommus' dt property are added. But in case of PCI devices the parent only has a iommu-map property. And it makes me wonder why Leo has an almost working setup. Maybe I'm missing some patches though. -michael ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2022-02-04 13:58 ` Michael Walle @ 2022-02-04 21:11 ` Stefano Stabellini 2022-02-04 22:42 ` Michael Walle 0 siblings, 1 reply; 16+ messages in thread From: Stefano Stabellini @ 2022-02-04 21:11 UTC (permalink / raw) To: Michael Walle Cc: stefano.stabellini, Bertrand.Marquis, Zhiqiang.Hou, brucea, cornelia.bruelhart, julien, leo.krueger, oleksandr_andrushchenko, peng.fan, xen-devel On Fri, 4 Feb 2022, Michael Walle wrote: > > In regards to the reserved-memory regions, maybe we are not seeing them > > because Leo posted the host device tree, not the one passed at runtime > > from u-boot to Linux? > > > > If so, Leo, could you please boot Linux on native (no Xen) and get the > > device tree from there at runtime using dtc -I fs -O dts > > /proc/device-tree ? > > > > > > However, the name of the reserved-memory region created by u-boot seems > > to be "lpi_rd_table". I cannot find any mentions of lpi_rd_table in the > > Linux kernel tree either. > > > > Zhiqiang, Leo is trying to boot Xen on sAL28. Linux booting on Xen > > throws errors in regards to GIC/ITS initialization. On other hardware > > Xen can use and virtualize GICv3 and ITS just fine. Could you please > > explain what is different about sAL28 and how Xen/Linux is expected to > > use the lpi_rd_table reserved-memory region? > > I actually stumbled across this thread after trying out Xen myself. I'm > using lastest vanilla u-boot (with pending PSCI patches), vanilla kernel > and vanilla Xen. > > So far I've discovered, that xen complains that it cannot route IRQ64 to > dom0. That is because on the LS1028A there is a dual UART (two 16550 > with one shared interrupt) and xen takes the first UART and then tries > to map the interrupt of the second UART to linux. For now, I don't know > how this is solved correctly. As a quick hack, I removed the second uart > node from the device tree. This is an interesting problem. Removing the second UART is a good workaround for now as there is no obvious solution I think. > But what is more severe is that the iommu isn't set up correctly. I'm > getting the following faults: > > (XEN) smmu: /soc/iommu@5000000: Unexpected global fault, this could be serious > (XEN) smmu: /soc/iommu@5000000: GFSR 0x80000002, GFSYNR0 0x00000000, GFSYNR1 0x0000042a, GFSYNR2 0x00000000 > > If I decode it correctly, the streamid should be 0x2a which would be one > of the PCI devices on the internal root complex. Probably the network > card. Yes there is DMA transaction with an "unknown" StreamID. I think the StreamID is 0x42a. It means that there is a DMA master on the board with StreamID 0x42a that is either: - not described in device tree - described in device tree with a different StreamID - the right StreamID is described device tree, but it is not picked up by Xen > This is the first developer experience with Xen, so please bear with me > :) It seems that Xen doesn't add the master to the IOMMU. To me it seems > that only devices with a 'iommus' dt property are added. But in case of > PCI devices the parent only has a iommu-map property. > > And it makes me wonder why Leo has an almost working setup. Maybe I'm > missing some patches though. Xen 4.16 is able to parse StreamID in the "iommus" property and also "mmu-masters" property. But It is not able to parse the "iommu-map" property yet. So if 0x42a is described in device tree using "iommu-map" then the error makes sense. A simple solution is to replace iommu-map with iommus in device tree. It is possible that someone in CC to this email might already have a patch to introduce parsing of iommu-map in Xen. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2022-02-04 21:11 ` Stefano Stabellini @ 2022-02-04 22:42 ` Michael Walle 2022-02-04 23:29 ` Julien Grall 0 siblings, 1 reply; 16+ messages in thread From: Michael Walle @ 2022-02-04 22:42 UTC (permalink / raw) To: Stefano Stabellini Cc: Bertrand.Marquis, Zhiqiang.Hou, brucea, cornelia.bruelhart, julien, leo.krueger, oleksandr_andrushchenko, peng.fan, xen-devel Hi Stefano, Am 2022-02-04 22:11, schrieb Stefano Stabellini: > On Fri, 4 Feb 2022, Michael Walle wrote: >> > In regards to the reserved-memory regions, maybe we are not seeing them >> > because Leo posted the host device tree, not the one passed at runtime >> > from u-boot to Linux? >> > >> > If so, Leo, could you please boot Linux on native (no Xen) and get the >> > device tree from there at runtime using dtc -I fs -O dts >> > /proc/device-tree ? >> > >> > >> > However, the name of the reserved-memory region created by u-boot seems >> > to be "lpi_rd_table". I cannot find any mentions of lpi_rd_table in the >> > Linux kernel tree either. >> > >> > Zhiqiang, Leo is trying to boot Xen on sAL28. Linux booting on Xen >> > throws errors in regards to GIC/ITS initialization. On other hardware >> > Xen can use and virtualize GICv3 and ITS just fine. Could you please >> > explain what is different about sAL28 and how Xen/Linux is expected to >> > use the lpi_rd_table reserved-memory region? >> >> I actually stumbled across this thread after trying out Xen myself. >> I'm >> using lastest vanilla u-boot (with pending PSCI patches), vanilla >> kernel >> and vanilla Xen. >> >> So far I've discovered, that xen complains that it cannot route IRQ64 >> to >> dom0. That is because on the LS1028A there is a dual UART (two 16550 >> with one shared interrupt) and xen takes the first UART and then tries >> to map the interrupt of the second UART to linux. For now, I don't >> know >> how this is solved correctly. As a quick hack, I removed the second >> uart >> node from the device tree. > > This is an interesting problem. Removing the second UART is a good > workaround for now as there is no obvious solution I think. But not a very user friendly one, though. I guess the first UART is disabled/removed by Xen? I haven't looked at how it is handled. Can't we search for other uarts with the same interrupt and disable these, too? Maybe conditionally by the SoC compatible? >> But what is more severe is that the iommu isn't set up correctly. I'm >> getting the following faults: >> >> (XEN) smmu: /soc/iommu@5000000: Unexpected global fault, this could be >> serious >> (XEN) smmu: /soc/iommu@5000000: GFSR 0x80000002, GFSYNR0 0x00000000, >> GFSYNR1 0x0000042a, GFSYNR2 0x00000000 >> >> If I decode it correctly, the streamid should be 0x2a which would be >> one >> of the PCI devices on the internal root complex. Probably the network >> card. > > Yes there is DMA transaction with an "unknown" StreamID. I think the > StreamID is 0x42a. It means that there is a DMA master on the board > with > StreamID 0x42a that is either: > > - not described in device tree > - described in device tree with a different StreamID > - the right StreamID is described device tree, but it is not picked up > by Xen See below. >> This is the first developer experience with Xen, so please bear with >> me >> :) It seems that Xen doesn't add the master to the IOMMU. To me it >> seems >> that only devices with a 'iommus' dt property are added. But in case >> of >> PCI devices the parent only has a iommu-map property. >> >> And it makes me wonder why Leo has an almost working setup. Maybe I'm >> missing some patches though. > > Xen 4.16 is able to parse StreamID in the "iommus" property and also > "mmu-masters" property. But It is not able to parse the "iommu-map" > property yet. So if 0x42a is described in device tree using "iommu-map" > then the error makes sense. > > A simple solution is to replace iommu-map with iommus in device tree. I'm not sure this is so easy, because they are dynamically assigned by the bootloader. Sure for now I could do that I guess, but iommu=0 works as well ;) I now got Xen and Linux booting and see the same problems with the GIC ITS, that is that the enetc interrupts aren't delivered to the dom0 linux. I've also applied the patch in this thread and I'm seeing the same as Leo. Full boot log is here [1]. I noticed the following. [ 0.168544] pci 0000:00:00.0: Failed to add - passthrough or MSI/MSI-X might fail! Not sure if it should work nonetheless. > It is possible that someone in CC to this email might already have a > patch to introduce parsing of iommu-map in Xen. I guess they've used the old mmu-masters property. Btw. I don't know if it matters, but the SMARC-sAL28 normally doesn't use TF-A and runs without it. Nonetheless, I've booted the board with the bl31 from NXP and it doesn't help either. There is still a difference between the NXP bootflow which uses bl1/bl2/bl31/u-boot and this board which uses u-boot-spl/u-boot or u-boot-spl/bl31/u-boot. I just found GIC setup code in the bl31. I'll also have a look there. -michael [1] https://pastebin.com/raw/XMjE3BvG ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2022-02-04 22:42 ` Michael Walle @ 2022-02-04 23:29 ` Julien Grall 2022-02-04 23:59 ` Michael Walle 2022-02-05 12:41 ` Julien Grall 0 siblings, 2 replies; 16+ messages in thread From: Julien Grall @ 2022-02-04 23:29 UTC (permalink / raw) To: Michael Walle Cc: Stefano Stabellini, Bertrand Marquis, Zhiqiang.Hou, brucea, cornelia.bruelhart, leo.krueger, Oleksandr Andrushchenko, Peng Fan, xen-devel Hi Michael, On Fri, 4 Feb 2022 at 22:42, Michael Walle <michael@walle.cc> wrote: > Am 2022-02-04 22:11, schrieb Stefano Stabellini: > > On Fri, 4 Feb 2022, Michael Walle wrote: > >> > In regards to the reserved-memory regions, maybe we are not seeing them > >> > because Leo posted the host device tree, not the one passed at runtime > >> > from u-boot to Linux? > >> > > >> > If so, Leo, could you please boot Linux on native (no Xen) and get the > >> > device tree from there at runtime using dtc -I fs -O dts > >> > /proc/device-tree ? > >> > > >> > > >> > However, the name of the reserved-memory region created by u-boot seems > >> > to be "lpi_rd_table". I cannot find any mentions of lpi_rd_table in the > >> > Linux kernel tree either. > >> > > >> > Zhiqiang, Leo is trying to boot Xen on sAL28. Linux booting on Xen > >> > throws errors in regards to GIC/ITS initialization. On other hardware > >> > Xen can use and virtualize GICv3 and ITS just fine. Could you please > >> > explain what is different about sAL28 and how Xen/Linux is expected to > >> > use the lpi_rd_table reserved-memory region? > >> > >> I actually stumbled across this thread after trying out Xen myself. > >> I'm > >> using lastest vanilla u-boot (with pending PSCI patches), vanilla > >> kernel > >> and vanilla Xen. > >> > >> So far I've discovered, that xen complains that it cannot route IRQ64 > >> to > >> dom0. That is because on the LS1028A there is a dual UART (two 16550 > >> with one shared interrupt) and xen takes the first UART and then tries > >> to map the interrupt of the second UART to linux. For now, I don't > >> know > >> how this is solved correctly. As a quick hack, I removed the second > >> uart > >> node from the device tree. > > > > This is an interesting problem. Removing the second UART is a good > > workaround for now as there is no obvious solution I think. > > But not a very user friendly one, though. I guess the first UART > is disabled/removed by Xen? I haven't looked at how it is handled. > > Can't we search for other uarts with the same interrupt and disable > these, too? Maybe conditionally by the SoC compatible? The problem sounds quite similar to the one we had on sunxi. Although the UART was on the same page rather than sharing interrupts. Xen has per-platform hook to prevent a device been assigned to dom0. For an example, you could look at: https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/platforms/sunxi.c > >> This is the first developer experience with Xen, so please bear with > >> me > >> :) It seems that Xen doesn't add the master to the IOMMU. To me it > >> seems > >> that only devices with a 'iommus' dt property are added. But in case > >> of > >> PCI devices the parent only has a iommu-map property. > >> > >> And it makes me wonder why Leo has an almost working setup. Maybe I'm > >> missing some patches though. > > > > Xen 4.16 is able to parse StreamID in the "iommus" property and also > > "mmu-masters" property. But It is not able to parse the "iommu-map" > > property yet. So if 0x42a is described in device tree using "iommu-map" > > then the error makes sense. > > > > A simple solution is to replace iommu-map with iommus in device tree. > > I'm not sure this is so easy, because they are dynamically assigned > by the bootloader. Sure for now I could do that I guess, but iommu=0 > works as well ;) > > I now got Xen and Linux booting and see the same problems with the > GIC ITS, that is that the enetc interrupts aren't delivered to the > dom0 linux. I've also applied the patch in this thread and I'm > seeing the same as Leo. Full boot log is here [1]. > > I noticed the following. > [ 0.168544] pci 0000:00:00.0: Failed to add - passthrough or > MSI/MSI-X might fail! This message is harmless. This is printed because Xen on Arm doesn't hypercall the hypercall to add a PCI device. On Arm, we don't need it yet (it might be necessary for PCI passthrough) and MSI/MSI-X are handled/registered the same way as on bare-metal (for your case through the ITS) > > Not sure if it should work nonetheless. I looked through the log and couldn't spot anything obvious. However, skimming through Linux, I noticed there are some changes in the ITS for freescale (now NXP) such as: drivers/irqchip/irq-gic-v3-its-fsl-mc-msi.c Is it something that might be used on the board you are using? If the answer is yes, then my suggestion would be to look how this is meant to interact with the ITS. It might be possible that we are missing some pieces in Xen to properly support it. > > > It is possible that someone in CC to this email might already have a > > patch to introduce parsing of iommu-map in Xen. Pass. I don't have any and couldn't find any submission on the ML. > > I guess they've used the old mmu-masters property. > > Btw. I don't know if it matters, but the SMARC-sAL28 normally doesn't > use TF-A and runs without it. Nonetheless, I've booted the board with > the bl31 from NXP and it doesn't help either. There is still a > difference between the NXP bootflow which uses bl1/bl2/bl31/u-boot > and this board which uses u-boot-spl/u-boot or u-boot-spl/bl31/u-boot. > > I just found GIC setup code in the bl31. I'll also have a look there. > > -michael > > [1] https://pastebin.com/raw/XMjE3BvG ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2022-02-04 23:29 ` Julien Grall @ 2022-02-04 23:59 ` Michael Walle 2022-02-05 12:41 ` Julien Grall 1 sibling, 0 replies; 16+ messages in thread From: Michael Walle @ 2022-02-04 23:59 UTC (permalink / raw) To: Julien Grall Cc: Stefano Stabellini, Bertrand Marquis, Zhiqiang.Hou, brucea, cornelia.bruelhart, leo.krueger, Oleksandr Andrushchenko, Peng Fan, xen-devel Hi Julien, Am 2022-02-05 00:29, schrieb Julien Grall: [..] >> But not a very user friendly one, though. I guess the first UART >> is disabled/removed by Xen? I haven't looked at how it is handled. >> >> Can't we search for other uarts with the same interrupt and disable >> these, too? Maybe conditionally by the SoC compatible? > > The problem sounds quite similar to the one we had on sunxi. Although > the UART was on the same page rather than sharing interrupts. > > Xen has per-platform hook to prevent a device been assigned > to dom0. For an example, you could look at: > > https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/platforms/sunxi.c Nice. At least, this is working now ;) I'll send a patch in the next few days. I guess it's safe to assume that we can always remove both UARTs on the LS1028A (probably on most layerscapes). The most common use case is to use the first UART for Xen. You could run Xen without console (?), then you'd miss the possibility to map the DUART. Or there could be a new driver for the LPUART on the LS1028A. In this case, the DUART wouldn't be used by Xen either. But I think we should start simple and just remove the DUART altogether via that hook. -michael ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Xen data from meta-virtualization layer 2022-02-04 23:29 ` Julien Grall 2022-02-04 23:59 ` Michael Walle @ 2022-02-05 12:41 ` Julien Grall 1 sibling, 0 replies; 16+ messages in thread From: Julien Grall @ 2022-02-05 12:41 UTC (permalink / raw) To: Michael Walle Cc: Stefano Stabellini, Bertrand Marquis, Zhiqiang.Hou, brucea, cornelia.bruelhart, leo.krueger, Oleksandr Andrushchenko, Peng Fan, xen-devel On 04/02/2022 23:29, Julien Grall wrote: > This message is harmless. This is printed because Xen on Arm > doesn't hypercall the hypercall to add a PCI device. On Arm, I meant "doesn't need the hypercall...". > we don't need it yet (it might be necessary for PCI passthrough) and > MSI/MSI-X are handled/registered the same way as on bare-metal > (for your case through the ITS) > >> >> Not sure if it should work nonetheless. > > I looked through the log and couldn't spot anything obvious. However, > skimming through Linux, I noticed there are some changes in the > ITS for freescale (now NXP) such as: > > drivers/irqchip/irq-gic-v3-its-fsl-mc-msi.c > > Is it something that might be used on the board you are using? > > If the answer is yes, then my suggestion would be to look > how this is meant to interact with the ITS. It might be possible > that we are missing some pieces in Xen to properly support it. Another tree you might want to look at is https://source.codeaurora.org/external/imx/imx-xen. It contains a bunch of patches for NXP SoC that was never upstreamed. > >> >>> It is possible that someone in CC to this email might already have a >>> patch to introduce parsing of iommu-map in Xen. > > Pass. I don't have any and couldn't find any submission on the ML. > > >> >> I guess they've used the old mmu-masters property. >> >> Btw. I don't know if it matters, but the SMARC-sAL28 normally doesn't >> use TF-A and runs without it. Nonetheless, I've booted the board with >> the bl31 from NXP and it doesn't help either. There is still a >> difference between the NXP bootflow which uses bl1/bl2/bl31/u-boot >> and this board which uses u-boot-spl/u-boot or u-boot-spl/bl31/u-boot. >> >> I just found GIC setup code in the bl31. I'll also have a look there. >> >> -michael >> >> [1] https://pastebin.com/raw/XMjE3BvG Cheers, -- Julien Grall ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2022-02-05 12:42 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <AM4PR0501MB2227089FDDF0209EF6E215D9E6100@AM4PR0501MB2227.eurprd05.prod.outlook.com>
[not found] ` <AM4PR0501MB22274E52A5A3BE912D477D8CE6EA0@AM4PR0501MB2227.eurprd05.prod.outlook.com>
[not found] ` <HE1PR05MB47941E23CE053CE72F18867C8BEA0@HE1PR05MB4794.eurprd05.prod.outlook.com>
[not found] ` <alpine.DEB.2.21.2011091858010.21307@sstabellini-ThinkPad-T480s>
[not found] ` <HE1PR05MB4794B5C57A54A29A48EE8EAE8BE90@HE1PR05MB4794.eurprd05.prod.outlook.com>
[not found] ` <alpine.DEB.2.21.2011101842500.21307@sstabellini-ThinkPad-T480s>
[not found] ` <DB6PR0402MB27608A03EC717053E392A92988E80@DB6PR0402MB2760.eurprd04.prod.outlook.com>
[not found] ` <HE1PR05MB47940ED4E5FDC0BADC54C8E78BE80@HE1PR05MB4794.eurprd05.prod.outlook.com>
[not found] ` <DB6PR0402MB2760CEEABA9F52CDEB27C1DB88E80@DB6PR0402MB2760.eurprd04.prod.outlook.com>
[not found] ` <HE1PR05MB47944761ED6A26D3E2CE15868BE40@HE1PR05MB4794.eurprd05.prod.outlook.com>
[not found] ` <alpine.DEB.2.21.2011161656080.20906@sstabellini-ThinkPad-T480s>
[not found] ` <HE1PR05MB4794569AC67109AF8B6517268BE20@HE1PR05MB4794.eurprd05.prod.outlook.com>
2020-11-17 23:53 ` AW: AW: AW: AW: Xen data from meta-virtualization layer Stefano Stabellini
2020-11-18 12:03 ` Rahul Singh
2020-11-18 12:23 ` AW: AW: AW: AW: " Julien Grall
2020-11-22 22:55 ` AW: " Leo Krueger
2020-11-23 11:41 ` Rahul Singh
2020-11-23 18:41 ` Julien Grall
2020-11-23 22:31 ` AW: " Leo Krueger
2020-11-23 18:27 ` AW: AW: AW: AW: " Julien Grall
2020-11-24 23:11 ` AW: " Leo Krueger
2020-11-25 2:14 ` Stefano Stabellini
2022-02-04 13:58 ` Michael Walle
2022-02-04 21:11 ` Stefano Stabellini
2022-02-04 22:42 ` Michael Walle
2022-02-04 23:29 ` Julien Grall
2022-02-04 23:59 ` Michael Walle
2022-02-05 12:41 ` Julien Grall
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.