b.a.t.m.a.n.lists.open-mesh.org archive mirror
 help / color / mirror / Atom feed
* Kernel crash when using batman-adv 2022.0 over VXLAN
@ 2022-04-16  0:23 Felix Kaechele
  2022-04-16 10:08 ` Sven Eckelmann
  0 siblings, 1 reply; 2+ messages in thread
From: Felix Kaechele @ 2022-04-16  0:23 UTC (permalink / raw)
  To: b.a.t.m.a.n

[-- Attachment #1: Type: text/plain, Size: 2136 bytes --]

Hi there,

I am seeing odd kernel crashes when using batman-adv over VXLAN.
This is in a Freifunk environment on our VPN gateway side where clients 
connect using L2TPv3 tunnels and the gateways are interconnected using 
VXLAN.

The setup is as follows on both ends:

- Both machines are Hetzner Cloud VMs (CX11)
- OS: Fedora 36 (Pre-release) / Kernel 5.17.2-300.fc36.x86_64
- batctl 2022.0 [batman-adv: 2022.0]
  (using mainline module provided by - the distro's 
kernel-modules-extras package)
- VXLAN on UDP going out through the WAN connection to the other machines
- VXLAN FDB using unicast with static flooding (00:00:00:00:00:00
  MAC entries with dst configured)
- VXLAN connectivity is confirmed to be working between all connected 
gateways using IP and ICMP
- batctl if add <vxlan-interface>
- There is also another bridge attached to the batman-adv interface that 
bundles all the L2TPv3 tunnels. It has bridge port isolation enabled to 
prevent the nodes from believing that they are connected to each other 
on the same L2 network.

The mesh is a fairly busy mesh with ~150 nodes and 250-350 client 
devices at any given time. We currently use 4 gateways (instances with 
batman-adv in gateway mode).

So far I have been able to figure out the follwing:
- The issue usually happens within 1-2 minutes of a certain(?) client 
connecting to the machine using a L2TPv3 tunnel
- The issue does not manifest itself when using GRETAP interfaces 
between the gateway machines in matching pairs on either side of the 
link and those GRETAP interfaces being added to the batman-adv interface
- The number of gateways interconnected using VXLAN does not matter. N-1 
machines will crash with this issue and the crashes are rolling in 
nature, disabling all gateways but one within the N*1-2 minute timeframe.
- I remember fighting this issue way back in the past (I think it was 
around 2017), so this issue has been there for a while in our setup.

I have attached a kdump dmesg log that has the panic message.
A vmcore file is available for kernel-5.17.2-300.fc36.x86_64, which I 
can provide if needed. It is 77MB.

Thanks,
Felix

[-- Attachment #2: vmcore-dmesg.txt --]
[-- Type: text/plain, Size: 68378 bytes --]

[    0.000000] Linux version 5.17.2-300.fc36.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 12.0.1 20220308 (Red Hat 12.0.1-0), GNU ld version 2.37-24.fc36) #1 SMP PREEMPT Fri Apr 8 14:35:55 UTC 2022
[    0.000000] Command line: BOOT_IMAGE=(hd0,gpt1)/boot/vmlinuz-5.17.2-300.fc36.x86_64 root=UUID=ce1b7344-16ae-42a9-a2d3-2c2aa35f2742 ro no_timer_check net.ifnames=0 console=tty1 console=ttyS0,115200n8 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: xstate_offset[5]: 1088, xstate_sizes[5]:   64
[    0.000000] x86/fpu: xstate_offset[6]: 1152, xstate_sizes[6]:  512
[    0.000000] x86/fpu: xstate_offset[7]: 1664, xstate_sizes[7]: 1024
[    0.000000] x86/fpu: xstate_offset[9]: 2688, xstate_sizes[9]:    8
[    0.000000] x86/fpu: Enabled xstate features 0x2e7, context size is 2696 bytes, using 'standard' format.
[    0.000000] signal: max sigframe size: 3632
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007cfdbfff] usable
[    0.000000] BIOS-e820: [mem 0x000000007cfdc000-0x000000007cffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.8 present.
[    0.000000] DMI: Hetzner vServer, BIOS 20171111 11/11/2017
[    0.000000] Hypervisor detected: KVM
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000002] kvm-clock: using sched offset of 909805710402 cycles
[    0.000006] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000018] tsc: Detected 2099.998 MHz processor
[    0.000939] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000948] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000958] last_pfn = 0x7cfdc max_arch_pfn = 0x400000000
[    0.001040] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT  
[    0.013658] found SMP MP-table at [mem 0x000f5c80-0x000f5c8f]
[    0.013709] Using GB pages for direct mapping
[    0.013979] RAMDISK: [mem 0x33c98000-0x35e43fff]
[    0.013992] ACPI: Early table checksum verification disabled
[    0.014008] ACPI: RSDP 0x00000000000F5AC0 000014 (v00 BOCHS )
[    0.014020] ACPI: RSDT 0x000000007CFE1522 000030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.014031] ACPI: FACP 0x000000007CFE13FE 000074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.014040] ACPI: DSDT 0x000000007CFE0040 0013BE (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.014046] ACPI: FACS 0x000000007CFE0000 000040
[    0.014052] ACPI: APIC 0x000000007CFE1472 000078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.014058] ACPI: HPET 0x000000007CFE14EA 000038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.014062] ACPI: Reserving FACP table memory at [mem 0x7cfe13fe-0x7cfe1471]
[    0.014065] ACPI: Reserving DSDT table memory at [mem 0x7cfe0040-0x7cfe13fd]
[    0.014067] ACPI: Reserving FACS table memory at [mem 0x7cfe0000-0x7cfe003f]
[    0.014069] ACPI: Reserving APIC table memory at [mem 0x7cfe1472-0x7cfe14e9]
[    0.014070] ACPI: Reserving HPET table memory at [mem 0x7cfe14ea-0x7cfe1521]
[    0.014567] No NUMA configuration found
[    0.014569] Faking a node at [mem 0x0000000000000000-0x000000007cfdbfff]
[    0.014581] NODE_DATA(0) allocated [mem 0x7cfb1000-0x7cfdbfff]
[    0.014987] Reserving 192MB of memory at 1792MB for crashkernel (System RAM: 1999MB)
[    0.027662] Zone ranges:
[    0.027668]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.027672]   DMA32    [mem 0x0000000001000000-0x000000007cfdbfff]
[    0.027675]   Normal   empty
[    0.027677]   Device   empty
[    0.027678] Movable zone start for each node
[    0.027682] Early memory node ranges
[    0.027683]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.027686]   node   0: [mem 0x0000000000100000-0x000000007cfdbfff]
[    0.027690] Initmem setup node 0 [mem 0x0000000000001000-0x000000007cfdbfff]
[    0.027697] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.027729] On node 0, zone DMA: 97 pages in unavailable ranges
[    0.031751] On node 0, zone DMA32: 12324 pages in unavailable ranges
[    0.032516] ACPI: PM-Timer IO Port: 0x608
[    0.032537] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[    0.032597] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.032602] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.032605] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.032607] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.032616] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.032618] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.032624] ACPI: Using ACPI (MADT) for SMP configuration information
[    0.032626] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.032633] TSC deadline timer available
[    0.032635] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.032674] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.032678] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.032680] PM: hibernation: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.032682] PM: hibernation: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.032684] [mem 0x7d000000-0xfeffbfff] available for PCI devices
[    0.032687] Booting paravirtualized kernel on KVM
[    0.032691] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns
[    0.041146] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
[    0.041573] percpu: Embedded 61 pages/cpu s212992 r8192 d28672 u2097152
[    0.041580] pcpu-alloc: s212992 r8192 d28672 u2097152 alloc=1*2097152
[    0.041584] pcpu-alloc: [0] 0 
[    0.041613] kvm-guest: PV spinlocks disabled, no host support
[    0.041619] Fallback order for Node 0: 0 
[    0.041623] Built 1 zonelists, mobility grouping on.  Total pages: 503708
[    0.041626] Policy zone: DMA32
[    0.041628] Kernel command line: BOOT_IMAGE=(hd0,gpt1)/boot/vmlinuz-5.17.2-300.fc36.x86_64 root=UUID=ce1b7344-16ae-42a9-a2d3-2c2aa35f2742 ro no_timer_check net.ifnames=0 console=tty1 console=ttyS0,115200n8 crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M
[    0.041750] Unknown kernel command line parameters "BOOT_IMAGE=(hd0,gpt1)/boot/vmlinuz-5.17.2-300.fc36.x86_64", will be passed to user space.
[    0.042048] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.042106] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.042160] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.048266] Memory: 1729940K/2047464K available (16393K kernel code, 3666K rwdata, 11180K rodata, 2708K init, 6188K bss, 317264K reserved, 0K cma-reserved)
[    0.048280] random: get_random_u64 called from __kmem_cache_create+0x2a/0x530 with crng_init=0
[    0.048426] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.048441] Kernel/User page tables isolation: enabled
[    0.048462] ftrace: allocating 49732 entries in 195 pages
[    0.064040] ftrace: allocated 195 pages with 4 groups
[    0.065946] Dynamic Preempt: voluntary
[    0.066019] rcu: Preemptible hierarchical RCU implementation.
[    0.066026] rcu: 	RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=1.
[    0.066030] 	Trampoline variant of Tasks RCU enabled.
[    0.066032] 	Rude variant of Tasks RCU enabled.
[    0.066034] 	Tracing variant of Tasks RCU enabled.
[    0.066039] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.
[    0.066043] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.075556] NR_IRQS: 524544, nr_irqs: 256, preallocated irqs: 16
[    0.076012] random: crng init done (trusting CPU's manufacturer)
[    0.120856] Console: colour VGA+ 80x25
[    0.191924] printk: console [tty1] enabled
[    0.329434] printk: console [ttyS0] enabled
[    0.331036] ACPI: Core revision 20211217
[    0.332691] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[    0.335863] APIC: Switch to symmetric I/O mode setup
[    0.338191] x2apic enabled
[    0.339797] Switched APIC routing to physical x2apic.
[    0.344332] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.346509] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x1e452ea631d, max_idle_ns: 440795244572 ns
[    0.350007] Calibrating delay loop (skipped) preset value.. 4199.99 BogoMIPS (lpj=2099998)
[    0.351004] pid_max: default: 32768 minimum: 301
[    0.352043] LSM: Security Framework initializing
[    0.353019] Yama: becoming mindful.
[    0.354010] SELinux:  Initializing.
[    0.355053] LSM support for eBPF active
[    0.356006] landlock: Up and running.
[    0.357032] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.358009] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.360038] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.361003] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[    0.362011] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.363005] Spectre V2 : Mitigation: Retpolines
[    0.364003] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.365010] Spectre V2 : Enabling Restricted Speculation for firmware calls
[    0.366015] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[    0.367010] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[    0.368013] TAA: Mitigation: Clear CPU buffers
[    0.369004] MDS: Mitigation: Clear CPU buffers
[    0.387499] Freeing SMP alternatives memory: 44K
[    0.388209] smpboot: CPU0: Intel Xeon Processor (Skylake, IBRS) (family: 0x6, model: 0x55, stepping: 0x4)
[    0.389261] cblist_init_generic: Setting adjustable number of callback queues.
[    0.390005] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.391061] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.392046] cblist_init_generic: Setting shift to 0 and lim to 1.
[    0.393049] Performance Events: unsupported p6 CPU model 85 no PMU driver, software events only.
[    0.394082] rcu: Hierarchical SRCU implementation.
[    0.395686] NMI watchdog: Perf NMI watchdog permanently disabled
[    0.396083] smp: Bringing up secondary CPUs ...
[    0.397010] smp: Brought up 1 node, 1 CPU
[    0.398006] smpboot: Max logical packages: 1
[    0.399007] smpboot: Total of 1 processors activated (4199.99 BogoMIPS)
[    0.400308] devtmpfs: initialized
[    0.401059] x86/mm: Memory block size: 128MB
[    0.402539] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[    0.403012] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[    0.404101] pinctrl core: initialized pinctrl subsystem
[    0.405248] PM: RTC time: 23:43:41, date: 2022-04-15
[    0.406176] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.407146] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations
[    0.408009] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.409019] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.410032] audit: initializing netlink subsys (disabled)
[    0.411213] thermal_sys: Registered thermal governor 'fair_share'
[    0.411215] thermal_sys: Registered thermal governor 'bang_bang'
[    0.412005] thermal_sys: Registered thermal governor 'step_wise'
[    0.413004] thermal_sys: Registered thermal governor 'user_space'
[    0.414035] audit: type=2000 audit(1650066221.259:1): state=initialized audit_enabled=0 res=1
[    0.416013] cpuidle: using governor menu
[    0.417110] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.418187] PCI: Using configuration type 1 for base access
[    0.420918] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[    0.421155] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.422033] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.423203] cryptd: max_cpu_qlen set to 1000
[    0.424209] raid6: skipped pq benchmark and selected avx512x4
[    0.425005] raid6: using avx512x2 recovery algorithm
[    0.426078] ACPI: Added _OSI(Module Device)
[    0.427010] ACPI: Added _OSI(Processor Device)
[    0.428014] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.429006] ACPI: Added _OSI(Processor Aggregator Device)
[    0.430007] ACPI: Added _OSI(Linux-Dell-Video)
[    0.431004] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.432010] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
[    0.433646] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.434752] ACPI: Interpreter enabled
[    0.435019] ACPI: PM: (supports S0 S5)
[    0.436007] ACPI: Using IOAPIC for interrupt routing
[    0.437028] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.438137] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.441650] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.442012] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI EDR HPX-Type3]
[    0.443005] acpi PNP0A03:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]
[    0.444018] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    0.445355] acpiphp: Slot [3] registered
[    0.446037] acpiphp: Slot [4] registered
[    0.447037] acpiphp: Slot [5] registered
[    0.448030] acpiphp: Slot [6] registered
[    0.449034] acpiphp: Slot [7] registered
[    0.450029] acpiphp: Slot [8] registered
[    0.451048] acpiphp: Slot [9] registered
[    0.452028] acpiphp: Slot [10] registered
[    0.453029] acpiphp: Slot [11] registered
[    0.454034] acpiphp: Slot [12] registered
[    0.455037] acpiphp: Slot [13] registered
[    0.456029] acpiphp: Slot [14] registered
[    0.457033] acpiphp: Slot [15] registered
[    0.458035] acpiphp: Slot [16] registered
[    0.459033] acpiphp: Slot [17] registered
[    0.460036] acpiphp: Slot [18] registered
[    0.461036] acpiphp: Slot [19] registered
[    0.462056] acpiphp: Slot [20] registered
[    0.463039] acpiphp: Slot [21] registered
[    0.464034] acpiphp: Slot [22] registered
[    0.465030] acpiphp: Slot [23] registered
[    0.466034] acpiphp: Slot [24] registered
[    0.467032] acpiphp: Slot [25] registered
[    0.468036] acpiphp: Slot [26] registered
[    0.469033] acpiphp: Slot [27] registered
[    0.470028] acpiphp: Slot [28] registered
[    0.471034] acpiphp: Slot [29] registered
[    0.472034] acpiphp: Slot [30] registered
[    0.473026] acpiphp: Slot [31] registered
[    0.474023] PCI host bridge to bus 0000:00
[    0.475007] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
[    0.476007] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.477008] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.478006] pci_bus 0000:00: root bus resource [mem 0x7d000000-0xfebfffff window]
[    0.479007] pci_bus 0000:00: root bus resource [mem 0x100000000-0x17fffffff window]
[    0.480009] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.481090] pci 0000:00:00.0: [8086:1237] type 00 class 0x060000
[    0.482674] pci 0000:00:01.0: [8086:7000] type 00 class 0x060100
[    0.483909] pci 0000:00:01.1: [8086:7010] type 00 class 0x010180
[    0.489008] pci 0000:00:01.1: reg 0x20: [io  0xc0c0-0xc0cf]
[    0.491526] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
[    0.492006] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
[    0.493007] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
[    0.494006] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
[    0.495308] pci 0000:00:01.3: [8086:7113] type 00 class 0x068000
[    0.496815] pci 0000:00:01.3: quirk: [io  0x0600-0x063f] claimed by PIIX4 ACPI
[    0.497022] pci 0000:00:01.3: quirk: [io  0x0700-0x070f] claimed by PIIX4 SMB
[    0.498414] pci 0000:00:02.0: [1234:1111] type 00 class 0x030000
[    0.500814] pci 0000:00:02.0: reg 0x10: [mem 0xfd000000-0xfdffffff pref]
[    0.504189] pci 0000:00:02.0: reg 0x18: [mem 0xfeb94000-0xfeb94fff]
[    0.514088] pci 0000:00:02.0: reg 0x30: [mem 0xfeb80000-0xfeb8ffff pref]
[    0.515214] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[    0.517304] pci 0000:00:03.0: [1af4:1000] type 00 class 0x020000
[    0.520009] pci 0000:00:03.0: reg 0x10: [io  0xc080-0xc09f]
[    0.523010] pci 0000:00:03.0: reg 0x14: [mem 0xfeb95000-0xfeb95fff]
[    0.531015] pci 0000:00:03.0: reg 0x20: [mem 0xfe000000-0xfe003fff 64bit pref]
[    0.535012] pci 0000:00:03.0: reg 0x30: [mem 0xfeb00000-0xfeb7ffff pref]
[    0.538153] pci 0000:00:04.0: [1af4:1004] type 00 class 0x010000
[    0.541010] pci 0000:00:04.0: reg 0x10: [io  0xc000-0xc03f]
[    0.544010] pci 0000:00:04.0: reg 0x14: [mem 0xfeb96000-0xfeb96fff]
[    0.551009] pci 0000:00:04.0: reg 0x20: [mem 0xfe004000-0xfe007fff 64bit pref]
[    0.557789] pci 0000:00:05.0: [1af4:1002] type 00 class 0x00ff00
[    0.561014] pci 0000:00:05.0: reg 0x10: [io  0xc0a0-0xc0bf]
[    0.571013] pci 0000:00:05.0: reg 0x20: [mem 0xfe008000-0xfe00bfff 64bit pref]
[    0.576460] pci 0000:00:06.0: [1af4:1003] type 00 class 0x078000
[    0.579009] pci 0000:00:06.0: reg 0x10: [io  0xc040-0xc07f]
[    0.582008] pci 0000:00:06.0: reg 0x14: [mem 0xfeb97000-0xfeb97fff]
[    0.587008] pci 0000:00:06.0: reg 0x20: [mem 0xfe00c000-0xfe00ffff 64bit pref]
[    0.590924] pci 0000:00:07.0: [1b36:000d] type 00 class 0x0c0330
[    0.591736] pci 0000:00:07.0: reg 0x10: [mem 0xfeb90000-0xfeb93fff 64bit]
[    0.596754] ACPI: PCI: Interrupt link LNKA configured for IRQ 10
[    0.597159] ACPI: PCI: Interrupt link LNKB configured for IRQ 10
[    0.598152] ACPI: PCI: Interrupt link LNKC configured for IRQ 11
[    0.599121] ACPI: PCI: Interrupt link LNKD configured for IRQ 11
[    0.600075] ACPI: PCI: Interrupt link LNKS configured for IRQ 9
[    0.601584] iommu: Default domain type: Translated 
[    0.602005] iommu: DMA domain TLB invalidation policy: lazy mode 
[    0.603132] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.604000] pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[    0.604009] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.605006] vgaarb: loaded
[    0.606130] SCSI subsystem initialized
[    0.607093] libata version 3.00 loaded.
[    0.607156] ACPI: bus type USB registered
[    0.608060] usbcore: registered new interface driver usbfs
[    0.609031] usbcore: registered new interface driver hub
[    0.610022] usbcore: registered new device driver usb
[    0.611089] pps_core: LinuxPPS API ver. 1 registered
[    0.612005] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.613011] PTP clock support registered
[    0.614101] EDAC MC: Ver: 3.0.0
[    0.615670] NetLabel: Initializing
[    0.616006] NetLabel:  domain hash size = 128
[    0.617004] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    0.618032] NetLabel:  unlabeled traffic allowed by default
[    0.619020] mctp: management component transport protocol core
[    0.620005] NET: Registered PF_MCTP protocol family
[    0.621024] PCI: Using ACPI for IRQ routing
[    0.622011] PCI: pci_cache_line_size set to 64 bytes
[    0.622288] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[    0.622294] e820: reserve RAM buffer [mem 0x7cfdc000-0x7fffffff]
[    0.622502] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.623004] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[    0.626096] clocksource: Switched to clocksource kvm-clock
[    0.644375] VFS: Disk quotas dquot_6.6.0
[    0.645537] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.647535] pnp: PnP ACPI init
[    0.648593] pnp 00:03: [dma 2]
[    0.648905] pnp: PnP ACPI: found 5 devices
[    0.664441] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.671674] NET: Registered PF_INET protocol family
[    0.673086] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.676398] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.678574] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.680592] TCP bind hash table entries: 16384 (order: 6, 262144 bytes, linear)
[    0.682501] TCP: Hash tables configured (established 16384 bind 16384)
[    0.684255] MPTCP token hash table entries: 2048 (order: 3, 49152 bytes, linear)
[    0.686221] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.688342] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.690524] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.692051] NET: Registered PF_XDP protocol family
[    0.693324] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
[    0.695059] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
[    0.713906] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
[    0.715867] pci_bus 0000:00: resource 7 [mem 0x7d000000-0xfebfffff window]
[    0.717697] pci_bus 0000:00: resource 8 [mem 0x100000000-0x17fffffff window]
[    0.719766] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.721413] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.723122] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.751018] ACPI: \_SB_.LNKC: Enabled at IRQ 11
[    0.778911] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0x6e0 took 52701 usecs
[    0.780934] PCI: CLS 0 bytes, default 64
[    0.782122] Trying to unpack rootfs image as initramfs...
[    0.791404] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1e452ea631d, max_idle_ns: 440795244572 ns
[    0.794753] Initialise system trusted keyrings
[    0.796028] Key type blacklist registered
[    0.798351] workingset: timestamp_bits=36 max_order=19 bucket_order=0
[    0.806690] zbud: loaded
[    0.808697] integrity: Platform Keyring initialized
[    0.833145] NET: Registered PF_ALG protocol family
[    0.837166] xor: automatically using best checksumming function   avx       
[    0.842313] Key type asymmetric registered
[    0.846047] Asymmetric key parser 'x509' registered
[    1.273658] Freeing initrd memory: 34480K
[    1.281735] alg: self-tests for CTR-KDF (hmac(sha256)) passed
[    1.283467] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 244)
[    1.285817] io scheduler mq-deadline registered
[    1.287122] io scheduler kyber registered
[    1.288412] io scheduler bfq registered
[    1.291238] atomic64_test: passed for x86-64 platform with CX8 and with SSE
[    1.293756] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    1.295973] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    1.298514] ACPI: button: Power Button [PWRF]
[    1.363941] ACPI: \_SB_.LNKD: Enabled at IRQ 10
[    1.391879] ACPI: \_SB_.LNKA: Enabled at IRQ 10
[    1.421221] ACPI: \_SB_.LNKB: Enabled at IRQ 11
[    1.424688] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
[    1.427183] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    1.432477] Non-volatile memory driver v1.3
[    1.434026] Linux agpgart interface v0.103
[    1.435671] ACPI: bus type drm_connector registered
[    1.437858] ata_piix 0000:00:01.1: version 2.13
[    1.439475] scsi host0: ata_piix
[    1.441492] scsi host1: ata_piix
[    1.442788] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc0c0 irq 14
[    1.444964] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc0c8 irq 15
[    1.448575] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.450677] ehci-pci: EHCI PCI platform driver
[    1.452572] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.454763] ohci-pci: OHCI PCI platform driver
[    1.456369] uhci_hcd: USB Universal Host Controller Interface driver
[    1.501248] xhci_hcd 0000:00:07.0: xHCI Host Controller
[    1.502792] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 1
[    1.505745] xhci_hcd 0000:00:07.0: hcc params 0x00087001 hci version 0x100 quirks 0x0000000000000010
[    1.510044] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.17
[    1.512743] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.515156] usb usb1: Product: xHCI Host Controller
[    1.516788] usb usb1: Manufacturer: Linux 5.17.2-300.fc36.x86_64 xhci-hcd
[    1.519025] usb usb1: SerialNumber: 0000:00:07.0
[    1.520830] hub 1-0:1.0: USB hub found
[    1.522241] hub 1-0:1.0: 4 ports detected
[    1.524145] xhci_hcd 0000:00:07.0: xHCI Host Controller
[    1.526099] xhci_hcd 0000:00:07.0: new USB bus registered, assigned bus number 2
[    1.528580] xhci_hcd 0000:00:07.0: Host supports USB 3.0 SuperSpeed
[    1.530785] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    1.533491] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.17
[    1.536272] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    1.538683] usb usb2: Product: xHCI Host Controller
[    1.540333] usb usb2: Manufacturer: Linux 5.17.2-300.fc36.x86_64 xhci-hcd
[    1.542467] usb usb2: SerialNumber: 0000:00:07.0
[    1.544525] hub 2-0:1.0: USB hub found
[    1.546080] hub 2-0:1.0: 4 ports detected
[    1.547992] usbcore: registered new interface driver usbserial_generic
[    1.550091] usbserial: USB Serial support registered for generic
[    1.552196] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    1.556046] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.557810] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.559773] mousedev: PS/2 mouse device common for all mice
[    1.562173] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    1.565646] rtc_cmos 00:00: RTC can wake from S4
[    1.568897] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input4
[    1.571714] rtc_cmos 00:00: registered as rtc0
[    1.573484] rtc_cmos 00:00: setting system clock to 2022-04-15T23:43:42 UTC (1650066222)
[    1.576172] rtc_cmos 00:00: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[    1.578672] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
[    1.582869] input: VirtualPS/2 VMware VMMouse as /devices/platform/i8042/serio1/input/input3
[    1.585987] device-mapper: uevent: version 1.0.3
[    1.587982] device-mapper: ioctl: 4.45.0-ioctl (2021-03-22) initialised: dm-devel@redhat.com
[    1.590938] intel_pstate: CPU model not supported
[    1.592708] hid: raw HID events driver (C) Jiri Kosina
[    1.594512] usbcore: registered new interface driver usbhid
[    1.596338] usbhid: USB HID core driver
[    1.597995] drop_monitor: Initializing network drop monitor service
[    1.600283] ata2: found unknown device (class 0)
[    1.602979] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
[    1.606451] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
[    1.610776] Initializing XFRM netlink socket
[    1.612487] NET: Registered PF_INET6 protocol family
[    1.621089] Segment Routing with IPv6
[    1.622451] RPL Segment Routing with IPv6
[    1.624724] sr 1:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[    1.626869] cdrom: Uniform CD-ROM driver Revision: 3.20
[    1.628975] In-situ OAM (IOAM) with IPv6
[    1.630484] mip6: Mobile IPv6
[    1.631670] NET: Registered PF_PACKET protocol family
[    1.633732] No MBM correction factor available
[    1.635282] IPI shorthand broadcast: enabled
[    1.636793] AVX2 version of gcm_enc/dec engaged.
[    1.638642] AES CTR mode by8 optimization enabled
[    1.640685] sched_clock: Marking stable (1372262559, 268038073)->(1867166328, -226865696)
[    1.643591] registered taskstats version 1
[    1.646030] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    1.646520] sr 1:0:0:0: Attached scsi generic sg0 type 5
[    1.649050] Loading compiled-in X.509 certificates
[    1.707024] Loaded X.509 cert 'Fedora kernel signing key: d22d9abf73d098d6d0e0a8f3725deb417ec7219b'
[    1.709503] zswap: loaded using pool lzo/zbud
[    1.710871] page_owner is disabled
[    1.712304] Key type ._fscrypt registered
[    1.713691] Key type .fscrypt registered
[    1.714775] Key type fscrypt-provisioning registered
[    1.716584] Btrfs loaded, crc32c=crc32c-generic, zoned=yes, fsverity=yes
[    1.718399] Key type big_key registered
[    1.724431] Key type encrypted registered
[    1.725613] ima: No TPM chip found, activating TPM-bypass!
[    1.727053] Loading compiled-in module X.509 certificates
[    1.729377] Loaded X.509 cert 'Fedora kernel signing key: d22d9abf73d098d6d0e0a8f3725deb417ec7219b'
[    1.732051] ima: Allocated hash algorithm: sha256
[    1.733399] ima: No architecture policies found
[    1.734835] evm: Initialising EVM extended attributes:
[    1.736408] evm: security.selinux
[    1.737390] evm: security.SMACK64 (disabled)
[    1.738585] evm: security.SMACK64EXEC (disabled)
[    1.740029] evm: security.SMACK64TRANSMUTE (disabled)
[    1.741566] evm: security.SMACK64MMAP (disabled)
[    1.742812] evm: security.apparmor (disabled)
[    1.743974] evm: security.ima
[    1.744989] evm: security.capability
[    1.746225] evm: HMAC attrs: 0x1
[    1.762032] usb 1-1: new high-speed USB device number 2 using xhci_hcd
[    1.787929] alg: No test for 842 (842-scomp)
[    1.789265] alg: No test for 842 (842-generic)
[    1.891747] usb 1-1: New USB device found, idVendor=0627, idProduct=0001, bcdDevice= 0.00
[    1.894055] usb 1-1: New USB device strings: Mfr=1, Product=3, SerialNumber=10
[    1.896092] usb 1-1: Product: QEMU USB Tablet
[    1.897245] usb 1-1: Manufacturer: QEMU
[    1.898290] usb 1-1: SerialNumber: 28754-0000:00:07.0-1
[    1.906595] input: QEMU QEMU USB Tablet as /devices/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input5
[    1.914143] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-1/input0
[    2.005733] PM:   Magic number: 2:241:757
[    2.007286] tty ttyS8: hash matches
[    2.008797] RAS: Correctable Errors collector initialized.
[    2.013782] Freeing unused decrypted memory: 2036K
[    2.016729] Freeing unused kernel image (initmem) memory: 2708K
[    2.018824] Write protecting the kernel read-only data: 30720k
[    2.023513] Freeing unused kernel image (text/rodata gap) memory: 2036K
[    2.026190] Freeing unused kernel image (rodata/data gap) memory: 1108K
[    2.142713] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    2.145486] rodata_test: all tests were successful
[    2.147381] x86/mm: Checking user space page tables
[    2.227439] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    2.229062] Run /init as init process
[    2.230091]   with arguments:
[    2.230093]     /init
[    2.230094]   with environment:
[    2.230095]     HOME=/
[    2.230096]     TERM=linux
[    2.230096]     BOOT_IMAGE=(hd0,gpt1)/boot/vmlinuz-5.17.2-300.fc36.x86_64
[    2.242275] systemd[1]: systemd v250.3-8.fc36 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    2.249988] systemd[1]: Detected virtualization kvm.
[    2.251455] systemd[1]: Detected architecture x86-64.
[    2.252931] systemd[1]: Running in initial RAM disk.
[    2.255968] systemd[1]: No hostname configured, using default hostname.
[    2.257833] systemd[1]: Hostname set to <fedora>.
[    2.259248] systemd[1]: Initializing machine ID from VM UUID.
[    2.290446] systemd[1]: Failed to open libbpf, LSM BPF is not supported: Operation not supported
[    2.402550] systemd[1]: Queued start job for default target initrd.target.
[    2.404593] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[    2.408506] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[    2.411713] systemd[1]: Reached target initrd-usr-fs.target - Initrd /usr File System.
[    2.415127] systemd[1]: Reached target local-fs.target - Local File Systems.
[    2.418206] systemd[1]: Reached target paths.target - Path Units.
[    2.420914] systemd[1]: Reached target slices.target - Slice Units.
[    2.423673] systemd[1]: Reached target swap.target - Swaps.
[    2.426322] systemd[1]: Reached target timers.target - Timer Units.
[    2.429521] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[    2.433456] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[    2.437135] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[    2.440249] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[    2.443624] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[    2.447255] systemd[1]: Reached target sockets.target - Socket Units.
[    2.451942] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[    2.455498] systemd[1]: memstrack.service - Memstrack Anylazing Service was skipped because all trigger condition checks failed.
[    2.463067] systemd[1]: Starting systemd-journald.service - Journal Service...
[    2.466564] systemd[1]: systemd-modules-load.service - Load Kernel Modules was skipped because all trigger condition checks failed.
[    2.479232] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[    2.491698] systemd[1]: Starting systemd-vconsole-setup.service - Setup Virtual Console...
[    2.507137] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[    2.518148] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[    2.525364] audit: type=1130 audit(1650066223.451:2): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    2.533436] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[    2.676632] systemd[1]: Started systemd-journald.service - Journal Service.
[    2.679762] audit: type=1130 audit(1650066223.605:3): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-journald comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    2.697179] audit: type=1130 audit(1650066223.614:4): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup-dev comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    2.719159] audit: type=1130 audit(1650066223.645:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    2.928046] audit: type=1130 audit(1650066223.853:6): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-vconsole-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.067041] audit: type=1130 audit(1650066223.992:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-cmdline comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.111035] audit: type=1130 audit(1650066224.036:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-udev comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    3.123035] audit: type=1334 audit(1650066224.036:9): prog-id=6 op=LOAD
[    3.129024] audit: type=1334 audit(1650066224.036:10): prog-id=7 op=LOAD
[    3.498920] scsi host2: Virtio SCSI HBA
[    3.515181] scsi 2:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[    3.547382] sd 2:0:0:0: Power-on or device reset occurred
[    3.549397] sd 2:0:0:0: Attached scsi generic sg1 type 0
[    3.551499] sd 2:0:0:0: [sda] 40001536 512-byte logical blocks: (20.5 GB/19.1 GiB)
[    3.555060] sd 2:0:0:0: [sda] Write Protect is off
[    3.556347] sd 2:0:0:0: [sda] Mode Sense: 63 00 00 08
[    3.561161] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.576598]  sda: sda1 sda14 sda15
[    3.581315] sd 2:0:0:0: [sda] Attached SCSI disk
[    4.199166] EXT4-fs (sda1): mounted filesystem with ordered data mode. Quota mode: none.
[    4.659432] systemd-journald[217]: Received SIGTERM from PID 1 (systemd).
[    4.879336] SELinux:  policy capability network_peer_controls=1
[    4.881374] SELinux:  policy capability open_perms=1
[    4.883130] SELinux:  policy capability extended_socket_class=1
[    4.885060] SELinux:  policy capability always_check_network=0
[    4.887016] SELinux:  policy capability cgroup_seclabel=1
[    4.888850] SELinux:  policy capability nnp_nosuid_transition=1
[    4.890827] SELinux:  policy capability genfs_seclabel_symlinks=1
[    4.892853] SELinux:  policy capability ioctl_skip_cloexec=0
[    4.945305] systemd[1]: Successfully loaded SELinux policy in 142.807ms.
[    5.014691] systemd[1]: Relabelled /dev, /dev/shm, /run, /sys/fs/cgroup in 44.262ms.
[    5.023449] systemd[1]: systemd v250.3-8.fc36 running in system mode (+PAM +AUDIT +SELINUX -APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN -IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    5.031407] systemd[1]: Detected virtualization kvm.
[    5.033201] systemd[1]: Detected architecture x86-64.
[    5.039276] systemd[1]: Hostname set to <gw2.fflev.de>.
[    5.374130] systemd[1]: LSM BPF program attached
[    6.137161] kauditd_printk_skb: 76 callbacks suppressed
[    6.137166] audit: type=1334 audit(1650066227.063:87): prog-id=34 op=LOAD
[    6.145508] audit: type=1334 audit(1650066227.070:88): prog-id=0 op=UNLOAD
[    6.147583] audit: type=1334 audit(1650066227.070:89): prog-id=35 op=LOAD
[    6.151897] systemd[1]: initrd-switch-root.service: Deactivated successfully.
[    6.154168] audit: type=1334 audit(1650066227.070:90): prog-id=36 op=LOAD
[    6.156710] audit: type=1334 audit(1650066227.070:91): prog-id=0 op=UNLOAD
[    6.158814] audit: type=1334 audit(1650066227.070:92): prog-id=0 op=UNLOAD
[    6.160898] audit: type=1131 audit(1650066227.076:93): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-journald comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    6.166768] systemd[1]: Stopped initrd-switch-root.service - Switch Root.
[    6.170589] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
[    6.173050] audit: type=1334 audit(1650066227.082:94): prog-id=0 op=UNLOAD
[    6.174899] audit: type=1130 audit(1650066227.096:95): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=initrd-switch-root comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    6.180927] systemd[1]: Created slice system-fastd.slice - Slice /system/fastd.
[    6.184978] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[    6.187129] audit: type=1131 audit(1650066227.096:96): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=initrd-switch-root comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[    6.194871] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[    6.199084] systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty.
[    6.203458] systemd[1]: Created slice system-sshd\x2dkeygen.slice - Slice /system/sshd-keygen.
[    6.207378] systemd[1]: Created slice system-systemd\x2dfsck.slice - Slice /system/systemd-fsck.
[    6.211743] systemd[1]: Created slice system-tunneldigger\x2dbroker.slice - Slice /system/tunneldigger-broker.
[    6.216373] systemd[1]: Created slice user.slice - User and Session Slice.
[    6.219596] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[    6.223789] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[    6.228641] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[    6.233233] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[    6.236721] systemd[1]: Stopped target initrd-switch-root.target - Switch Root.
[    6.240041] systemd[1]: Stopped target initrd-fs.target - Initrd File Systems.
[    6.243317] systemd[1]: Stopped target initrd-root-fs.target - Initrd Root File System.
[    6.246860] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[    6.250712] systemd[1]: Reached target paths.target - Path Units.
[    6.253502] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[    6.256528] systemd[1]: Reached target slices.target - Slice Units.
[    6.259238] systemd[1]: Reached target swap.target - Swaps.
[    6.261711] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[    6.286786] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[    6.291064] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[    6.295615] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[    6.300877] systemd[1]: Listening on systemd-oomd.socket - Userspace Out-Of-Memory (OOM) Killer Socket.
[    6.305502] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[    6.309245] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[    6.313390] systemd[1]: Listening on systemd-userdbd.socket - User Database Manager Socket.
[    6.318872] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    6.325830] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[    6.338777] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[    6.348444] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[    6.356657] systemd[1]: Mounting tmp.mount - Temporary Directory /tmp...
[    6.367250] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[    6.378028] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[    6.386674] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[    6.410783] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[    6.422051] systemd[1]: Stopped systemd-journald.service - Journal Service.
[    6.435571] systemd[1]: Starting systemd-journald.service - Journal Service...
[    6.454331] systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules...
[    6.479254] systemd[1]: Starting systemd-network-generator.service - Generate network units from Kernel command line...
[    6.504394] fuse: init (API version 7.36)
[    6.520511] systemd[1]: Starting systemd-remount-fs.service - Remount Root and Kernel File Systems...
[    6.563354] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because all trigger condition checks failed.
[    6.584616] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[    6.589045] EXT4-fs (sda1): re-mounted. Quota mode: none.
[    6.609980] systemd[1]: Started systemd-journald.service - Journal Service.
[    6.775560] systemd-journald[476]: Received client request to flush runtime journal.
[    6.799414] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    7.297604] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    7.304472] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    7.314140] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[    7.336040] cfg80211: failed to load regulatory.db
[    7.465068] batman_adv: B.A.T.M.A.N. advanced 2022.0 (compatibility version 15) loaded
[    7.620480] l2tp_core: L2TP core driver, V2.0
[    7.648442] l2tp_netlink: L2TP netlink interface
[    7.681063] l2tp_eth: L2TP ethernet pseudowire support (L2TPv3)
[    8.409250] piix4_smbus 0000:00:01.3: SMBus Host Controller at 0x700, revision 0
[    8.680496] input: PC Speaker as /devices/platform/pcspkr/input/input6
[    8.998082] bochs-drm 0000:00:02.0: vgaarb: deactivate vga console
[    9.753339] Console: switching to colour dummy device 80x25
[    9.857016] [drm] Found bochs VGA, ID 0xb0c0.
[    9.857797] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfeb94000.
[    9.902634] [drm] Found EDID data blob.
[    9.930153] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:02.0 on minor 0
[    9.988224] fbcon: bochs-drmdrmfb (fb0) is primary device
[   10.003905] Error: Driver 'pcspkr' is already registered, aborting...
[   10.056009] Console: switching to colour frame buffer device 128x48
[   10.086954] bochs-drm 0000:00:02.0: [drm] fb0: bochs-drmdrmfb frame buffer device
[   10.666378] gre: GRE over IPv4 demultiplexor driver
[   10.696852] ip_gre: GRE over IPv4 tunneling driver
[   10.796363] ff-lev: port 1(mesh-lev) entered blocking state
[   10.797356] ff-lev: port 1(mesh-lev) entered disabled state
[   10.808110] device mesh-lev entered promiscuous mode
[   10.825256] batman_adv: mesh-lev: Adding interface: tun-lev
[   10.826339] batman_adv: mesh-lev: The MTU of interface tun-lev is too small (1500) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1560 would solve the problem.
[   10.830642] batman_adv: mesh-lev: Not using interface tun-lev (retrying later): interface not active
[   10.861475] batman_adv: mesh-lev: Adding interface: bb-lev-4
[   10.862574] batman_adv: mesh-lev: The MTU of interface bb-lev-4 is too small (1458) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1560 would solve the problem.
[   10.866767] batman_adv: mesh-lev: Not using interface bb-lev-4 (retrying later): interface not active
[   10.888918] batman_adv: mesh-lev: Adding interface: bb-lev-3
[   10.889951] batman_adv: mesh-lev: The MTU of interface bb-lev-3 is too small (1458) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1560 would solve the problem.
[   10.894041] batman_adv: mesh-lev: Not using interface bb-lev-3 (retrying later): interface not active
[   10.919923] batman_adv: mesh-lev: Adding interface: bb-lev-1
[   10.922226] batman_adv: mesh-lev: The MTU of interface bb-lev-1 is too small (1458) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1560 would solve the problem.
[   10.929645] batman_adv: mesh-lev: Not using interface bb-lev-1 (retrying later): interface not active
[   10.976166] batman_adv: mesh-lev: Interface activated: tun-lev
[   10.987089] batman_adv: mesh-lev: Interface activated: bb-lev-4
[   10.994533] batman_adv: mesh-lev: Interface activated: bb-lev-3
[   11.002941] batman_adv: mesh-lev: Interface activated: bb-lev-1
[   11.041494] ff-lev: port 1(mesh-lev) entered blocking state
[   11.043284] ff-lev: port 1(mesh-lev) entered forwarding state
[   11.045709] IPv6: ADDRCONF(NETDEV_CHANGE): ff-lev: link becomes ready
[   11.249011] batman_adv: mesh-lev: No IGMP Querier present - multicast optimizations disabled
[   11.250923] batman_adv: mesh-lev: No MLD Querier present - multicast optimizations disabled
[   11.549601] tun-lev: port 1(lev-dc503bc1507) entered blocking state
[   11.552676] tun-lev: port 1(lev-dc503bc1507) entered disabled state
[   11.590892] device lev-dc503bc1507 entered promiscuous mode
[   11.611089] tun-lev: port 1(lev-dc503bc1507) entered blocking state
[   11.614271] tun-lev: port 1(lev-dc503bc1507) entered forwarding state
[   11.618183] IPv6: ADDRCONF(NETDEV_CHANGE): tun-lev: link becomes ready
[   11.770083] tun-lev: port 2(lev-87f0bb95b21) entered blocking state
[   11.772185] tun-lev: port 2(lev-87f0bb95b21) entered disabled state
[   11.794875] device lev-87f0bb95b21 entered promiscuous mode
[   11.806475] tun-lev: port 2(lev-87f0bb95b21) entered blocking state
[   11.808747] tun-lev: port 2(lev-87f0bb95b21) entered forwarding state
[   12.023296] tun-lev: port 3(lev-d51423b1405) entered blocking state
[   12.029689] tun-lev: port 3(lev-d51423b1405) entered disabled state
[   12.055327] device lev-d51423b1405 entered promiscuous mode
[   12.082940] tun-lev: port 3(lev-d51423b1405) entered blocking state
[   12.085163] tun-lev: port 3(lev-d51423b1405) entered forwarding state
[   12.118010] tun-lev: port 4(lev-c9f214a30ff) entered blocking state
[   12.120668] tun-lev: port 4(lev-c9f214a30ff) entered disabled state
[   12.142749] device lev-c9f214a30ff entered promiscuous mode
[   12.197727] tun-lev: port 5(lev-e89ad3f21f7) entered blocking state
[   12.200843] tun-lev: port 5(lev-e89ad3f21f7) entered disabled state
[   12.244217] device lev-e89ad3f21f7 entered promiscuous mode
[   12.254722] tun-lev: port 4(lev-c9f214a30ff) entered blocking state
[   12.256678] tun-lev: port 4(lev-c9f214a30ff) entered forwarding state
[   12.267743] tun: Universal TUN/TAP device driver, 1.6
[   12.286508] tun-lev: port 5(lev-e89ad3f21f7) entered blocking state
[   12.288543] tun-lev: port 5(lev-e89ad3f21f7) entered forwarding state
[   12.346697] tun-lev: port 6(lev-204d90d2ce4) entered blocking state
[   12.349025] tun-lev: port 6(lev-204d90d2ce4) entered disabled state
[   12.368939] device lev-204d90d2ce4 entered promiscuous mode
[   12.385501] tun-lev: port 7(lev-7bc04a73000) entered blocking state
[   12.387580] tun-lev: port 7(lev-7bc04a73000) entered disabled state
[   12.409965] device lev-7bc04a73000 entered promiscuous mode
[   12.425951] tun-lev: port 8(lev-09fa53c178b) entered blocking state
[   12.427875] tun-lev: port 8(lev-09fa53c178b) entered disabled state
[   12.444054] device lev-09fa53c178b entered promiscuous mode
[   12.483704] tun-lev: port 6(lev-204d90d2ce4) entered blocking state
[   12.486276] tun-lev: port 6(lev-204d90d2ce4) entered forwarding state
[   12.515115] tun-lev: port 7(lev-7bc04a73000) entered blocking state
[   12.517554] tun-lev: port 7(lev-7bc04a73000) entered forwarding state
[   12.545149] tun-lev: port 8(lev-09fa53c178b) entered blocking state
[   12.551743] tun-lev: port 8(lev-09fa53c178b) entered forwarding state
[   12.626088] tun-lev: port 9(lev-f5506073194) entered blocking state
[   12.627934] tun-lev: port 9(lev-f5506073194) entered disabled state
[   12.646982] device lev-f5506073194 entered promiscuous mode
[   12.658549] tun-lev: port 9(lev-f5506073194) entered blocking state
[   12.660236] tun-lev: port 9(lev-f5506073194) entered forwarding state
[   12.758974] tun-lev: port 10(lev-7381b272fc7) entered blocking state
[   12.760728] tun-lev: port 10(lev-7381b272fc7) entered disabled state
[   12.776924] device lev-7381b272fc7 entered promiscuous mode
[   12.784604] tun-lev: port 10(lev-7381b272fc7) entered blocking state
[   12.786574] tun-lev: port 10(lev-7381b272fc7) entered forwarding state
[   12.934062] tun-lev: port 11(lev-eb3f6743d79) entered blocking state
[   12.936518] tun-lev: port 11(lev-eb3f6743d79) entered disabled state
[   12.960863] device lev-eb3f6743d79 entered promiscuous mode
[   12.974643] tun-lev: port 11(lev-eb3f6743d79) entered blocking state
[   12.977174] tun-lev: port 11(lev-eb3f6743d79) entered forwarding state
[   13.007021] tun-lev: port 12(lev-f41ca3dccc3) entered blocking state
[   13.008887] tun-lev: port 12(lev-f41ca3dccc3) entered disabled state
[   13.022911] device lev-f41ca3dccc3 entered promiscuous mode
[   13.031437] tun-lev: port 12(lev-f41ca3dccc3) entered blocking state
[   13.033259] tun-lev: port 12(lev-f41ca3dccc3) entered forwarding state
[   13.099954] tun-lev: port 13(lev-0dd3cb49be3) entered blocking state
[   13.101770] tun-lev: port 13(lev-0dd3cb49be3) entered disabled state
[   13.119328] device lev-0dd3cb49be3 entered promiscuous mode
[   13.151378] tun-lev: port 14(lev-40f323bebf5) entered blocking state
[   13.153187] tun-lev: port 14(lev-40f323bebf5) entered disabled state
[   13.172890] device lev-40f323bebf5 entered promiscuous mode
[   13.209854] tun-lev: port 13(lev-0dd3cb49be3) entered blocking state
[   13.211793] tun-lev: port 13(lev-0dd3cb49be3) entered forwarding state
[   13.239164] tun-lev: port 14(lev-40f323bebf5) entered blocking state
[   13.240507] tun-lev: port 14(lev-40f323bebf5) entered forwarding state
[   13.263971] tun-lev: port 15(lev-0f904de0ddc) entered blocking state
[   13.265740] tun-lev: port 15(lev-0f904de0ddc) entered disabled state
[   13.276861] device lev-0f904de0ddc entered promiscuous mode
[   13.289090] tun-lev: port 16(lev-ec525bb7020) entered blocking state
[   13.290526] tun-lev: port 16(lev-ec525bb7020) entered disabled state
[   13.302012] device lev-ec525bb7020 entered promiscuous mode
[   13.316962] tun-lev: port 15(lev-0f904de0ddc) entered blocking state
[   13.318473] tun-lev: port 15(lev-0f904de0ddc) entered forwarding state
[   13.330935] tun-lev: port 16(lev-ec525bb7020) entered blocking state
[   13.332193] tun-lev: port 16(lev-ec525bb7020) entered forwarding state
[   13.471425] tun-lev: port 17(lev-0d4a551f3d9) entered blocking state
[   13.472821] tun-lev: port 17(lev-0d4a551f3d9) entered disabled state
[   13.485864] device lev-0d4a551f3d9 entered promiscuous mode
[   13.492041] tun-lev: port 17(lev-0d4a551f3d9) entered blocking state
[   13.493293] tun-lev: port 17(lev-0d4a551f3d9) entered forwarding state
[   13.510903] tun-lev: port 18(lev-58bcadf807d) entered blocking state
[   13.512183] tun-lev: port 18(lev-58bcadf807d) entered disabled state
[   13.523866] device lev-58bcadf807d entered promiscuous mode
[   13.530386] tun-lev: port 18(lev-58bcadf807d) entered blocking state
[   13.531840] tun-lev: port 18(lev-58bcadf807d) entered forwarding state
[   13.558445] tun-lev: port 19(lev-f224831dd47) entered blocking state
[   13.559798] tun-lev: port 19(lev-f224831dd47) entered disabled state
[   13.566835] device lev-f224831dd47 entered promiscuous mode
[   13.573442] tun-lev: port 19(lev-f224831dd47) entered blocking state
[   13.574788] tun-lev: port 19(lev-f224831dd47) entered forwarding state
[   13.600870] tun-lev: port 20(lev-1a9b4e1215c) entered blocking state
[   13.602418] tun-lev: port 20(lev-1a9b4e1215c) entered disabled state
[   13.610378] device lev-1a9b4e1215c entered promiscuous mode
[   13.618278] tun-lev: port 20(lev-1a9b4e1215c) entered blocking state
[   13.619570] tun-lev: port 20(lev-1a9b4e1215c) entered forwarding state
[   13.639183] tun-lev: port 21(lev-329f2638304) entered blocking state
[   13.640461] tun-lev: port 21(lev-329f2638304) entered disabled state
[   13.651886] device lev-329f2638304 entered promiscuous mode
[   13.659310] tun-lev: port 21(lev-329f2638304) entered blocking state
[   13.660638] tun-lev: port 21(lev-329f2638304) entered forwarding state
[   13.686924] tun-lev: port 22(lev-7cd3791e7f1) entered blocking state
[   13.688299] tun-lev: port 22(lev-7cd3791e7f1) entered disabled state
[   13.701864] device lev-7cd3791e7f1 entered promiscuous mode
[   13.778468] tun-lev: port 23(lev-8f24d236b50) entered blocking state
[   13.780641] tun-lev: port 23(lev-8f24d236b50) entered disabled state
[   13.802920] device lev-8f24d236b50 entered promiscuous mode
[   13.833456] tun-lev: port 24(lev-4f259735034) entered blocking state
[   13.835848] tun-lev: port 24(lev-4f259735034) entered disabled state
[   13.859846] device lev-4f259735034 entered promiscuous mode
[   13.877400] tun-lev: port 25(lev-34e6b46ceb1) entered blocking state
[   13.879504] tun-lev: port 25(lev-34e6b46ceb1) entered disabled state
[   13.903914] device lev-34e6b46ceb1 entered promiscuous mode
[   13.917359] tun-lev: port 26(lev-0c175ab1e2e) entered blocking state
[   13.919063] tun-lev: port 26(lev-0c175ab1e2e) entered disabled state
[   13.938811] device lev-0c175ab1e2e entered promiscuous mode
[   13.956528] tun-lev: port 27(lev-d7b0630c6a5) entered blocking state
[   13.958955] tun-lev: port 27(lev-d7b0630c6a5) entered disabled state
[   13.980879] device lev-d7b0630c6a5 entered promiscuous mode
[   13.998865] tun-lev: port 28(lev-55b9fa760e9) entered blocking state
[   14.000620] tun-lev: port 28(lev-55b9fa760e9) entered disabled state
[   14.018169] device lev-55b9fa760e9 entered promiscuous mode
[   14.027779] tun-lev: port 22(lev-7cd3791e7f1) entered blocking state
[   14.029522] tun-lev: port 22(lev-7cd3791e7f1) entered forwarding state
[   14.051599] tun-lev: port 23(lev-8f24d236b50) entered blocking state
[   14.053401] tun-lev: port 23(lev-8f24d236b50) entered forwarding state
[   14.080272] tun-lev: port 29(lev-9f6488d1c43) entered blocking state
[   14.081971] tun-lev: port 29(lev-9f6488d1c43) entered disabled state
[   14.095766] device lev-9f6488d1c43 entered promiscuous mode
[   14.104201] tun-lev: port 24(lev-4f259735034) entered blocking state
[   14.105926] tun-lev: port 24(lev-4f259735034) entered forwarding state
[   14.117093] tun-lev: port 25(lev-34e6b46ceb1) entered blocking state
[   14.119405] tun-lev: port 25(lev-34e6b46ceb1) entered forwarding state
[   14.137156] tun-lev: port 26(lev-0c175ab1e2e) entered blocking state
[   14.138904] tun-lev: port 26(lev-0c175ab1e2e) entered forwarding state
[   14.159305] tun-lev: port 30(lev-771a3092fc2) entered blocking state
[   14.160987] tun-lev: port 30(lev-771a3092fc2) entered disabled state
[   14.180247] device lev-771a3092fc2 entered promiscuous mode
[   14.185584] tun-lev: port 27(lev-d7b0630c6a5) entered blocking state
[   14.187324] tun-lev: port 27(lev-d7b0630c6a5) entered forwarding state
[   14.204301] tun-lev: port 28(lev-55b9fa760e9) entered blocking state
[   14.206767] tun-lev: port 28(lev-55b9fa760e9) entered forwarding state
[   14.254248] tun-lev: port 31(lev-1f07a34eff3) entered blocking state
[   14.255959] tun-lev: port 31(lev-1f07a34eff3) entered disabled state
[   14.269748] device lev-1f07a34eff3 entered promiscuous mode
[   14.275137] tun-lev: port 29(lev-9f6488d1c43) entered blocking state
[   14.276963] tun-lev: port 29(lev-9f6488d1c43) entered forwarding state
[   14.288384] tun-lev: port 30(lev-771a3092fc2) entered blocking state
[   14.290066] tun-lev: port 30(lev-771a3092fc2) entered forwarding state
[   14.307455] tun-lev: port 31(lev-1f07a34eff3) entered blocking state
[   14.309189] tun-lev: port 31(lev-1f07a34eff3) entered forwarding state
[   14.487332] tun-lev: port 32(lev-06e368ccdbe) entered blocking state
[   14.489343] tun-lev: port 32(lev-06e368ccdbe) entered disabled state
[   14.506856] device lev-06e368ccdbe entered promiscuous mode
[   14.514382] tun-lev: port 32(lev-06e368ccdbe) entered blocking state
[   14.516305] tun-lev: port 32(lev-06e368ccdbe) entered forwarding state
[   14.641096] tun-lev: port 33(lev-94d8f878b77) entered blocking state
[   14.642812] tun-lev: port 33(lev-94d8f878b77) entered disabled state
[   14.655237] device lev-94d8f878b77 entered promiscuous mode
[   14.663486] tun-lev: port 33(lev-94d8f878b77) entered blocking state
[   14.665189] tun-lev: port 33(lev-94d8f878b77) entered forwarding state
[   14.692947] tun-lev: port 34(lev-38ed619b8a4) entered blocking state
[   14.694951] tun-lev: port 34(lev-38ed619b8a4) entered disabled state
[   14.709357] device lev-38ed619b8a4 entered promiscuous mode
[   14.721456] tun-lev: port 34(lev-38ed619b8a4) entered blocking state
[   14.723121] tun-lev: port 34(lev-38ed619b8a4) entered forwarding state
[   16.413383] tun-lev: port 35(lev-cad502f0f34) entered blocking state
[   16.417971] tun-lev: port 35(lev-cad502f0f34) entered disabled state
[   16.436778] device lev-cad502f0f34 entered promiscuous mode
[   16.442826] tun-lev: port 35(lev-cad502f0f34) entered blocking state
[   16.444476] tun-lev: port 35(lev-cad502f0f34) entered forwarding state
[   16.496108] tun-lev: port 36(lev-8267f12add7) entered blocking state
[   16.497946] tun-lev: port 36(lev-8267f12add7) entered disabled state
[   16.512751] device lev-8267f12add7 entered promiscuous mode
[   16.539175] tun-lev: port 36(lev-8267f12add7) entered blocking state
[   16.540832] tun-lev: port 36(lev-8267f12add7) entered forwarding state
[   17.058709] tun-lev: port 37(lev-6764805ac40) entered blocking state
[   17.061013] tun-lev: port 37(lev-6764805ac40) entered disabled state
[   17.078265] device lev-6764805ac40 entered promiscuous mode
[   17.087440] tun-lev: port 37(lev-6764805ac40) entered blocking state
[   17.089745] tun-lev: port 37(lev-6764805ac40) entered forwarding state
[   35.893809] tun-lev: port 38(l2tp100-100) entered blocking state
[   35.898318] tun-lev: port 38(l2tp100-100) entered disabled state
[   35.930860] device l2tp100-100 entered promiscuous mode
[   35.941067] tun-lev: port 38(l2tp100-100) entered blocking state
[   35.942202] tun-lev: port 38(l2tp100-100) entered forwarding state
[  159.613868] batman_adv: mesh-lev: Adding interface: bb-mesh-lev
[  159.624515] batman_adv: mesh-lev: The MTU of interface bb-mesh-lev is too small (1500) to handle the transport of batman-adv packets. Packets going over this interface will be fragmented on layer2 which could impact the performance. Setting the MTU to 1560 would solve the problem.
[  159.650329] batman_adv: mesh-lev: Interface activated: bb-mesh-lev
[  159.671656] batman_adv: mesh-lev: Interface deactivated: bb-lev-1
[  159.675152] batman_adv: mesh-lev: Removing interface: bb-lev-1
[  159.702829] batman_adv: mesh-lev: Interface deactivated: bb-lev-4
[  159.704046] batman_adv: mesh-lev: Removing interface: bb-lev-4
[  159.715772] batman_adv: mesh-lev: Interface deactivated: bb-lev-3
[  159.716897] batman_adv: mesh-lev: Removing interface: bb-lev-3
[  161.406413] tun-lev: port 38(l2tp100-100) entered disabled state
[  161.429278] device l2tp100-100 left promiscuous mode
[  161.430231] tun-lev: port 38(l2tp100-100) entered disabled state
[  164.731356] tun-lev: port 38(l2tp101-101) entered blocking state
[  164.732494] tun-lev: port 38(l2tp101-101) entered disabled state
[  164.738332] device l2tp101-101 entered promiscuous mode
[  164.742172] tun-lev: port 38(l2tp101-101) entered blocking state
[  164.743260] tun-lev: port 38(l2tp101-101) entered forwarding state
[  240.816265] ------------[ cut here ]------------
[  240.819307] kernel BUG at net/core/skbuff.c:2247!
[  240.822038] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[  240.824163] CPU: 0 PID: 13 Comm: ksoftirqd/0 Kdump: loaded Not tainted 5.17.2-300.fc36.x86_64 #1
[  240.827627] Hardware name: Hetzner vServer, BIOS 20171111 11/11/2017
[  240.829406] RIP: 0010:__pskb_pull_tail+0x3b0/0x400
[  240.830860] Code: 8b b4 24 bc 00 00 00 4d 8b 84 24 c0 00 00 00 49 8d 14 30 48 8b 7a 08 e9 2d fe ff ff 48 c7 c6 e8 bc 5e 8b e8 e2 30 83 ff 0f 0b <0f> 0b 29 c2 48 89 7c 24 08 42 8d 34 3a e8 3e fc ff ff 48 8b 7c 24
[  240.837601] RSP: 0018:fffface380073830 EFLAGS: 00010282
[  240.839764] RAX: 00000000fffffff2 RBX: ffff921d9f863b00 RCX: 0000000000000001
[  240.842436] RDX: 0000000000000001 RSI: ffff921d8f145640 RDI: 0000000000000380
[  240.844451] RBP: fffface380073868 R08: ffff921d84dd9cac R09: 0000000000000022
[  240.846478] R10: 0000000000000001 R11: 00000000000002bf R12: ffff921d9f863b00
[  240.848510] R13: 00000000000002eb R14: 0000000000000000 R15: 0000000000000001
[  240.850622] FS:  0000000000000000(0000) GS:ffff921dfcc00000(0000) knlGS:0000000000000000
[  240.852960] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  240.854570] CR2: 000056045b049188 CR3: 0000000004dd6004 CR4: 00000000007706f0
[  240.856238] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  240.857883] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  240.859503] PKRU: 55555554
[  240.861767] Call Trace:
[  240.863187]  <TASK>
[  240.864458]  validate_xmit_skb+0x244/0x300
[  240.866141]  validate_xmit_skb_list+0x48/0x70
[  240.868019]  sch_direct_xmit+0x157/0x330
[  240.869946]  __dev_queue_xmit+0x9ad/0xcb0
[  240.871507]  ? packet_rcv+0x40/0x4a0
[  240.872908]  ? __br_forward+0x140/0x1c0 [bridge]
[  240.874740]  br_dev_queue_push_xmit+0x8b/0x170 [bridge]
[  240.876785]  br_dev_xmit+0x280/0x460 [bridge]
[  240.878537]  ? dev_queue_xmit_nit+0x261/0x2a0
[  240.880264]  dev_hard_start_xmit+0xb4/0x220
[  240.881885]  __dev_queue_xmit+0x831/0xcb0
[  240.883393]  ? nf_ct_deliver_cached_events+0x5a/0x70 [nf_conntrack]
[  240.885479]  ? __build_skb+0x1f/0x50
[  240.886950]  ? kmem_cache_alloc+0x162/0x2c0
[  240.888709]  batadv_send_skb_packet+0xe0/0x110 [batman_adv]
[  240.890611]  batadv_send_unicast_skb+0x1c/0x90 [batman_adv]
[  240.892551]  batadv_frag_send_packet+0x216/0x3e0 [batman_adv]
[  240.894473]  batadv_send_skb_to_orig+0xac/0xe0 [batman_adv]
[  240.896484]  batadv_route_unicast_packet+0xc5/0x1c0 [batman_adv]
[  240.898417]  batadv_recv_unicast_packet+0x39e/0x6d0 [batman_adv]
[  240.900277]  batadv_batman_skb_recv+0x120/0x1d0 [batman_adv]
[  240.902309]  __netif_receive_skb_list_core+0x24a/0x2a0
[  240.904458]  netif_receive_skb_list_internal+0x1b7/0x2d0
[  240.906350]  napi_complete_done+0x6d/0x190
[  240.908177]  gro_cell_poll+0x70/0x90
[  240.909499]  __napi_poll+0x2b/0x160
[  240.910763]  net_rx_action+0x224/0x2d0
[  240.912082]  __do_softirq+0xfc/0x30e
[  240.913511]  ? smpboot_register_percpu_thread+0xf0/0xf0
[  240.915107]  run_ksoftirqd+0x33/0x40
[  240.916346]  smpboot_thread_fn+0x188/0x220
[  240.917643]  kthread+0xe9/0x110
[  240.919179]  ? kthread_complete_and_exit+0x20/0x20
[  240.921504]  ret_from_fork+0x22/0x30
[  240.923424]  </TASK>
[  240.924914] Modules linked in: tun vxlan ip_gre ip_tunnel gre intel_rapl_msr intel_rapl_common isst_if_common nfit libnvdimm snd_pcm nft_nat snd_timer nft_masq snd virtio_net nft_meta_bridge nft_ct nft_chain_nat nf_nat bochs nf_conntrack soundcore joydev nf_defrag_ipv6 nf_defrag_ipv4 drm_vram_helper net_failover pcspkr drm_ttm_helper virtio_balloon ttm nf_tables failover i2c_piix4 nfnetlink l2tp_eth l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel tcp_westwood batman_adv cfg80211 vfat fat bridge stp llc rfkill fuse crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw virtio_console virtio_scsi ata_generic qemu_fw_cfg pata_acpi

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: Kernel crash when using batman-adv 2022.0 over VXLAN
  2022-04-16  0:23 Kernel crash when using batman-adv 2022.0 over VXLAN Felix Kaechele
@ 2022-04-16 10:08 ` Sven Eckelmann
  0 siblings, 0 replies; 2+ messages in thread
From: Sven Eckelmann @ 2022-04-16 10:08 UTC (permalink / raw)
  To: Felix Kaechele; +Cc: b.a.t.m.a.n

[-- Attachment #1: Type: text/plain, Size: 12187 bytes --]

On Saturday, 16 April 2022 02:23:59 CEST Felix Kaechele wrote:
> Hi there,
> 
> I am seeing odd kernel crashes when using batman-adv over VXLAN.
> This is in a Freifunk environment on our VPN gateway side where clients 
> connect using L2TPv3 tunnels and the gateways are interconnected using 
> VXLAN.
> 
> The setup is as follows on both ends:
> 
> - Both machines are Hetzner Cloud VMs (CX11)
> - OS: Fedora 36 (Pre-release) / Kernel 5.17.2-300.fc36.x86_64

So it doesn't crash in batman-adv but the skbuff code complains about it when 
it went through the bridge. The code which crashes is:

    void *__pskb_pull_tail(struct sk_buff *skb, int delta)
    {
    	/* If skb has not enough free space at tail, get new one
    	 * plus 128 bytes for future expansions. If we have enough
    	 * room at tail, reallocate without expansion only if skb is cloned.
    	 */
    	int i, k, eat = (skb->tail + delta) - skb->end;
    
    	if (eat > 0 || skb_cloned(skb)) {
    		if (pskb_expand_head(skb, 0, eat > 0 ? eat + 128 : 0,
    				     GFP_ATOMIC))
    			return NULL;
    	}
    
    	BUG_ON(skb_copy_bits(skb, skb_headlen(skb),
    			     skb_tail_pointer(skb), delta));


So it seems like skb_copy_bits fails because it is unable to copy (delta) 
bytes from the skb at the start of the header to tail. Afaik, this can only 
happen when skb_headlen(skb) + delta is larger than the complete length of the 
skb. Or when the same thing is true for any of its fragments.

Question is now: what is calling the function with which parameter. So the 
first step was to try to decode the stacktrace (more or less well) via the
./scripts/decode_stacktrace.sh and the vmlinux from
https://kojipkgs.fedoraproject.org//packages/kernel/5.17.2/300.fc36/x86_64/kernel-debug-debuginfo-5.17.2-300.fc36.x86_64.rpm

    $ ~/linux-5.17.2/scripts/decode_stacktrace.sh  vmlinux < ~/decoded 
    [  240.816265] ------------[ cut here ]------------
    [  240.819307] kernel BUG at net/core/skbuff.c:2247!
    [  240.822038] invalid opcode: 0000 [#1] PREEMPT SMP PTI
    [  240.824163] CPU: 0 PID: 13 Comm: ksoftirqd/0 Kdump: loaded Not tainted 5.17.2-300.fc36.x86_64 #1
    [  240.827627] Hardware name: Hetzner vServer, BIOS 20171111 11/11/2017
    [  240.829406] RIP: 0010:__pskb_pull_tail (./include/linux/mm.h:717 ./include/linux/mm.h:723 ./include/linux/mm.h:1198 ./include/linux/mm.h:1237 ./include/linux/skbuff.h:3188 ./include/linux/skbuff.h:3200 net/core/skbuff.c:2327) 
    [ 240.830860] Code: 8b b4 24 bc 00 00 00 4d 8b 84 24 c0 00 00 00 49 8d 14 30 48 8b 7a 08 e9 2d fe ff ff 48 c7 c6 e8 bc 5e 8b e8 e2 30 83 ff 0f 0b <0f> 0b 29 c2 48 89 7c 24 08 42 8d 34 3a e8 3e fc ff ff 48 8b 7c 24
    All code
    ========
       0:   8b b4 24 bc 00 00 00    mov    0xbc(%rsp),%esi
       7:   4d 8b 84 24 c0 00 00    mov    0xc0(%r12),%r8
       e:   00 
       f:   49 8d 14 30             lea    (%r8,%rsi,1),%rdx
      13:   48 8b 7a 08             mov    0x8(%rdx),%rdi
      17:   e9 2d fe ff ff          jmpq   0xfffffffffffffe49
      1c:   48 c7 c6 e8 bc 5e 8b    mov    $0xffffffff8b5ebce8,%rsi
      23:   e8 e2 30 83 ff          callq  0xffffffffff83310a
      28:   0f 0b                   ud2    
      2a:*  0f 0b                   ud2             <-- trapping instruction
      2c:   29 c2                   sub    %eax,%edx
      2e:   48 89 7c 24 08          mov    %rdi,0x8(%rsp)
      33:   42 8d 34 3a             lea    (%rdx,%r15,1),%esi
      37:   e8 3e fc ff ff          callq  0xfffffffffffffc7a
      3c:   48                      rex.W
      3d:   8b                      .byte 0x8b
      3e:   7c 24                   jl     0x64
    
    Code starting with the faulting instruction
    ===========================================
       0:   0f 0b                   ud2    
       2:   29 c2                   sub    %eax,%edx
       4:   48 89 7c 24 08          mov    %rdi,0x8(%rsp)
       9:   42 8d 34 3a             lea    (%rdx,%r15,1),%esi
       d:   e8 3e fc ff ff          callq  0xfffffffffffffc50
      12:   48                      rex.W
      13:   8b                      .byte 0x8b
      14:   7c 24                   jl     0x3a
    [  240.837601] RSP: 0018:fffface380073830 EFLAGS: 00010282
    [  240.839764] RAX: 00000000fffffff2 RBX: ffff921d9f863b00 RCX: 0000000000000001
    [  240.842436] RDX: 0000000000000001 RSI: ffff921d8f145640 RDI: 0000000000000380
    [  240.844451] RBP: fffface380073868 R08: ffff921d84dd9cac R09: 0000000000000022
    [  240.846478] R10: 0000000000000001 R11: 00000000000002bf R12: ffff921d9f863b00
    [  240.848510] R13: 00000000000002eb R14: 0000000000000000 R15: 0000000000000001
    [  240.850622] FS:  0000000000000000(0000) GS:ffff921dfcc00000(0000) knlGS:0000000000000000
    [  240.852960] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    [  240.854570] CR2: 000056045b049188 CR3: 0000000004dd6004 CR4: 00000000007706f0
    [  240.856238] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    [  240.857883] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
    [  240.859503] PKRU: 55555554
    [  240.861767] Call Trace:
    [  240.863187]  <TASK>
    [  240.864458] validate_xmit_skb (./include/linux/skbuff.h:3465 net/core/dev.c:3563) 
    [  240.866141] validate_xmit_skb_list (net/core/dev.c:3605) 
    [  240.868019] sch_direct_xmit (net/sched/sch_generic.c:330) 
    [  240.869946] __dev_queue_xmit (./include/net/pkt_sched.h:131 net/core/dev.c:3709 net/core/dev.c:4081) 
    [  240.871507] ? packet_rcv (net/packet/af_packet.c:2127) 
    [  240.872908] ? __br_forward (./include/linux/rcupdate.h:273 ./include/linux/rcupdate.h:727 ./include/linux/netfilter.h:264 ./include/linux/netfilter.h:305 net/bridge/br_forward.c:115) bridge
    [  240.874740] br_dev_queue_push_xmit (net/bridge/br_forward.c:53) bridge
    [  240.876785] br_dev_xmit (net/bridge/br_device.c:108) bridge
    [  240.878537] ? dev_queue_xmit_nit (./include/linux/skbuff.h:1471 ./include/linux/skbuff.h:1510 ./include/linux/skbuff.h:2895 net/core/dev.c:2242) 
    [  240.880264] dev_hard_start_xmit (./include/linux/netdevice.h:4684 ./include/linux/netdevice.h:4699 net/core/dev.c:3473 net/core/dev.c:3489) 
    [  240.881885] __dev_queue_xmit (net/core/dev.c:3700 net/core/dev.c:4081) 
    [  240.883393] ? nf_ct_deliver_cached_events (net/netfilter/nf_conntrack_ecache.c:234) nf_conntrack
    [  240.885479] ? __build_skb (net/core/skbuff.c:239) 
    [  240.886950] ? kmem_cache_alloc (mm/slub.c:3214 mm/slub.c:3238 mm/slub.c:3243) 
    [  240.888709] batadv_send_skb_packet (net/batman-adv/send.c:109) batman_adv
    [  240.890611] batadv_send_unicast_skb (net/batman-adv/send.c:150) batman_adv
    [  240.892551] batadv_frag_send_packet (net/batman-adv/fragmentation.c:424 net/batman-adv/fragmentation.c:508) batman_adv
    [  240.894473] batadv_send_skb_to_orig (net/batman-adv/send.c:197) batman_adv
    [  240.896484] batadv_route_unicast_packet (net/batman-adv/routing.c:706) batman_adv
    [  240.898417] batadv_recv_unicast_packet (net/batman-adv/routing.c:1020) batman_adv
    [  240.900277] batadv_batman_skb_recv (net/batman-adv/hard-interface.h:92 net/batman-adv/main.c:486) batman_adv
    [  240.902309] __netif_receive_skb_list_core (net/core/dev.c:5397 net/core/dev.c:5383 net/core/dev.c:5442) 
    [  240.904458] netif_receive_skb_list_internal (./include/linux/sched/mm.h:330 net/core/dev.c:5489 net/core/dev.c:5585) 
    [  240.906350] napi_complete_done (./include/linux/list.h:37 ./include/net/gro.h:426 ./include/net/gro.h:421 net/core/dev.c:5932) 
    [  240.908177] gro_cell_poll (net/core/gro_cells.c:54) 
    [  240.909499] __napi_poll+0x2b/0x160 
    [  240.910763] net_rx_action (net/core/dev.c:6538) 
    [  240.912082] __do_softirq (kernel/softirq.c:560) 
    [  240.913511] ? smpboot_register_percpu_thread (kernel/smpboot.c:107) 
    [  240.915107] run_ksoftirqd (kernel/softirq.c:916) 
    [  240.916346] smpboot_thread_fn (kernel/smpboot.c:144 (discriminator 3)) 
    [  240.917643] kthread (kernel/kthread.c:376) 
    [  240.919179] ? kthread_complete_and_exit (kernel/kthread.c:332) 
    [  240.921504] ret_from_fork (arch/x86/entry/entry_64.S:301) 
    [  240.923424]  </TASK>
    [  240.924914] Modules linked in: tun vxlan ip_gre ip_tunnel gre intel_rapl_msr intel_rapl_common isst_if_common nfit libnvdimm snd_pcm nft_nat snd_timer nft_masq snd virtio_net nft_meta_bridge nft_ct nft_chain_nat nf_nat bochs nf_conntrack soundcore joydev nf_defrag_ipv6 nf_defrag_ipv4 drm_vram_helper net_failover pcspkr drm_ttm_helper virtio_balloon ttm nf_tables failover i2c_piix4 nfnetlink l2tp_eth l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel tcp_westwood batman_adv cfg80211 vfat fat bridge stp llc rfkill fuse crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw virtio_console virtio_scsi ata_generic qemu_fw_cfg pata_acpi


As we can see, the __pskb_pull_tail comes from __skb_linearize. And its done 
using skb->data_len. According to our previous investigation for the non-
frag_list check, this would mean that
skb_headlen(skb) + skb->data_len > skb->len. And the definition of skb_headlen 
is skb->len - skb->data_len. So it can be transformed to:

     skb->len - skb->data_len + skb->data_len > skb->len

or

     skb->len > skb->len

Of course, this contradicts itself. So we have to assume that it either failed 
on the fragments or data_len changed - which I don't think that it does this 
in pskb_expand_head. But maybe someone can correct me.

But we can say that skb_needs_linearize in  net/core/dev.c:3562 returns true 
for some reason. batman-adv should not create any skb_fragments but will 
happily forward it to skb_split to create two skbs (and thus one batman-adv 
fragment packet might actually contain the skb fragments which were coming 
from the outside). So linearization by the dev_xmit code would be required 
when:

* skb is non linear (skb->data_len != 0)
* AND either:
  - skb has a skb fraglist (not batman-adv fraglist) && underlying interface 
    doesn't support NETIF_F_FRAGLIST
    (afaik, this can happen with skb_gro_receive but isn't handled by 
     skb_split)
  - skb has frags but underlying interface doesn't support support NETIF_F_SG

I am not sure what was used to receive the packet and what was then used to 
outgoing interface. But it seems like the receiving interface created either 
an skb with fraglist + the outgoing interface doesn't support it. Or the 
receiving interface created skb frags but the outgoing interface doesn't 
support scatter gather.

Still doesn't explain 100% why the skb_copy_bits fails. If you want then you 
can try to add a skb_linearize(skb) to the start of batadv_batman_skb_recv 
(directly after the variable declarations):

	/* keep skb linear */
	if (skb_linearize(skb) < 0)
		goto err_out;

to check if batman-adv is (indirectly) forwarding the problem with the skb or 
really introducing it (by incorrectly creating skb fragments somewhere). If 
this solves it, please remove the change and test if it also helps to just get 
rid of the fraglist:

	if (skb_has_frag_list(skb) && __skb_linearize(skb))
		goto err_out;

And if this really helps, please move this code code to 
batadv_frag_send_packet. This should look like this

    diff --git a/net/batman-adv/fragmentation.c b/net/batman-adv/fragmentation.c
    index 0899a729..5fc07263 100644
    --- a/net/batman-adv/fragmentation.c
    +++ b/net/batman-adv/fragmentation.c
    @@ -475,6 +475,15 @@ int batadv_frag_send_packet(struct sk_buff *skb,
     		goto free_skb;
     	}
     
    +	/* GRO might have added fragments to the fragment list instead of
    +	 * frags[]. But this is not handled by skb_split and must be
    +	 * linearized before it is given to it
    +	 */
    +	if (skb_has_frag_list(skb) && __skb_linearize(skb)) {
    +		ret = -ENOMEM;
    +		goto free_skb;
    +	}
    +
     	/* Create one header to be copied to all fragments */
     	frag_header.packet_type = BATADV_UNICAST_FRAG;
     	frag_header.version = BATADV_COMPAT_VERSION;


Thanks for the report and good luck with the testing - and I hope you find 
better easter eggs this year than this bug ;)

Kind regards,
	Sven

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-04-16 10:08 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-16  0:23 Kernel crash when using batman-adv 2022.0 over VXLAN Felix Kaechele
2022-04-16 10:08 ` Sven Eckelmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).