Re: [Xen-devel] dom0less + sched=null => broken in staging

[Stefano Stabellini <sstabellini@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

On Wed, 14 Aug 2019, Dario Faggioli wrote:
> On Tue, 2019-08-13 at 14:14 -0700, Stefano Stabellini wrote:
> > On Tue, 13 Aug 2019, Dario Faggioli wrote:
> > > 
> > > I am attaching an updated debug patch, with an additional printk
> > > when
> > > we reach the point, within the null scheduler, when the vcpu would
> > > wake
> > > up (to check whether the problem is that we never reach that point,
> > > or
> > > something else).
> > 
> > See attached.
> >
> Ok, so we're not missing an "online call" nor a wakeup.
> 
> As Julien has identified, we seem to be stuck in a loop.
> 
> Now, while staring at the code of that loop, I've seen that pick_cpu()
> may mess up with the scratch cpumask for the CPU, which I don't think
> is a good thing.
> 
> So, can you also try this third debug-patch?

Yep, see attached

[-- Attachment #2: Type: text/plain, Size: 6479 bytes --]

(XEN) Xen version 4.13-unstable (sstabellini@) (aarch64-linux-gnu-gcc (Linaro GCC 5.3-2016.05) 5.3.1 20160412) debug=y  Wed Aug 14 09:24:00 PDT 2019
(XEN) Latest ChangeSet: Fri Dec 21 13:44:30 2018 +0000 git:243cc95d48-dirty
(XEN) build-id: 992c2d9483465d0f9d8f1e9ec372e1cfc4b90bb3
(XEN) Processor: 410fd034: "ARM Limited", variant: 0x0, part 0xd03, rev 0x4
(XEN) 64-bit Execution:
(XEN)   Processor Features: 1100000000002222 0000000000000000
(XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN)     Extensions: FloatingPoint AdvancedSIMD
(XEN)   Debug Features: 0000000010305106 0000000000000000
(XEN)   Auxiliary Features: 0000000000000000 0000000000000000
(XEN)   Memory Model Features: 0000000000001122 0000000000000000
(XEN)   ISA Features:  0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00001231:00011011
(XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 ThumbEE Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 03010066
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10101105 40000000 01260000 02102211
(XEN)  ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Using SMC Calling Convention v1.1
(XEN) Using PSCI v1.1
(XEN) SMP: Allowing 4 CPUs
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 50000 KHz
(XEN) GICv2 initialization:
(XEN)         gic_dist_addr=00000000f9010000
(XEN)         gic_cpu_addr=00000000f9020000
(XEN)         gic_hyp_addr=00000000f9040000
(XEN)         gic_vcpu_addr=00000000f9060000
(XEN)         gic_maintenance_irq=25
(XEN) GICv2: Adjusting CPU interface base to 0xf902f000
(XEN) GICv2: 192 lines, 4 cpus (IID 00000000).
(XEN) XSM Framework v1.0.0 initialized
(XEN) Initialising XSM SILO mode
(XEN) WARNING: hypervisor-timer IRQ26 is not level triggered.
(XEN) WARNING: virtual-timer IRQ27 is not level triggered.
(XEN) WARNING: NS-physical-timer IRQ30 is not level triggered.
(XEN) Using scheduler: null Scheduler (null)
(XEN) Initializing null scheduler
(XEN) WARNING: This is experimental software in development.
(XEN) Use at your own risk.
(XEN) Allocated console ring of 32 KiB.
(XEN) CPU0: Guest atomics will try 1 times before pausing the domain
(XEN) Bringing up CPU1
(XEN) WARNING: hypervisor-timer IRQ26 is not level triggered.
(XEN) WARNING: virtual-timer IRQ27 is not level triggered.
(XEN) WARNING: NS-physical-timer IRQ30 is not level triggered.
(XEN) CPU1: Guest atomics will try 1 times before pausing the domain
(XEN) CPU 1 booted.
(XEN) Bringing up CPU2
(XEN) WARNING: hypervisor-timer IRQ26 is not level triggered.
(XEN) WARNING: virtual-timer IRQ27 is not level triggered.
(XEN) WARNING: NS-physical-timer IRQ30 is not level triggered.
(XEN) CPU2: Guest atomics will try 1 times before pausing the domain
(XEN) CPU 2 booted.
(XEN) Bringing up CPU3
(XEN) WARNING: hypervisor-timer IRQ26 is not level triggered.
(XEN) WARNING: virtual-timer IRQ27 is not level triggered.
(XEN) WARNING: NS-physical-timer IRQ30 is not level triggered.
(XEN) CPU3: Guest atomics will try 1 times before pausing the domain
(XEN) CPU 3 booted.
(XEN) Brought up 4 CPUs
(XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID
(XEN) P2M: 3 levels with order-1 root, VTCR 0x80023558
(XEN) smmu: /amba/smmu@fd800000: probing hardware configuration...
(XEN) smmu: /amba/smmu@fd800000: SMMUv2 with:
(XEN) smmu: /amba/smmu@fd800000:        stage 2 translation
(XEN) smmu: /amba/smmu@fd800000:        stream matching with 48 register groups, mask 0x7fff
(XEN) smmu: /amba/smmu@fd800000:        16 context banks (0 stage-2 only)
(XEN) smmu: /amba/smmu@fd800000:        Stage-2: 40-bit IPA -> 48-bit PA
(XEN) smmu: /amba/smmu@fd800000: registered 26 master devices
/amba@0/smmu0@0xFD800000: Decode error: write to 6c=0
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) alternatives: Patching with alt table 00000000002bbe68 -> 00000000002bc528
(XEN) sched_null.c:458: Not inserting d0v0 (not online!)
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading d0 kernel from boot module @ 0000000001000000
(XEN) Loading ramdisk from boot module @ 0000000002400000
(XEN) Allocating 1:1 mappings totalling 700MB for dom0:
(XEN) BANK[0] 0x00000020000000-0x00000040000000 (512MB)
(XEN) BANK[1] 0x00000070000000-0x00000078000000 (128MB)
(XEN) BANK[2] 0x0000007c000000-0x0000007fc00000 (60MB)
(XEN) Grant table range: 0x00000000e00000-0x00000000e40000
(XEN) smmu: /amba/smmu@fd800000: d0: p2maddr 0x000000087ffa2000
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading zImage from 0000000001000000 to 0000000020080000-0000000021372200
(XEN) Loading dom0 initrd from 0000000002400000 to 0x0000000028200000-0x00000000297fa954
(XEN) Loading dom0 DTB to 0x0000000028000000-0x0000000028006d75
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input to DOM0 (type 'CTRL-a' three times to switch input)
(XEN) *** LOADING DOMU cpus=1 memory=40000KB ***
(XEN) sched_null.c:458: Not inserting d1v0 (not online!)
(XEN) Loading d1 kernel from boot module @ 0000000004c00000
(XEN) Loading ramdisk from boot module @ 0000000005c00000
(XEN) Allocating mappings totalling 256MB for d1:
(XEN) d1 BANK[0] 0x00000040000000-0x00000050000000 (256MB)
(XEN) d1 BANK[1] 0x00000200000000-0x00000200000000 (0MB)
(XEN) Loading zImage from 0000000004c00000 to 0000000040080000-000000004105fa00
(XEN) Loading dom0 initrd from 0000000005c00000 to 0x0000000048200000-0x0000000048383400
(XEN) Loading dom0 DTB to 0x0000000048000000-0x00000000480004bd
(XEN) sched_null.c:578: d1v0 is waking up after having been offline
(XEN) *** LOADING DOMU cpus=1 memory=40000KB ***
(XEN) sched_null.c:458: Not inserting d2v0 (not online!)
(XEN) Loading d2 kernel from boot module @ 0000000003a00000
(XEN) Loading ramdisk from boot module @ 0000000004a00000
(XEN) Allocating mappings totalling 256MB for d2:
(XEN) d2 BANK[0] 0x00000040000000-0x00000050000000 (256MB)
(XEN) d2 BANK[1] 0x00000200000000-0x00000200000000 (0MB)
(XEN) Loading zImage from 0000000003a00000 to 0000000040080000-000000004105fa00
(XEN) Loading dom0 initrd from 0000000004a00000 to 0x0000000048200000-0x0000000048383400
(XEN) Loading dom0 DTB to 0x0000000048000000-0x00000000480004bd
(XEN) sched_null.c:578: d2v0 is waking up after having been offline


[-- Attachment #3: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel