Re: [Xen-devel] dom0less + sched=null => broken in staging

[Dario Faggioli <dfaggioli@suse.com>]

[-- Attachment #1.1.1: Type: text/plain, Size: 1655 bytes --]

On Fri, 2019-08-09 at 11:30 -0700, Stefano Stabellini wrote:
> On Fri, 9 Aug 2019, Dario Faggioli wrote:
> > Can you help me with this, e.g., by providing some more info and,
> > if
> > possible, logs?
> 
> I am attaching the logs. 
>
Thanks!

> Interestingly, I get a bunch of:
> 
> (XEN) *** LOADING DOMU cpus=1 memory=40000KB ***
> (XEN) sched_null.c:458: Not inserting d2v0 (not online!)
> 
> Maybe we are missing a call to online the vcpus somewhere in
> xen/arch/arm/domain_build.c:construct_domain?
> 
Actually, those lines are normal, because vCPUs are created offline.
(see the set_bit(_VPF_down) in vcpu_create()).

The problem is why aren't they coming up. Basically, you're missing a
call to vcpu_wake().

In my (x86 and "dom0full") testbox, this seems to come from
domain_unpause_by_systemcontroller(dom0) called by
xen/arch/x86/setup.c:init_done(), at the very end of __start_xen().

I don't know if domain construction in an ARM dom0less system works
similarly, though. What we want, is someone calling either vcpu_wake()
or vcpu_unpause(), after having cleared _VPF_down from pause_flags.

I am attaching an updated debug patch, with an additional printk when
we reach the point, within the null scheduler, when the vcpu would wake
up (to check whether the problem is that we never reach that point, or
something else).

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)


[-- Attachment #1.1.2: xen-sched-null-vcpu-onoff-debug-v2.patc --]
[-- Type: text/x-patch, Size: 2001 bytes --]

diff --git a/xen/common/sched_null.c b/xen/common/sched_null.c
index 26c6f0f129..e78afadf5c 100644
--- a/xen/common/sched_null.c
+++ b/xen/common/sched_null.c
@@ -455,6 +455,7 @@ static void null_vcpu_insert(const struct scheduler *ops, struct vcpu *v)
 
     if ( unlikely(!is_vcpu_online(v)) )
     {
+        dprintk(XENLOG_G_INFO, "Not inserting %pv (not online!)\n", v);
         vcpu_schedule_unlock_irq(lock, v);
         return;
     }
@@ -516,6 +517,7 @@ static void null_vcpu_remove(const struct scheduler *ops, struct vcpu *v)
     /* If offline, the vcpu shouldn't be assigned, nor in the waitqueue */
     if ( unlikely(!is_vcpu_online(v)) )
     {
+        dprintk(XENLOG_G_INFO, "Not removing %pv (wasn't online!)\n", v);
         ASSERT(per_cpu(npc, v->processor).vcpu != v);
         ASSERT(list_empty(&nvc->waitq_elem));
         goto out;
@@ -571,6 +573,7 @@ static void null_vcpu_wake(const struct scheduler *ops, struct vcpu *v)
      */
     if ( unlikely(per_cpu(npc, cpu).vcpu != v && list_empty(&nvc->waitq_elem)) )
     {
+        dprintk(XENLOG_G_INFO, "%pv is waking up after having been offline\n", v);
         spin_lock(&prv->waitq_lock);
         list_add_tail(&nvc->waitq_elem, &prv->waitq);
         spin_unlock(&prv->waitq_lock);
@@ -635,6 +638,8 @@ static void null_vcpu_sleep(const struct scheduler *ops, struct vcpu *v)
         }
         else if ( per_cpu(npc, cpu).vcpu == v )
             tickled = vcpu_deassign(prv, v);
+
+        dprintk(XENLOG_G_INFO, "%pv is, apparently, going offline (tickled=%d)\n", v, tickled);
     }
 
     /* If v is not assigned to a pCPU, or is not running, no need to bother */
@@ -697,6 +702,8 @@ static void null_vcpu_migrate(const struct scheduler *ops, struct vcpu *v,
      */
     if ( unlikely(!is_vcpu_online(v)) )
     {
+        dprintk(XENLOG_G_INFO, "%pv is, apparently, going offline\n", v);
+
         spin_lock(&prv->waitq_lock);
         list_del_init(&nvc->waitq_elem);
         spin_unlock(&prv->waitq_lock);

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel