* null scheduler bug
@ 2018-09-12 23:03 Stefano Stabellini
2018-09-13 7:04 ` Dario Faggioli
0 siblings, 1 reply; 26+ messages in thread
From: Stefano Stabellini @ 2018-09-12 23:03 UTC (permalink / raw)
To: dfaggioli; +Cc: xen-devel, sstabellini, milanboberic94
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1167 bytes --]
Hi Dario,
Milan has just found a bug in the null scheduler: apparently it is not
possible to start a VM again after it has been destroyed.
My initial suspicion was that the VM wasn't properly destroyed, but I
asked Milan to double check with xl list, and the VM doesn't show in the
list anymore.
Any ideas?
Cheers,
Stefano
On Wed, 12 Sep 2018, Milan Boberic wrote:
> I double checked it, when I remove sched=null and vwfi=native I can create and destroy bare-metal application as many times as I
> want.
> When I include sched=null and vwfi=native I can create it once and when I destroy it I can't create it again, again the same
> error.
>
> root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
> Parsing config from bm1.cfg
> (XEN) IRQ 48 is already used by domain 1
> libxl: error: libxl_create.c:1278:domcreate_launch_dm: Domain 2:failed give domain access to irq 48: Device or resource busy
> libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain 2:Non-existant domain
> libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain 2:Unable to destroy guest
> libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain 2:Destruction of domain failed
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-12 23:03 null scheduler bug Stefano Stabellini
@ 2018-09-13 7:04 ` Dario Faggioli
[not found] ` <CADJ6SV2G=51BK2p-bouNGfiS5sP2tiV6ztZZ7PFGjiptRw_P3w@mail.gmail.com>
0 siblings, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-13 7:04 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: xen-devel, milanboberic94
[-- Attachment #1.1: Type: text/plain, Size: 1229 bytes --]
On Wed, 2018-09-12 at 16:03 -0700, Stefano Stabellini wrote:
> Hi Dario,
>
> Milan has just found a bug in the null scheduler: apparently it is
> not
> possible to start a VM again after it has been destroyed.
>
> My initial suspicion was that the VM wasn't properly destroyed, but I
> asked Milan to double check with xl list, and the VM doesn't show in
> the
> list anymore.
>
Ok. With what version of Xen? Is PCI passthrough/device assignment
involved? I'm asking because this highly reminds me of this:
https://lists.xen.org/archives/html/xen-devel/2017-07/msg00501.html
Which then led to this:
https://www.mail-archive.com/xen-devel@lists.xen.org/msg105388.html
And to the following changes (related to RCU, which is how complete
destruction of a domain occurs):
2b936ea7b "xen: RCU: avoid busy waiting until the end of grace period."
38ad8151f "xen: RCU: don't let a CPU with a callback go idle."
...
Are these commit there?
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
[not found] ` <CADJ6SV2G=51BK2p-bouNGfiS5sP2tiV6ztZZ7PFGjiptRw_P3w@mail.gmail.com>
@ 2018-09-13 8:24 ` Dario Faggioli
2018-09-13 15:18 ` Milan Boberic
0 siblings, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-13 8:24 UTC (permalink / raw)
To: Milan Boberic; +Cc: xen-devel, Stefano Stabellini
[-- Attachment #1.1: Type: text/plain, Size: 3354 bytes --]
Hi,
So, first of all:
1. use plaintext, not HTML
2. don't drop the xen-devel list (and other Cc-s) when replying.
:-)
That being said...
On Thu, 2018-09-13 at 09:38 +0200, Milan Boberic wrote:
> Hi Dario,
> yes passtrhough is involved.
>
Ok.
> This is everything I did so far:
>
> I implemented Xen Hypervisor 4.9.2 on UltraZed-EG board with carrier
> card following these steps:
> 1.) installed petalinux 2018.2 on Ubuntu 16.04
> 2.) dowloaded UltraZed-EG IO Carrier Card - PetaLinux 2018.2
> Standard BSP
> 3.) created project: petalinux-create -t project –s <path_to_BSP>
> 4.) copied xen-overlay.dtsi from ZCU102 project (also made with BSP)
> to my project
> 5.) built xen hypervisor following this guide link (booting with SD
> card)
> 6.) made baremetal application that blinks PS LED with Vivado
> 7.) measured signals jitted when application is on board with and
> with out Xen
>
I am not familiar with building an SD-Card image for an ARM system with
Xen on it. What I can tell, though, is that Xen 4.9.2 does not look to
me to include the RCU subsystem fixes that I mentioned in my reply.
I don't recall why we did not backport them. It may be that they're not
entirely trivial, and they fix a behavior which manifests only in not
fully supported cases. Or that we (well, I :-/) forgot.
I can have a look, at how challenging it is to apply the patches to
4.9.x (but no hard feelings if anyone beats me at it, I promise
:-D).
In the meantime, if you have the chance of trying Xen 4.10 or 4.10.1,
which has those fixes, that would be great. In fact, in case everything
would work with such version, that would be another clue that the RCU
issue is the actual culprit.
> I menaged to decrease jitter adding sched=null vwfi=native in xen-
> overlay.dtsi file in xen-bootargs.
>
> The problem is when I add sched=null vwfi=native, I can create my
> bare-metal application with xl create and destroy it with xl destroy
> (it disappears from xl list) but when I want to start it again (xl
> create) this error pops:
>
> root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
> Parsing config from bm1.cfg
> (XEN) IRQ 48 is already used by domain 1
> libxl: error: libxl_create.c:1278:domcreate_launch_dm: Domain
> 2:failed give domain access to irq 48: Device or resource busy
> libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain 2:Non-
> existant domain
> libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain
> 2:Unable to destroy guest
> libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain
> 2:Destruction of domain failed
>
> When I remove sched=null vwfi=native and build project again
> everything works fine, I can create and destroy bm app as many times
> as I want.
>
Yes. If you look at the email (and to the full thread) I put links to,
you'll see that the behavior is exactly the same. It mentions the
Credit2 scheduler not working, rather than null, but the problem is the
same, i.e., that there is no periodic timer tick neither in Credit2 nor
in null.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-13 8:24 ` Dario Faggioli
@ 2018-09-13 15:18 ` Milan Boberic
2018-09-13 17:39 ` Dario Faggioli
0 siblings, 1 reply; 26+ messages in thread
From: Milan Boberic @ 2018-09-13 15:18 UTC (permalink / raw)
To: Dario Faggioli; +Cc: xen-devel, stefano
I'm sorry for html and dropping xen-devel and dropping other CCs,
missed to read the rules.
I tried 4.10 version and checked for commits you asked in earlier reply.
2b936ea7b "xen: RCU: avoid busy waiting until the end of grace period."
38ad8151f "xen: RCU: don't let a CPU with a callback go idle."
Commits are there and I will definitely continue with 4.10 version.
But it didn't solve my problem entirely.
I create my bare-metal application (with xl create) and destroy it
with xl destroy (it disappears from xl list) and when I try to create
it again same error pops but if I immediately run xl create command
again it creates it without error.
If I run xl create twice fast sometimes bare-metal application isn't
in any state (it should be in "running" state).
If I wait some time (approximately between 30 and 90 seconds) after
destruction of that bm app and then run xl create it will create it
without error.
This is github repository of Xen I use:
https://github.com/Xilinx/xen/tree/xilinx/stable-4.10
Is there anything else I can send that would be helpful?
Thanks in advance, best regards, Milan Boberic.
On Thu, Sep 13, 2018 at 10:24 AM Dario Faggioli <dfaggioli@suse.com> wrote:
>
> Hi,
>
> So, first of all:
> 1. use plaintext, not HTML
> 2. don't drop the xen-devel list (and other Cc-s) when replying.
>
> :-)
>
> That being said...
>
> On Thu, 2018-09-13 at 09:38 +0200, Milan Boberic wrote:
> > Hi Dario,
> > yes passtrhough is involved.
> >
> Ok.
>
> > This is everything I did so far:
> >
> > I implemented Xen Hypervisor 4.9.2 on UltraZed-EG board with carrier
> > card following these steps:
> > 1.) installed petalinux 2018.2 on Ubuntu 16.04
> > 2.) dowloaded UltraZed-EG IO Carrier Card - PetaLinux 2018.2
> > Standard BSP
> > 3.) created project: petalinux-create -t project –s <path_to_BSP>
> > 4.) copied xen-overlay.dtsi from ZCU102 project (also made with BSP)
> > to my project
> > 5.) built xen hypervisor following this guide link (booting with SD
> > card)
> > 6.) made baremetal application that blinks PS LED with Vivado
> > 7.) measured signals jitted when application is on board with and
> > with out Xen
> >
> I am not familiar with building an SD-Card image for an ARM system with
> Xen on it. What I can tell, though, is that Xen 4.9.2 does not look to
> me to include the RCU subsystem fixes that I mentioned in my reply.
>
> I don't recall why we did not backport them. It may be that they're not
> entirely trivial, and they fix a behavior which manifests only in not
> fully supported cases. Or that we (well, I :-/) forgot.
>
> I can have a look, at how challenging it is to apply the patches to
> 4.9.x (but no hard feelings if anyone beats me at it, I promise
> :-D).
>
> In the meantime, if you have the chance of trying Xen 4.10 or 4.10.1,
> which has those fixes, that would be great. In fact, in case everything
> would work with such version, that would be another clue that the RCU
> issue is the actual culprit.
>
> > I menaged to decrease jitter adding sched=null vwfi=native in xen-
> > overlay.dtsi file in xen-bootargs.
> >
> > The problem is when I add sched=null vwfi=native, I can create my
> > bare-metal application with xl create and destroy it with xl destroy
> > (it disappears from xl list) but when I want to start it again (xl
> > create) this error pops:
> >
> > root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
> > Parsing config from bm1.cfg
> > (XEN) IRQ 48 is already used by domain 1
> > libxl: error: libxl_create.c:1278:domcreate_launch_dm: Domain
> > 2:failed give domain access to irq 48: Device or resource busy
> > libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain 2:Non-
> > existant domain
> > libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain
> > 2:Unable to destroy guest
> > libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain
> > 2:Destruction of domain failed
> >
> > When I remove sched=null vwfi=native and build project again
> > everything works fine, I can create and destroy bm app as many times
> > as I want.
> >
> Yes. If you look at the email (and to the full thread) I put links to,
> you'll see that the behavior is exactly the same. It mentions the
> Credit2 scheduler not working, rather than null, but the problem is the
> same, i.e., that there is no periodic timer tick neither in Credit2 nor
> in null.
>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Software Engineer @ SUSE https://www.suse.com/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-13 15:18 ` Milan Boberic
@ 2018-09-13 17:39 ` Dario Faggioli
2018-09-14 9:21 ` Milan Boberic
0 siblings, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-13 17:39 UTC (permalink / raw)
To: Milan Boberic; +Cc: xen-devel, stefano
[-- Attachment #1.1: Type: text/plain, Size: 1887 bytes --]
On Thu, 2018-09-13 at 17:18 +0200, Milan Boberic wrote:
> Commits are there and I will definitely continue with 4.10 version.
> But it didn't solve my problem entirely.
>
> I create my bare-metal application (with xl create) and destroy it
> with xl destroy (it disappears from xl list) and when I try to create
> it again same error pops but if I immediately run xl create command
> again it creates it without error.
> If I run xl create twice fast sometimes bare-metal application isn't
> in any state (it should be in "running" state).
> If I wait some time (approximately between 30 and 90 seconds) after
> destruction of that bm app and then run xl create it will create it
> without error.
>
Ok, thanks for trying and reporting back.
If possible, help me understand things a bit better.
So, can you confirm that the situation _actually_improves_ with Xen
4.10 ?
Basically, as far as I've understood things, with Xen 4.9, you destroy
a guest, and you can _never_ re-create it, not even after 30 seconds,
90 seconds, 2 minutes, 1 hour, ecc. Is that correct?
With Xen 4.10, it may still fail, if you try to re-create it within ~30
to 90 seconds, but after that, it works. Is that also correct?
I need to know this, because I want to understand if the issue is, at
least partially, cured by the RCU fixes, although having to wait 30
seconds is definitely not what I was expecting (i.e., there might be
something else).
Another question, in case I manage to produce a debug patch, are you ok
to put it in your source tree, build with it, and tell us what you see?
Thanks again and Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-13 17:39 ` Dario Faggioli
@ 2018-09-14 9:21 ` Milan Boberic
2018-09-20 13:04 ` Milan Boberic
0 siblings, 1 reply; 26+ messages in thread
From: Milan Boberic @ 2018-09-14 9:21 UTC (permalink / raw)
To: dfaggioli; +Cc: xen-devel, stefano
[-- Attachment #1: Type: text/plain, Size: 6814 bytes --]
Thank you for taking your time to deal with this problem.
I did more testing just to be sure and I also measured time (using
stopwatch on my phone which isn't precise at all, just wanted You to
get the feeling of what time intervals are we talking about).
Yes, I can confirm that that situation actually improves with Xen
4.10, which is why I'm going to continue to use it.
With Xen 4.9.2 after I create a guest and destroy it (note that it is
a guest with pass through which blinks GPIO PS LED) I can't re-create
it again. Never. Not even after 30 seconds, 2 minutes, 5 minutes,
etc...
These are testing results with Xen 4.10:
1.) I created a guest, destroyed it and immediately after that tried
to create it again (manualy, over keyboard, for that I need maybe half
a second or a second to hit twice "arrow up" and "enter" buttons on
keyboard) and this shows:
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
root@uz3eg-iocc-2018-2:~# xl destroy bm1
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
(XEN) IRQ 48 is already used by domain 27
libxl: error: libxl_create.c:1325:domcreate_launch_dm: Domain
28:failed give domain access to irq 48: Device or resource busy
libxl: error: libxl_domain.c:1000:libxl__destroy_domid: Domain
28:Non-existant domain
libxl: error: libxl_domain.c:959:domain_destroy_callback: Domain
28:Unable to destroy guest
libxl: error: libxl_domain.c:886:domain_destroy_cb: Domain
28:Destruction of domain failed
2.) Here I createed a guest, destroyed it and then immediately ran xl
create twice, fast. For that I also need like half a second or second.
Note that guest isn't in any state, is should be in "running" state
because I need that PS LED to blink.
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
root@uz3eg-iocc-2018-2:~# xl destroy bm1
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
(XEN) IRQ 48 is already used by domain 32
libxl: error: libxl_create.c:1325:domcreate_launch_dm: Domain
33:failed give domain access to irq 48: Device or resource busy
libxl: error: libxl_domain.c:1000:libxl__destroy_domid: Domain
33:Non-existant domain
libxl: error: libxl_domain.c:959:domain_destroy_callback: Domain
33:Unable to destroy guest
libxl: error: libxl_domain.c:886:domain_destroy_cb: Domain
33:Destruction of domain failed
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
root@uz3eg-iocc-2018-2:~# xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 768 1
r----- 1936.2
bm1 34 8 1
------ 0.0
3.) Here I did same thing like in 2.) except I waited 6-7 seconds
after error pops and then ran xl create and guest worked fine (it is
in "running state"), so:
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
root@uz3eg-iocc-2018-2:~# xl destroy bm1
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
(XEN) IRQ 48 is already used by domain 57
libxl: error: libxl_create.c:1325:domcreate_launch_dm: Domain
58:failed give domain access to irq 48: Device or resource busy
libxl: error: libxl_domain.c:1000:libxl__destroy_domid: Domain
58:Non-existant domain
libxl: error: libxl_domain.c:959:domain_destroy_callback: Domain
58:Unable to destroy guest
libxl: error: libxl_domain.c:886:domain_destroy_cb: Domain
58:Destruction of domain failed
/* waited for approximately 6-7 seconds and then ran command bellow */
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
root@uz3eg-iocc-2018-2:~# xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 768 1 r----- 3071.5
bm1 59 8 1
r----- 8.2
4.) Here I createed a guest, destroyed it and then waited for
approximately 7 seconds and then ran xl create and everything worked
fine:
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
root@uz3eg-iocc-2018-2:~# xl destroy bm1
/* waited for approximately 7 seconds and then ran command bellow */
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
root@uz3eg-iocc-2018-2:~# xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 768 1 r----- 3641.1
bm1 70 8 1
r----- 7.1
It looks like guest needs approximately 7 seconds to be fully
destroyed and to fully release IRQ.
And yes, if you menage to produce patch I will put it in my source
tree, build with it, test it and send you back the results.
In attachment I included dmesg, xl dmesg from xen 4.10.
On Thu, Sep 13, 2018 at 7:39 PM Dario Faggioli <dfaggioli@suse.com> wrote:
>
> On Thu, 2018-09-13 at 17:18 +0200, Milan Boberic wrote:
> > Commits are there and I will definitely continue with 4.10 version.
> > But it didn't solve my problem entirely.
> >
> > I create my bare-metal application (with xl create) and destroy it
> > with xl destroy (it disappears from xl list) and when I try to create
> > it again same error pops but if I immediately run xl create command
> > again it creates it without error.
> > If I run xl create twice fast sometimes bare-metal application isn't
> > in any state (it should be in "running" state).
> > If I wait some time (approximately between 30 and 90 seconds) after
> > destruction of that bm app and then run xl create it will create it
> > without error.
> >
> Ok, thanks for trying and reporting back.
>
> If possible, help me understand things a bit better.
>
> So, can you confirm that the situation _actually_improves_ with Xen
> 4.10 ?
>
> Basically, as far as I've understood things, with Xen 4.9, you destroy
> a guest, and you can _never_ re-create it, not even after 30 seconds,
> 90 seconds, 2 minutes, 1 hour, ecc. Is that correct?
>
> With Xen 4.10, it may still fail, if you try to re-create it within ~30
> to 90 seconds, but after that, it works. Is that also correct?
>
> I need to know this, because I want to understand if the issue is, at
> least partially, cured by the RCU fixes, although having to wait 30
> seconds is definitely not what I was expecting (i.e., there might be
> something else).
>
> Another question, in case I manage to produce a debug patch, are you ok
> to put it in your source tree, build with it, and tell us what you see?
>
> Thanks again and Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #2: dmesg XEN 4.10.txt --]
[-- Type: text/plain, Size: 24484 bytes --]
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.0-xilinx-v2018.2 (oe-user@oe-host) (gcc version 7.2.0 (GCC)) #1 SMP Thu Sep 13 15:01:54 CEST 2018
[ 0.000000] Boot CPU: AArch64 Processor [410fd034]
[ 0.000000] Machine model: xlnx,zynqmp
[ 0.000000] Xen 4.10 support found
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 256 MiB at 0x0000000060000000
[ 0.000000] On node 0 totalpages: 196608
[ 0.000000] DMA zone: 2688 pages used for memmap
[ 0.000000] DMA zone: 0 pages reserved
[ 0.000000] DMA zone: 196608 pages, LIFO batch:31
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.1 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: Trusted OS migration not required
[ 0.000000] random: fast init done
[ 0.000000] percpu: Embedded 21 pages/cpu @ffffffc03ffb7000 s46488 r8192 d31336 u86016
[ 0.000000] pcpu-alloc: s46488 r8192 d31336 u86016 alloc=21*4096
[ 0.000000] pcpu-alloc: [0] 0
[ 0.000000] Detected VIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for ARM erratum 845719
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 193920
[ 0.000000] Kernel command line: console=hvc0 earlycon=xen earlyprintk=xen maxcpus=1 clk_ignore_unused
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.000000] Memory: 423788K/786432K available (9980K kernel code, 644K rwdata, 3132K rodata, 512K init, 2168K bss, 100500K reserved, 262144K cma-reserved)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] modules : 0xffffff8000000000 - 0xffffff8008000000 ( 128 MB)
[ 0.000000] vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000 ( 250 GB)
[ 0.000000] .text : 0xffffff8008080000 - 0xffffff8008a40000 ( 9984 KB)
[ 0.000000] .rodata : 0xffffff8008a40000 - 0xffffff8008d60000 ( 3200 KB)
[ 0.000000] .init : 0xffffff8008d60000 - 0xffffff8008de0000 ( 512 KB)
[ 0.000000] .data : 0xffffff8008de0000 - 0xffffff8008e81200 ( 645 KB)
[ 0.000000] .bss : 0xffffff8008e81200 - 0xffffff800909f2b0 ( 2169 KB)
[ 0.000000] fixed : 0xffffffbefe7fd000 - 0xffffffbefec00000 ( 4108 KB)
[ 0.000000] PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000 ( 16 MB)
[ 0.000000] vmemmap : 0xffffffbf00000000 - 0xffffffc000000000 ( 4 GB maximum)
[ 0.000000] 0xffffffbf00700000 - 0xffffffbf01880000 ( 17 MB actual)
[ 0.000000] memory : 0xffffffc020000000 - 0xffffffc070000000 ( 1280 MB)
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU event tracing is enabled.
[ 0.000000] RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] arch_timer: cp15 timer(s) running at 99.99MHz (virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171015c90f, max_idle_ns: 440795203080 ns
[ 0.000003] sched_clock: 56 bits at 99MHz, resolution 10ns, wraps every 4398046511101ns
[ 0.000291] Console: colour dummy device 80x25
[ 0.283130] console [hvc0] enabled
[ 0.286601] Calibrating delay loop (skipped), value calculated using timer frequency.. 199.99 BogoMIPS (lpj=399996)
[ 0.297057] pid_max: default: 32768 minimum: 301
[ 0.301817] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.308482] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.316407] ASID allocator initialised with 65536 entries
[ 0.321590] xen:grant_table: Grant tables using version 1 layout
[ 0.327182] Grant table initialized
[ 0.330727] xen:events: Using FIFO-based ABI
[ 0.335051] Xen: initializing cpu0
[ 0.338558] Hierarchical SRCU implementation.
[ 0.343218] EFI services will not be available.
[ 0.347512] zynqmp_plat_init Platform Management API v1.0
[ 0.352941] zynqmp_plat_init Trustzone version v1.0
[ 0.357916] smp: Bringing up secondary CPUs ...
[ 0.362454] smp: Brought up 1 node, 1 CPU
[ 0.366519] SMP: Total of 1 processors activated.
[ 0.371278] CPU features: detected feature: 32-bit EL0 Support
[ 0.377161] CPU: All CPU(s) started at EL1
[ 0.381319] alternatives: patching kernel code
[ 0.386222] devtmpfs: initialized
[ 0.392866] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.398964] futex hash table entries: 256 (order: 3, 32768 bytes)
[ 0.411052] xor: measuring software checksum speed
[ 0.449281] 8regs : 2111.000 MB/sec
[ 0.489338] 8regs_prefetch: 1882.000 MB/sec
[ 0.529393] 32regs : 2594.000 MB/sec
[ 0.569451] 32regs_prefetch: 2180.000 MB/sec
[ 0.569489] xor: using function: 32regs (2594.000 MB/sec)
[ 0.574046] pinctrl core: initialized pinctrl subsystem
[ 0.580519] NET: Registered protocol family 16
[ 0.585126] vdso: 2 pages (1 code @ ffffff8008a46000, 1 data @ ffffff8008de4000)
[ 0.591194] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.599062] DMA: preallocated 256 KiB pool for atomic allocations
[ 0.604220] xen:swiotlb_xen: Warning: only able to allocate 4 MB for software IO TLB
[ 0.613071] software IO TLB [mem 0x3d400000-0x3d800000] (4MB) mapped at [ffffffc03d400000-ffffffc03d7fffff]
[ 0.654888] reset_zynqmp reset-controller: Xilinx zynqmp reset driver probed
[ 0.656876] ARM CCI_400_r1 PMU driver probed
[ 0.662209] zynqmp-pinctrl ff180000.pinctrl: zynqmp pinctrl initialized
[ 0.698249] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.764079] raid6: int64x1 gen() 369 MB/s
[ 0.832096] raid6: int64x1 xor() 408 MB/s
[ 0.900205] raid6: int64x2 gen() 630 MB/s
[ 0.968363] raid6: int64x2 xor() 553 MB/s
[ 1.036434] raid6: int64x4 gen() 956 MB/s
[ 1.104556] raid6: int64x4 xor() 680 MB/s
[ 1.172688] raid6: int64x8 gen() 898 MB/s
[ 1.240787] raid6: int64x8 xor() 683 MB/s
[ 1.308971] raid6: neonx1 gen() 666 MB/s
[ 1.377040] raid6: neonx1 xor() 782 MB/s
[ 1.445170] raid6: neonx2 gen() 1072 MB/s
[ 1.513240] raid6: neonx2 xor() 1101 MB/s
[ 1.581375] raid6: neonx4 gen() 1377 MB/s
[ 1.649468] raid6: neonx4 xor() 1317 MB/s
[ 1.717570] raid6: neonx8 gen() 1510 MB/s
[ 1.785688] raid6: neonx8 xor() 1398 MB/s
[ 1.785724] raid6: using algorithm neonx8 gen() 1510 MB/s
[ 1.789856] raid6: .... xor() 1398 MB/s, rmw enabled
[ 1.794871] raid6: using neon recovery algorithm
[ 1.800812] XGpio: /amba_pl@0/gpio@80000000: registered, base is 504
[ 1.806763] XGpio: /amba_pl@0/gpio@80001000: registered, base is 496
[ 1.812725] XGpio: /amba_pl@0/gpio@80002000: registered, base is 493
[ 1.819148] xen:balloon: Initialising balloon driver
[ 1.824179] SCSI subsystem initialized
[ 1.827598] libata version 3.00 loaded.
[ 1.827726] usbcore: registered new interface driver usbfs
[ 1.833137] usbcore: registered new interface driver hub
[ 1.838496] usbcore: registered new device driver usb
[ 1.843632] media: Linux media interface: v0.10
[ 1.848176] Linux video capture interface: v2.00
[ 1.852861] pps_core: LinuxPPS API ver. 1 registered
[ 1.857845] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 1.867023] PTP clock support registered
[ 1.871007] EDAC MC: Ver: 3.0.0
[ 1.876596] zynqmp-ipi ff9905c0.mailbox: Probed ZynqMP IPI Mailbox driver.
[ 1.881289] FPGA manager framework
[ 1.884672] fpga-region fpga-full: FPGA Region probed
[ 1.889763] Advanced Linux Sound Architecture Driver Initialized.
[ 1.896083] Bluetooth: Core ver 2.22
[ 1.899475] NET: Registered protocol family 31
[ 1.903951] Bluetooth: HCI device and connection manager initialized
[ 1.910354] Bluetooth: HCI socket layer initialized
[ 1.915283] Bluetooth: L2CAP socket layer initialized
[ 1.920397] Bluetooth: SCO socket layer initialized
[ 1.927168] clocksource: Switched to clocksource arch_sys_counter
[ 1.931555] VFS: Disk quotas dquot_6.6.0
[ 1.935482] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 1.950007] NET: Registered protocol family 2
[ 1.950399] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[ 1.955988] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[ 1.962624] TCP: Hash tables configured (established 8192 bind 8192)
[ 1.968973] UDP hash table entries: 512 (order: 2, 16384 bytes)
[ 1.974886] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[ 1.981362] NET: Registered protocol family 1
[ 1.986582] RPC: Registered named UNIX socket transport module.
[ 1.991653] RPC: Registered udp transport module.
[ 1.996404] RPC: Registered tcp transport module.
[ 2.001160] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 2.007652] PCI: CLS 0 bytes, default 128
[ 2.007764] Trying to unpack rootfs image as initramfs...
[ 4.324257] Freeing initrd memory: 53404K
[ 4.325239] audit: initializing netlink subsys (disabled)
[ 4.329065] audit: type=2000 audit(4.092:1): state=initialized audit_enabled=0 res=1
[ 4.336137] workingset: timestamp_bits=62 max_order=18 bucket_order=0
[ 4.343286] NFS: Registering the id_resolver key type
[ 4.347591] Key type id_resolver registered
[ 4.351808] Key type id_legacy registered
[ 4.355877] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 4.362631] jffs2: version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc.
[ 4.398785] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[ 4.400638] io scheduler noop registered
[ 4.404614] io scheduler deadline registered
[ 4.408955] io scheduler cfq registered (default)
[ 4.413696] io scheduler mq-deadline registered
[ 4.418280] io scheduler kyber registered
[ 4.423339] OF: /amba/dma@fd500000: could not find phandle
[ 4.428197] xilinx-zynqmp-dma fd500000.dma: ZynqMP DMA driver Probe success
[ 4.434929] OF: /amba/dma@fd510000: could not find phandle
[ 4.440546] xilinx-zynqmp-dma fd510000.dma: ZynqMP DMA driver Probe success
[ 4.447464] OF: /amba/dma@fd520000: could not find phandle
[ 4.453080] xilinx-zynqmp-dma fd520000.dma: ZynqMP DMA driver Probe success
[ 4.460009] OF: /amba/dma@fd530000: could not find phandle
[ 4.465626] xilinx-zynqmp-dma fd530000.dma: ZynqMP DMA driver Probe success
[ 4.472549] OF: /amba/dma@fd540000: could not find phandle
[ 4.478175] xilinx-zynqmp-dma fd540000.dma: ZynqMP DMA driver Probe success
[ 4.485091] OF: /amba/dma@fd550000: could not find phandle
[ 4.490715] xilinx-zynqmp-dma fd550000.dma: ZynqMP DMA driver Probe success
[ 4.497633] OF: /amba/dma@fd560000: could not find phandle
[ 4.503256] xilinx-zynqmp-dma fd560000.dma: ZynqMP DMA driver Probe success
[ 4.510177] OF: /amba/dma@fd570000: could not find phandle
[ 4.515796] xilinx-zynqmp-dma fd570000.dma: ZynqMP DMA driver Probe success
[ 4.522911] xilinx-zynqmp-dma ffa80000.dma: ZynqMP DMA driver Probe success
[ 4.529847] xilinx-zynqmp-dma ffa90000.dma: ZynqMP DMA driver Probe success
[ 4.536845] xilinx-zynqmp-dma ffaa0000.dma: ZynqMP DMA driver Probe success
[ 4.543852] xilinx-zynqmp-dma ffab0000.dma: ZynqMP DMA driver Probe success
[ 4.550863] xilinx-zynqmp-dma ffac0000.dma: ZynqMP DMA driver Probe success
[ 4.557867] xilinx-zynqmp-dma ffad0000.dma: ZynqMP DMA driver Probe success
[ 4.564874] xilinx-zynqmp-dma ffae0000.dma: ZynqMP DMA driver Probe success
[ 4.571881] xilinx-zynqmp-dma ffaf0000.dma: ZynqMP DMA driver Probe success
[ 4.581873] xen:xen_evtchn: Event-channel device installed
[ 4.636361] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 4.641223] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 4.649880] brd: module loaded
[ 4.654505] loop: module loaded
[ 4.654552] Invalid max_queues (4), will use default max: 1.
[ 4.658573] OF: /amba/ahci@fd0c0000: could not find phandle
[ 4.663756] ahci-ceva fd0c0000.ahci: AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl platform mode
[ 4.672453] ahci-ceva fd0c0000.ahci: flags: 64bit ncq sntf pm clo only pmp fbs pio slum part ccc sds apst
[ 4.684336] scsi host0: ahci-ceva
[ 4.685762] scsi host1: ahci-ceva
[ 4.688998] ata1: SATA max UDMA/133 mmio [mem 0xfd0c0000-0xfd0c1fff] port 0x100 irq 32
[ 4.696842] ata2: SATA max UDMA/133 mmio [mem 0xfd0c0000-0xfd0c1fff] port 0x180 irq 32
[ 4.704947] mtdoops: mtd device (mtddev=name/number) must be supplied
[ 4.711608] OF: /amba/spi@ff0f0000: could not find phandle
[ 4.718089] m25p80 spi0.0: found n25q256a, expected m25p80
[ 4.722720] m25p80 spi0.0: n25q256a (65536 Kbytes)
[ 4.727227] 3 ofpart partitions found on MTD device spi0.0
[ 4.732737] Creating 3 MTD partitions on "spi0.0":
[ 4.737585] 0x000000000000-0x000001360000 : "boot"
[ 4.742967] 0x000001360000-0x0000013a0000 : "bootenv"
[ 4.748108] 0x0000013a0000-0x000002aa0000 : "kernel"
[ 4.754008] libphy: Fixed MDIO Bus: probed
[ 4.757760] tun: Universal TUN/TAP device driver, 1.6
[ 4.768363] CAN device driver interface
[ 4.768803] OF: /amba/ethernet@ff0e0000: could not find phandle
[ 4.772990] macb ff0e0000.ethernet: Not enabling partial store and forward
[ 4.780062] libphy: MACB_mii_bus: probed
[ 4.787810] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 25 (00:0a:35:00:22:01)
[ 4.793405] TI DP83867 ff0e0000.ethernet-ffffffff:09: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0e0000.ethernet-ffffffff:09, irq=POLL)
[ 4.807018] xen_netfront: Initialising Xen virtual ethernet driver
[ 4.812761] usbcore: registered new interface driver asix
[ 4.818170] usbcore: registered new interface driver ax88179_178a
[ 4.824303] usbcore: registered new interface driver cdc_ether
[ 4.830187] usbcore: registered new interface driver net1080
[ 4.835894] usbcore: registered new interface driver cdc_subset
[ 4.841862] usbcore: registered new interface driver zaurus
[ 4.847494] usbcore: registered new interface driver cdc_ncm
[ 4.854659] xilinx-axipmon ffa00000.perf-monitor: Probed Xilinx APM
[ 4.859885] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 4.866062] ehci-pci: EHCI PCI platform driver
[ 4.870826] usbcore: registered new interface driver cdc_acm
[ 4.876268] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[ 4.884338] usbcore: registered new interface driver uas
[ 4.889706] usbcore: registered new interface driver usb-storage
[ 4.897032] rtc_zynqmp ffa60000.rtc: rtc core: registered ffa60000.rtc as rtc0
[ 4.903052] i2c /dev entries driver
[ 4.907505] cdns-i2c ff030000.i2c: 400 kHz mmio ff030000 irq 26
[ 4.939776] i2c i2c-0: Added multiplexed i2c bus 1
[ 4.940022] i2c i2c-0: Added multiplexed i2c bus 2
[ 4.943869] pca954x 0-0070: registered 2 multiplexed busses for I2C mux pca9542
[ 4.951299] IR NEC protocol handler initialized
[ 4.955798] IR RC5(x/sz) protocol handler initialized
[ 4.960900] IR RC6 protocol handler initialized
[ 4.965484] IR JVC protocol handler initialized
[ 4.970068] IR Sony protocol handler initialized
[ 4.974739] IR SANYO protocol handler initialized
[ 4.979497] IR Sharp protocol handler initialized
[ 4.984254] IR MCE Keyboard/mouse protocol handler initialized
[ 4.990136] IR XMP protocol handler initialized
[ 4.995725] usbcore: registered new interface driver uvcvideo
[ 5.000518] USB Video Class driver (1.1.1)
[ 5.006677] cdns-wdt fd4d0000.watchdog: Xilinx Watchdog Timer at ffffff800917d000 with timeout 10s
[ 5.014081] Bluetooth: HCI UART driver ver 2.3
[ 5.018171] Bluetooth: HCI UART protocol H4 registered
[ 5.023359] Bluetooth: HCI UART protocol BCSP registered
[ 5.028736] Bluetooth: HCI UART protocol LL registered
[ 5.033907] Bluetooth: HCI UART protocol ATH3K registered
[ 5.039356] Bluetooth: HCI UART protocol Three-wire (H5) registered
[ 5.045708] Bluetooth: HCI UART protocol Intel registered
[ 5.051121] Bluetooth: HCI UART protocol QCA registered
[ 5.056430] usbcore: registered new interface driver bcm203x
[ 5.062136] usbcore: registered new interface driver bpa10x
[ 5.067761] usbcore: registered new interface driver bfusb
[ 5.073294] usbcore: registered new interface driver btusb
[ 5.078801] Bluetooth: Generic Bluetooth SDIO driver ver 0.1
[ 5.084554] usbcore: registered new interface driver ath3k
[ 5.090171] EDAC MC: ECC not enabled
[ 5.093832] EDAC DEVICE0: Giving out device to module zynqmp-ocm-edac controller zynqmp_ocm: DEV ff960000.memory-controller (INTERRUPT)
[ 5.106143] cpu cpu0: failed to get clock: -2
[ 5.110294] cpufreq-dt: probe of cpufreq-dt failed with error -2
[ 5.116482] sdhci: Secure Digital Host Controller Interface driver
[ 5.122571] sdhci: Copyright(c) Pierre Ossman
[ 5.126981] sdhci-pltfm: SDHCI platform and OF driver helper
[ 5.132796] OF: /amba/sdhci@ff160000: could not find phandle
[ 5.139212] ata2: SATA link down (SStatus 0 SControl 330)
[ 5.143885] ata1: SATA link down (SStatus 0 SControl 330)
[ 5.195153] mmc0: SDHCI controller on ff160000.sdhci [ff160000.sdhci] using ADMA 64-bit
[ 5.197642] PLL: shutdown
[ 5.200372] PLL: enable
[ 5.208514] OF: /amba/sdhci@ff170000: could not find phandle
[ 5.251150] mmc1: SDHCI controller on ff170000.sdhci [ff170000.sdhci] using ADMA 64-bit
[ 5.259548] ledtrig-cpu: registered to indicate activity on CPUs
[ 5.260188] usbcore: registered new interface driver usbhid
[ 5.265627] usbhid: USB HID core driver
[ 5.278353] fpga_manager fpga0: Xilinx ZynqMP FPGA Manager registered
[ 5.280716] pktgen: Packet Generator for packet performance testing. Version: 2.75
[ 5.287535] Netfilter messages via NETLINK v0.30.
[ 5.291744] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 5.297012] Initializing XFRM netlink socket
[ 5.301358] NET: Registered protocol family 10
[ 5.306305] Segment Routing with IPv6
[ 5.309565] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 5.314995] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 5.321249] NET: Registered protocol family 17
[ 5.325441] NET: Registered protocol family 15
[ 5.329941] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 5.342905] Ebtables v2.0 registered
[ 5.352191] can: controller area network core (rev 20170425 abi 9)
[ 5.352859] NET: Registered protocol family 29
[ 5.357318] can: raw protocol (rev 20170425)
[ 5.361641] can: broadcast manager protocol (rev 20170425 t)
[ 5.367353] can: netlink gateway (rev 20170425) max_hops=1
[ 5.372989] Bluetooth: RFCOMM TTY layer initialized
[ 5.377833] Bluetooth: RFCOMM socket layer initialized
[ 5.383014] Bluetooth: RFCOMM ver 1.11
[ 5.386816] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 5.392175] Bluetooth: BNEP filters: protocol multicast
[ 5.397455] Bluetooth: BNEP socket layer initialized
[ 5.402469] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[ 5.408440] Bluetooth: HIDP socket layer initialized
[ 5.413568] 9pnet: Installing 9P2000 support
[ 5.417794] Key type dns_resolver registered
[ 5.422399] mmc0: new HS200 MMC card at address 0001
[ 5.427474] mmcblk0: mmc0:0001 Q2J55L 7.09 GiB
[ 5.431789] mmcblk0boot0: mmc0:0001 Q2J55L partition 1 16.0 MiB
[ 5.437761] mmcblk0boot1: mmc0:0001 Q2J55L partition 2 16.0 MiB
[ 5.443751] mmcblk0rpmb: mmc0:0001 Q2J55L partition 3 4.00 MiB
[ 5.450232] registered taskstats version 1
[ 5.454101] mmcblk0: p1
[ 5.457841] Btrfs loaded, crc32c=crc32c-generic
[ 5.467665] dwc3-of-simple ff9d0000.usb0: dwc3_simple_set_phydata: Can't find usb3-phy
[ 5.470452] OF: /amba/usb0@ff9d0000/dwc3@fe200000: could not find phandle
[ 5.477807] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[ 5.482434] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 1
[ 5.490493] xhci-hcd xhci-hcd.0.auto: hcc params 0x0238f625 hci version 0x100 quirks 0x22010010
[ 5.498908] xhci-hcd xhci-hcd.0.auto: irq 57, io mem 0xfe200000
[ 5.504962] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[ 5.511659] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 5.518910] usb usb1: Product: xHCI Host Controller
[ 5.523840] usb usb1: Manufacturer: Linux 4.14.0-xilinx-v2018.2 xhci-hcd
[ 5.530587] usb usb1: SerialNumber: xhci-hcd.0.auto
[ 5.536501] mmc1: new high speed SDHC card at address 1234
[ 5.541382] hub 1-0:1.0: USB hub found
[ 5.544992] mmcblk1: mmc1:1234 SA08G 7.21 GiB (ro)
[ 5.549881] hub 1-0:1.0: 1 port detected
[ 5.554056] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[ 5.559239] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 2
[ 5.567034] mmcblk1: p1
[ 5.570000] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[ 5.577740] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[ 5.584484] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 5.591745] usb usb2: Product: xHCI Host Controller
[ 5.596675] usb usb2: Manufacturer: Linux 4.14.0-xilinx-v2018.2 xhci-hcd
[ 5.603421] usb usb2: SerialNumber: xhci-hcd.0.auto
[ 5.608620] hub 2-0:1.0: USB hub found
[ 5.612263] hub 2-0:1.0: 1 port detected
[ 5.617250] rtc_zynqmp ffa60000.rtc: setting system clock to 1970-01-01 00:00:16 UTC (16)
[ 5.624463] clk: Not disabling unused clocks
[ 5.628726] ALSA device list:
[ 5.631705] No soundcards found.
[ 5.635927] Freeing unused kernel memory: 512K
[ 5.715392] udevd[1717]: starting version 3.2.2
[ 5.722981] udevd[1718]: starting eudev-3.2.2
[ 6.751052] export_store: invalid GPIO 350
[ 6.751404] blinky[1981]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006, in libc-2.26.so[7f8f58e000+138000]
[ 6.763242] CPU: 0 PID: 1981 Comm: blinky Not tainted 4.14.0-xilinx-v2018.2 #1
[ 6.770489] Hardware name: xlnx,zynqmp (DT)
[ 6.774728] task: ffffffc03cad9200 task.stack: ffffff800b318000
[ 6.780697] PC is at 0x7f8f5efabc
[ 6.784069] LR is at 0x400dfc
[ 6.787091] pc : [<0000007f8f5efabc>] lr : [<0000000000400dfc>] pstate: 60000000
[ 6.794535] sp : 0000007fc87a8f00
[ 6.797908] x29: 0000007fc87a8f00 x28: 0000000000000000
[ 6.803272] x27: 0000000000000000 x26: 0000007fc87a9edf
[ 6.808635] x25: 0000007fc87a9014 x24: 0000000000401140
[ 6.813998] x23: 0000000000412000 x22: 0000000000000003
[ 6.819361] x21: 0000000000000001 x20: 0000007fc87a8f68
[ 6.824724] x19: 0000000000000000 x18: 0000007fc87a8cad
[ 6.830087] x17: 0000007f8f5efaa0 x16: 0000000000412058
[ 6.835450] x15: 000000000000000a x14: 000000000000015e
[ 6.840813] x13: 0000000000000000 x12: 0000000000000000
[ 6.846176] x11: 0000000000000020 x10: 0000007fc87a8cb0
[ 6.851539] x9 : 0000000000000000 x8 : 0000000000000038
[ 6.856902] x7 : 0000000000000000 x6 : 0000000000401126
[ 6.862265] x5 : 0000000000000000 x4 : 000000002d3b4260
[ 6.867628] x3 : 0000000000000000 x2 : 0000000000000001
[ 6.872991] x1 : 0000000000000001 x0 : 0000007fc87a8f68
[ 7.075663] pps pps0: new PPS source ptp0
[ 7.075722] macb ff0e0000.ethernet: gem-ptp-timer ptp clock registered.
[ 7.080863] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 9.099432] macb ff0e0000.ethernet eth0: link up (1000/Full)
[ 9.099552] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 222.035394] random: crng init done
[-- Attachment #3: xl dmesg XEN 4.10.txt --]
[-- Type: text/plain, Size: 5550 bytes --]
(XEN) Checking for initrd in /chosen
(XEN) Initrd 0000000002bd8000-0000000005fffeaf
(XEN) RAM: 0000000000000000 - 000000007fefffff
(XEN)
(XEN) MODULE[0]: 0000000007ff4000 - 0000000007ffc080 Device Tree
(XEN) MODULE[1]: 0000000002bd8000 - 0000000005fffeaf Ramdisk
(XEN) MODULE[2]: 0000000000080000 - 0000000003180000 Kernel
(XEN) RESVD[0]: 0000000007ff4000 - 0000000007ffc000
(XEN) RESVD[1]: 0000000002bd8000 - 0000000005fffeaf
(XEN)
(XEN) Command line: console=dtuart dtuart=serial0 dom0_mem=768M bootscrub=0 maxcpus=1 dom0_max_vcpus=1 dom0_vcpus_pin=true timer_slop=0 core_parking=performance cpufreq=xen:performance sched=null vwfi=native
(XEN) parameter "maxcpus" unknown!
(XEN) parameter "core_parking" unknown!
(XEN) parameter "cpufreq" unknown!
(XEN) Placing Xen at 0x000000007fc00000-0x000000007fe00000
(XEN) Update BOOTMOD_XEN from 0000000006000000-0000000006108d81 => 000000007fc00000-000000007fd08d81
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN) Looking for dtuart at "serial0", options ""
Xen 4.10.1-pre
(XEN) Xen version 4.10.1-pre (milan@) (aarch64-xilinx-linux-gcc (GCC) 7.2.0) debug=n Thu Sep 13 14:46:56 CEST 2018
(XEN) Latest ChangeSet: Thu Mar 22 22:02:18 2018 +0100 git:3bc83b8f57-dirty
(XEN) Processor: 410fd034: "ARM Limited", variant: 0x0, part 0xd03, rev 0x4
(XEN) 64-bit Execution:
(XEN) Processor Features: 0000000000002222 0000000000000000
(XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN) Extensions: FloatingPoint AdvancedSIMD
(XEN) Debug Features: 0000000010305106 0000000000000000
(XEN) Auxiliary Features: 0000000000000000 0000000000000000
(XEN) Memory Model Features: 0000000000001122 0000000000000000
(XEN) ISA Features: 0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN) Processor Features: 00000131:00011011
(XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN) Extensions: GenericTimer Security
(XEN) Debug Features: 03010066
(XEN) Auxiliary Features: 00000000
(XEN) Memory Model Features: 10201105 40000000 01260000 02102211
(XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 99999 KHz
(XEN) GICv2 initialization:
(XEN) gic_dist_addr=00000000f9010000
(XEN) gic_cpu_addr=00000000f9020000
(XEN) gic_hyp_addr=00000000f9040000
(XEN) gic_vcpu_addr=00000000f9060000
(XEN) gic_maintenance_irq=25
(XEN) GICv2: Adjusting CPU interface base to 0xf902f000
(XEN) GICv2: 192 lines, 4 cpus, secure (IID 0200143b).
(XEN) Using scheduler: null Scheduler (null)
(XEN) Initializing null scheduler
(XEN) WARNING: This is experimental software in development.
(XEN) Use at your own risk.
(XEN) Allocated console ring of 16 KiB.
(XEN) Bringing up CPU1
(XEN) Bringing up CPU2
(XEN) Bringing up CPU3
(XEN) Brought up 4 CPUs
(XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID
(XEN) P2M: 3 levels with order-1 root, VTCR 0x80023558
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 0000000000080000
(XEN) Loading ramdisk from boot module @ 0000000002bd8000
(XEN) Allocating 1:1 mappings totalling 768MB for dom0:
(XEN) BANK[0] 0x00000020000000-0x00000040000000 (512MB)
(XEN) BANK[1] 0x00000060000000-0x00000070000000 (256MB)
(XEN) Grant table range: 0x0000007fc00000-0x0000007fc40000
(XEN) Loading zImage from 0000000000080000 to 0000000020080000-0000000023180000
(XEN) Loading dom0 initrd from 0000000002bd8000 to 0x0000000028200000-0x000000002b627eaf
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading dom0 DTB to 0x0000000028000000-0x0000000028006e76
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 276kB init memory.
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER4
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER8
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER12
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER16
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER20
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER0
(XEN) IRQ 48 is already used by domain 2
(XEN) IRQ 48 is already used by domain 4
(XEN) IRQ 48 is already used by domain 6
(XEN) IRQ 48 is already used by domain 8
(XEN) IRQ 48 is already used by domain 11
(XEN) IRQ 48 is already used by domain 13
(XEN) IRQ 48 is already used by domain 16
(XEN) IRQ 48 is already used by domain 21
(XEN) IRQ 48 is already used by domain 25
(XEN) IRQ 48 is already used by domain 27
(XEN) IRQ 48 is already used by domain 30
(XEN) IRQ 48 is already used by domain 34
(XEN) IRQ 48 is already used by domain 36
(XEN) IRQ 48 is already used by domain 38
(XEN) IRQ 48 is already used by domain 41
(XEN) IRQ 48 is already used by domain 43
(XEN) IRQ 48 is already used by domain 45
(XEN) IRQ 48 is already used by domain 47
(XEN) IRQ 48 is already used by domain 49
(XEN) IRQ 48 is already used by domain 51
(XEN) IRQ 48 is already used by domain 53
(XEN) IRQ 48 is already used by domain 55
(XEN) IRQ 48 is already used by domain 57
(XEN) IRQ 48 is already used by domain 61
(XEN) IRQ 48 is already used by domain 66
[-- Attachment #4: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-14 9:21 ` Milan Boberic
@ 2018-09-20 13:04 ` Milan Boberic
2018-09-20 16:09 ` Dario Faggioli
0 siblings, 1 reply; 26+ messages in thread
From: Milan Boberic @ 2018-09-20 13:04 UTC (permalink / raw)
To: Dario Faggioli; +Cc: xen-devel, stefano
I ran some more tests and managed to successfully create and destroy
domU as many times as I want, without any delay between destroy and
create.
I added:
printk("End of a domain_destroy function");
in domain_destroy function and
printk("End of a complete_domain_destroy function"); in
complete_domain_destroy function, at the end of the functions.
Those functions are in domain.c file.
Now, after every xl create it prints:
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
<G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
This line never printed before but it doesn't affect anything:
<G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
I tried removing printk from functions and I got same problem like before.
Do you guys have any idea what is going on here?
Thanks in advance, best regards!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-20 13:04 ` Milan Boberic
@ 2018-09-20 16:09 ` Dario Faggioli
2018-09-21 14:14 ` Milan Boberic
0 siblings, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-20 16:09 UTC (permalink / raw)
To: Milan Boberic; +Cc: xen-devel, stefano
[-- Attachment #1.1: Type: text/plain, Size: 1899 bytes --]
Hey,
Sorry for not having followed up. I was (and still am) planning to, but
am also a bit busy.
On Thu, 2018-09-20 at 15:04 +0200, Milan Boberic wrote:
> I ran some more tests and managed to successfully create and destroy
> domU as many times as I want, without any delay between destroy and
> create.
> I added:
> printk("End of a domain_destroy function");
> in domain_destroy function and
> printk("End of a complete_domain_destroy function"); in
> complete_domain_destroy function, at the end of the functions.
> Those functions are in domain.c file.
>
Right. This is exactly the kind of debug patch I wanted to
suggest/send. It is, in fact, what was being use to diagnose/fix the
RCU issue, when it first came up, as you may have seen.
> Now, after every xl create it prints:
>
> root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
> Parsing config from bm1.cfg
> <G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
>
Mmm... So, you added a printk() (or, actually, two of them) in the
domain destruction path, and are seeing (different) things being
printed during domain creation? How nice! :-D
I'm not sure how that happens, and whether/how this new output relates
to the problem. However, what about the printk() you added? Do you see
their output? It may be visible only on the console and/or in `xl
dmesg'.
While there, if you can, I'd add timestamsp, e.g.:
printk("t=%"PRI_stime":End of a domain_destroy function", NOW());
printk("t=%"PRI_stime":End of a complete_domain_destroy function", NOW());
and check what the outpur looks like both under credit or credit2, and
under null.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-20 16:09 ` Dario Faggioli
@ 2018-09-21 14:14 ` Milan Boberic
2018-09-21 16:20 ` Dario Faggioli
0 siblings, 1 reply; 26+ messages in thread
From: Milan Boberic @ 2018-09-21 14:14 UTC (permalink / raw)
To: Dario Faggioli; +Cc: xen-devel, stefano
Hey,
yes, I can see prink's outputs on console and in xl dmesg. Also added
timestamps, here are the results (created and destroyed domU a few
times, just to get more values), this is from xl dmesg:
NULL SCHEDULER - Not stressed PetaLinux host domain.
(XEN) t=218000327743:End of a domain_destroy function
(XEN) t=218000420874:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=250216234232:End of a domain_destroy function
(XEN) t=250216326943:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom3 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=842722811648:End of a domain_destroy function
(XEN) t=842722906089:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom4 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=845884879948:End of a domain_destroy function
(XEN) t=845884972539:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom5 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=847696699336:End of a domain_destroy function
(XEN) t=847696793097:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom6 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=919260640576:End of a domain_destroy function
(XEN) t=919260732907:End of a complete_domain_destroy function
Stressed PetaLinux host with command: yes > /dev/null &
(XEN) t=3247747255872:End of a domain_destroy function
(XEN) t=3247747349863:End of a complete_domain_destroy function
(XEN) t=3253855263752:End of a domain_destroy function
(XEN) t=3253855357563:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom10 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=3256797406444:End of a domain_destroy function
(XEN) t=3256797498584:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom11 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=3259750219352:End of a domain_destroy function
(XEN) t=3259750313903:End of a complete_domain_destroy function
(XEN) <G><3>memory_map:add: dom12 gfn=ff0a0 mfn=ff0a0 nr=1
(XEN) t=3262418200662:End of a domain_destroy function
(XEN) t=3262418295912:End of a complete_domain_destroy function
CREDIT SCHEDULER - not stressed PetaLinux host
(XEN) t=86245669606:End of a domain_destroy function
(XEN) t=86245761127:End of a complete_domain_destroy function
(XEN) t=93862614736:End of a domain_destroy function
(XEN) t=93862702657:End of a complete_domain_destroy function
(XEN) t=96298736227:End of a domain_destroy function
(XEN) t=96298826618:End of a complete_domain_destroy function
(XEN) t=98042498304:End of a domain_destroy function
(XEN) t=98042590255:End of a complete_domain_destroy function
(XEN) t=99755209642:End of a domain_destroy function
(XEN) t=99755302643:End of a complete_domain_destroy function
(XEN) t=101441462434:End of a domain_destroy function
(XEN) t=101441553985:End of a complete_domain_destroy function
Stressed PetaLinux host with yes > /dev/null &
(XEN) t=331229997499:End of a domain_destroy function
(XEN) t=331230091770:End of a complete_domain_destroy function
(XEN) t=334493673726:End of a domain_destroy function
(XEN) t=334493765647:End of a complete_domain_destroy function
(XEN) t=336834521515:End of a domain_destroy function
(XEN) t=336834613326:End of a complete_domain_destroy function
(XEN) t=339215230042:End of a domain_destroy function
(XEN) t=339215321153:End of a complete_domain_destroy function
Also thank you for taking your time to help me, best regards!
On Thu, Sep 20, 2018 at 6:09 PM Dario Faggioli <dfaggioli@suse.com> wrote:
>
> Hey,
>
> Sorry for not having followed up. I was (and still am) planning to, but
> am also a bit busy.
>
> On Thu, 2018-09-20 at 15:04 +0200, Milan Boberic wrote:
> > I ran some more tests and managed to successfully create and destroy
> > domU as many times as I want, without any delay between destroy and
> > create.
> > I added:
> > printk("End of a domain_destroy function");
> > in domain_destroy function and
> > printk("End of a complete_domain_destroy function"); in
> > complete_domain_destroy function, at the end of the functions.
> > Those functions are in domain.c file.
> >
> Right. This is exactly the kind of debug patch I wanted to
> suggest/send. It is, in fact, what was being use to diagnose/fix the
> RCU issue, when it first came up, as you may have seen.
>
> > Now, after every xl create it prints:
> >
> > root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
> > Parsing config from bm1.cfg
> > <G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
> >
> Mmm... So, you added a printk() (or, actually, two of them) in the
> domain destruction path, and are seeing (different) things being
> printed during domain creation? How nice! :-D
>
> I'm not sure how that happens, and whether/how this new output relates
> to the problem. However, what about the printk() you added? Do you see
> their output? It may be visible only on the console and/or in `xl
> dmesg'.
>
> While there, if you can, I'd add timestamsp, e.g.:
>
> printk("t=%"PRI_stime":End of a domain_destroy function", NOW());
> printk("t=%"PRI_stime":End of a complete_domain_destroy function", NOW());
>
> and check what the outpur looks like both under credit or credit2, and
> under null.
>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Software Engineer @ SUSE https://www.suse.com/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-21 14:14 ` Milan Boberic
@ 2018-09-21 16:20 ` Dario Faggioli
2018-09-24 21:46 ` Julien Grall
0 siblings, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-21 16:20 UTC (permalink / raw)
To: Milan Boberic; +Cc: xen-devel, Julien Grall, stefano
[-- Attachment #1.1: Type: text/plain, Size: 2233 bytes --]
[Adding Julien as well.
Julien, this seems related to the RCU issue we fought on ARM when using
Credit2, although this is null, but it's being even more weird...]
On Fri, 2018-09-21 at 16:14 +0200, Milan Boberic wrote:
> Hey,
> yes, I can see prink's outputs on console and in xl dmesg. Also added
> timestamps, here are the results (created and destroyed domU a few
> times, just to get more values), this is from xl dmesg:
>
> NULL SCHEDULER - Not stressed PetaLinux host domain.
>
> (XEN) t=218000327743:End of a domain_destroy function
> (XEN) t=218000420874:End of a complete_domain_destroy function
> (XEN) <G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
> ...
>
> Stressed PetaLinux host with command: yes > /dev/null &
>
> (XEN) t=3247747255872:End of a domain_destroy function
> (XEN) t=3247747349863:End of a complete_domain_destroy function
> ...
>
> CREDIT SCHEDULER - not stressed PetaLinux host
>
> (XEN) t=86245669606:End of a domain_destroy function
> (XEN) t=86245761127:End of a complete_domain_destroy function
> ...
>
> Stressed PetaLinux host with yes > /dev/null &
>
> (XEN) t=331229997499:End of a domain_destroy function
> (XEN) t=331230091770:End of a complete_domain_destroy function
> ...
>
Which, if I'm doing the math properly, tells us that
complete_domain_destroy() is called within ~90us, for both schedulers,
and in all stress/load conditions. That wouldn't be too bad, I think.
And in fact, if I remember correctly, you're saying that adding the
printk()s fixes the issue in null. I wonder why that is... Can you like
kill the printks, store 5 or so of the timestamps (or just the delta)
in a static array or something, and print it from somewhere else (like
a debug-key handler in the same file)?
What I'm after, is how log, after domain_destroy(),
complete_domain_destroy() is called, and whether/how it relates the the
grace period idle timer we've added in the RCU code.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-21 16:20 ` Dario Faggioli
@ 2018-09-24 21:46 ` Julien Grall
2018-09-25 9:02 ` Dario Faggioli
0 siblings, 1 reply; 26+ messages in thread
From: Julien Grall @ 2018-09-24 21:46 UTC (permalink / raw)
To: Dario Faggioli, Milan Boberic; +Cc: xen-devel, stefano
Hi,
On 09/21/2018 05:20 PM, Dario Faggioli wrote:
> [Adding Julien as well.
>
> Julien, this seems related to the RCU issue we fought on ARM when using
> Credit2, although this is null, but it's being even more weird...]
>
> On Fri, 2018-09-21 at 16:14 +0200, Milan Boberic wrote:
>> Hey,
>> yes, I can see prink's outputs on console and in xl dmesg. Also added
>> timestamps, here are the results (created and destroyed domU a few
>> times, just to get more values), this is from xl dmesg:
>>
>> NULL SCHEDULER - Not stressed PetaLinux host domain.
>>
>> (XEN) t=218000327743:End of a domain_destroy function
>> (XEN) t=218000420874:End of a complete_domain_destroy function
>> (XEN) <G><3>memory_map:add: dom2 gfn=ff0a0 mfn=ff0a0 nr=1
>> ...
>>
>> Stressed PetaLinux host with command: yes > /dev/null &
>>
>> (XEN) t=3247747255872:End of a domain_destroy function
>> (XEN) t=3247747349863:End of a complete_domain_destroy function
>> ...
>>
>> CREDIT SCHEDULER - not stressed PetaLinux host
>>
>> (XEN) t=86245669606:End of a domain_destroy function
>> (XEN) t=86245761127:End of a complete_domain_destroy function
>> ...
>>
>> Stressed PetaLinux host with yes > /dev/null &
>>
>> (XEN) t=331229997499:End of a domain_destroy function
>> (XEN) t=331230091770:End of a complete_domain_destroy function
>> ...
>>
> Which, if I'm doing the math properly, tells us that
> complete_domain_destroy() is called within ~90us, for both schedulers,
> and in all stress/load conditions. That wouldn't be too bad, I think.
>
> And in fact, if I remember correctly, you're saying that adding the
> printk()s fixes the issue in null. I wonder why that is... Can you like
> kill the printks, store 5 or so of the timestamps (or just the delta)
> in a static array or something, and print it from somewhere else (like
> a debug-key handler in the same file)?
I think you may end up to receive an interrupt on CPU0 when using
printk. What is the mapping between vCPU and pCPU on your system?
>
> What I'm after, is how log, after domain_destroy(),
> complete_domain_destroy() is called, and whether/how it relates the the
> grace period idle timer we've added in the RCU code.
NULL scheduler and vwfi=native will inevitably introduce a latency when
destroying a domain. vwfi=native means the guest will not trap when it
has nothing to do and switch to the idle vCPU. So, in such
configuration, it is extremely unlikely the execute the idle_loop or
even enter in the hypervisor unless there are an interrupt on that pCPU.
Per my understanding of call_rcu, the calls will be queued until the RCU
reached a threshold. We don't have many place where call_rcu is called,
so reaching the threeshold may just never happen. But nothing will tell
that vCPU to go in Xen and say "I am done with RCU". Did I miss anything?
I have the feeling the problem might just be exacerbated the problem
(simlar to the idle bug with credit2) by vwfi=ative. Milan, would it be
possible to run the test without that option?
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-24 21:46 ` Julien Grall
@ 2018-09-25 9:02 ` Dario Faggioli
2018-09-25 10:12 ` Milan Boberic
2018-09-25 11:15 ` Julien Grall
0 siblings, 2 replies; 26+ messages in thread
From: Dario Faggioli @ 2018-09-25 9:02 UTC (permalink / raw)
To: Julien Grall, Milan Boberic; +Cc: xen-devel, stefano
[-- Attachment #1.1: Type: text/plain, Size: 2104 bytes --]
On Mon, 2018-09-24 at 22:46 +0100, Julien Grall wrote:
> On 09/21/2018 05:20 PM, Dario Faggioli wrote:
> >
> > What I'm after, is how log, after domain_destroy(),
> > complete_domain_destroy() is called, and whether/how it relates the
> > the
> > grace period idle timer we've added in the RCU code.
>
> NULL scheduler and vwfi=native will inevitably introduce a latency
> when
> destroying a domain. vwfi=native means the guest will not trap when
> it
> has nothing to do and switch to the idle vCPU. So, in such
> configuration, it is extremely unlikely the execute the idle_loop or
> even enter in the hypervisor unless there are an interrupt on that
> pCPU.
>
Ah! I'm not familiar with wfi=native --and in fact I was completely
ignoring it-- but this analysis makes sense to me.
> Per my understanding of call_rcu, the calls will be queued until the
> RCU
> reached a threshold. We don't have many place where call_rcu is
> called,
> so reaching the threeshold may just never happen. But nothing will
> tell
> that vCPU to go in Xen and say "I am done with RCU". Did I miss
> anything?
>
Yeah, and in fact we added the timer _but_, in this case, it does not
look that the timer is firing. It looks much more like "some random
interrupt happens", as you're suggesting. OTOH, in the case where there
are no printk()s, it might be that the timer does fire, but the vcpu
has not gone through Xen, so the grace period is, as far as we know,
not expired yet (which is also in accordance with Julien's analysis, as
far as I understood it).
> I have the feeling the problem might just be exacerbated the problem
> (simlar to the idle bug with credit2) by vwfi=ative. Milan, would it
> be
> possible to run the test without that option?
>
Indeed. And thanks, Julien, for having a look! :-)
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-25 9:02 ` Dario Faggioli
@ 2018-09-25 10:12 ` Milan Boberic
2018-09-25 10:56 ` Julien Grall
2018-09-25 11:15 ` Julien Grall
1 sibling, 1 reply; 26+ messages in thread
From: Milan Boberic @ 2018-09-25 10:12 UTC (permalink / raw)
To: Dario Faggioli; +Cc: xen-devel, julien.grall, stefano
[-- Attachment #1: Type: text/plain, Size: 1045 bytes --]
Hello guys,
mapping on my system is:
dom0 have one vCPU and it is pinned on pCPU0
domU also have one vCPU and it's pinned for pCPU2
I removed only vwfi=native and everything works fine. I can destroy
and create a guest as many times as I want with out any error (still
using sched=null).
These are xen bootargs in Xen-overlay.dtsi file:
xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=768M
bootscrub=0 maxcpus=1 dom0_max_vcpus=1 dom0_vcpus_pin=true
timer_slop=0 core_parking=performance cpufreq=xen:performance
sched=null";
There is whole xen-overlay.dtsi file included in attachment.
Purpose of my work is implementing xen on UltraZed-EG board with
maximum performance (and lowest jitter, which is why I'm using null
scheduler and "hard" vcpu pinning) which is my master's thesis on
faculty. By removing vwfi=native I'm not getting the best performance,
right? vwfi=native should decrease interrupt latency by ~60% as I read
here:
https://blog.xenproject.org/author/stefano-stabellini/
Big thanks to Julien for having a look!
[-- Attachment #2: xen-overlay.txt --]
[-- Type: text/plain, Size: 1201 bytes --]
/ {
chosen {
#address-cells = <2>;
#size-cells = <1>;
xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=768M bootscrub=0 maxcpus=1 dom0_max_vcpus=1 dom0_vcpus_pin=true timer_slop=0 core_parking=performance cpufreq=xen:performance sched=null";
xen,dom0-bootargs = "console=hvc0 earlycon=xen earlyprintk=xen maxcpus=1 clk_ignore_unused";
dom0 {
compatible = "xen,linux-zimage", "xen,multiboot-module";
reg = <0x0 0x80000 0x3100000>;
};
};
};
&smmu {
status = "okay";
mmu-masters = < &gem0 0x874
&gem1 0x875
&gem2 0x876
&gem3 0x877
&dwc3_0 0x860
&dwc3_1 0x861
&qspi 0x873
&lpd_dma_chan1 0x868
&lpd_dma_chan2 0x869
&lpd_dma_chan3 0x86a
&lpd_dma_chan4 0x86b
&lpd_dma_chan5 0x86c
&lpd_dma_chan6 0x86d
&lpd_dma_chan7 0x86e
&lpd_dma_chan8 0x86f
&fpd_dma_chan1 0x14e8
&fpd_dma_chan2 0x14e9
&fpd_dma_chan3 0x14ea
&fpd_dma_chan4 0x14eb
&fpd_dma_chan5 0x14ec
&fpd_dma_chan6 0x14ed
&fpd_dma_chan7 0x14ee
&fpd_dma_chan8 0x14ef
&sdhci0 0x870
&sdhci1 0x871
&nand0 0x872>;
};
&uart1 {
xen,passthrough = <0x1>;
};
&gpio {
xen,passthrough = <0x1>;
};
[-- Attachment #3: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-25 10:12 ` Milan Boberic
@ 2018-09-25 10:56 ` Julien Grall
0 siblings, 0 replies; 26+ messages in thread
From: Julien Grall @ 2018-09-25 10:56 UTC (permalink / raw)
To: Milan Boberic, Dario Faggioli; +Cc: xen-devel, stefano
On 09/25/2018 11:12 AM, Milan Boberic wrote:
> Hello guys,
Hi Milan,
> mapping on my system is:
> dom0 have one vCPU and it is pinned on pCPU0
> domU also have one vCPU and it's pinned for pCPU2
Your platform has 4 CPUs, right? What does the other do? Just sitting in
the idle loop?
>
> I removed only vwfi=native and everything works fine. I can destroy
> and create a guest as many times as I want with out any error (still
> using sched=null).
Thank you for testing, this is quite helpful to know that removing
vwfi=native makes a difference.
>
> These are xen bootargs in Xen-overlay.dtsi file:
> xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=768M
> bootscrub=0 maxcpus=1 dom0_max_vcpus=1 dom0_vcpus_pin=true
> timer_slop=0 core_parking=performance cpufreq=xen:performance
> sched=null";
Where does this command line comes from?
There are options that does not make sense for Arm:
- maxvcpus=1
- core_parking=performance
- cpufreq=xen:performance
All of those options are not supported on Arm. The first one is quite
concerning because you request to limit the number of pCPUs used by Xen.
This is not available today. If we ever add support, you would end to
have only 1 pCPUs.
>
> There is whole xen-overlay.dtsi file included in attachment.
>
> Purpose of my work is implementing xen on UltraZed-EG board with
> maximum performance (and lowest jitter, which is why I'm using null
> scheduler and "hard" vcpu pinning) which is my master's thesis on
> faculty. By removing vwfi=native I'm not getting the best performance,
> right? vwfi=native should decrease interrupt latency by ~60% as I read
> here:
>
> https://blog.xenproject.org/author/stefano-stabellini/
IIRC the test was not done with NULL scheduler. So the interrupt latency
may slightly be better. However, there will still be a performance
impact as the scheduler may decide to switch to idle vCPU.
It is possible to reduce the overhead of switch to idle vCPU by
optimizing the context switch.
Anyway, vwfi=native should not affect destroying guest. This should
probably be fixed. I will answer on Dario's e-mail.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-25 9:02 ` Dario Faggioli
2018-09-25 10:12 ` Milan Boberic
@ 2018-09-25 11:15 ` Julien Grall
2018-09-25 11:51 ` Milan Boberic
2018-09-25 17:49 ` Dario Faggioli
1 sibling, 2 replies; 26+ messages in thread
From: Julien Grall @ 2018-09-25 11:15 UTC (permalink / raw)
To: Dario Faggioli, Milan Boberic; +Cc: xen-devel, stefano
Hi Dario,
On 09/25/2018 10:02 AM, Dario Faggioli wrote:
> On Mon, 2018-09-24 at 22:46 +0100, Julien Grall wrote:
>> On 09/21/2018 05:20 PM, Dario Faggioli wrote:
>>>
>>> What I'm after, is how log, after domain_destroy(),
>>> complete_domain_destroy() is called, and whether/how it relates the
>>> the
>>> grace period idle timer we've added in the RCU code.
>>
>> NULL scheduler and vwfi=native will inevitably introduce a latency
>> when
>> destroying a domain. vwfi=native means the guest will not trap when
>> it
>> has nothing to do and switch to the idle vCPU. So, in such
>> configuration, it is extremely unlikely the execute the idle_loop or
>> even enter in the hypervisor unless there are an interrupt on that
>> pCPU.
>>
> Ah! I'm not familiar with wfi=native --and in fact I was completely
> ignoring it-- but this analysis makes sense to me.
>
>> Per my understanding of call_rcu, the calls will be queued until the
>> RCU
>> reached a threshold. We don't have many place where call_rcu is
>> called,
>> so reaching the threeshold may just never happen. But nothing will
>> tell
>> that vCPU to go in Xen and say "I am done with RCU". Did I miss
>> anything?
>>
> Yeah, and in fact we added the timer _but_, in this case, it does not
> look that the timer is firing. It looks much more like "some random
> interrupt happens", as you're suggesting. OTOH, in the case where there
> are no printk()s, it might be that the timer does fire, but the vcpu
> has not gone through Xen, so the grace period is, as far as we know,
> not expired yet (which is also in accordance with Julien's analysis, as
> far as I understood it).
The timer is only activated when sched_tick_suspend() is called. With
vwfi=native, you will never reach the idle_loop() and therefore never
setup a timer.
Milan confirmed that guest can be destroyed with vwfi=native removed. So
this is confirming my thinking. Trapping wfi will end up to switch to
idle vCPU and trigger the grace period.
I am not entirely sure you will be able to reproduce it on x86, but I
don't think it is a Xen Arm specific.
When I looked at the code, I don't see any grace period in other context
than idle_loop. Rather than adding another grace period, I would just
force quiescence for every call_rcu.
This should not be have a big performance impact as we don't use much
call_rcu and it would allow domain to be fully destroyed in timely manner.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-25 11:15 ` Julien Grall
@ 2018-09-25 11:51 ` Milan Boberic
2018-09-25 17:49 ` Dario Faggioli
1 sibling, 0 replies; 26+ messages in thread
From: Milan Boberic @ 2018-09-25 11:51 UTC (permalink / raw)
To: julien.grall; +Cc: xen-devel, stefano, Dario Faggioli
Reply for Julien,
yes, my platform have 4 CPUs it's UltraZed-EG board with carrier card.
I use only 2 CPUs, one for dom0 which is PetaLinux and one for domU
which is bare-metal application that blinks LED on the board (I use it
to measure jitter with oscilloscope), other two CPUs are unused (in
idle loop).
About command, commad is from xen-overlay.dtsi file which is included
in system-user.dtsi file in my project. Whole file is included in
atatchment in my earlier reply.
About this options:
maxvcpus=1
core_parking=performance
cpufreq=xen:performance
I was just testing them to see will I get any performance improvement,
I will remove them right away.
Best regards, Milan Boberic!
On Tue, Sep 25, 2018 at 1:15 PM Julien Grall <julien.grall@arm.com> wrote:
>
> Hi Dario,
>
> On 09/25/2018 10:02 AM, Dario Faggioli wrote:
> > On Mon, 2018-09-24 at 22:46 +0100, Julien Grall wrote:
> >> On 09/21/2018 05:20 PM, Dario Faggioli wrote:
> >>>
> >>> What I'm after, is how log, after domain_destroy(),
> >>> complete_domain_destroy() is called, and whether/how it relates the
> >>> the
> >>> grace period idle timer we've added in the RCU code.
> >>
> >> NULL scheduler and vwfi=native will inevitably introduce a latency
> >> when
> >> destroying a domain. vwfi=native means the guest will not trap when
> >> it
> >> has nothing to do and switch to the idle vCPU. So, in such
> >> configuration, it is extremely unlikely the execute the idle_loop or
> >> even enter in the hypervisor unless there are an interrupt on that
> >> pCPU.
> >>
> > Ah! I'm not familiar with wfi=native --and in fact I was completely
> > ignoring it-- but this analysis makes sense to me.
> >
> >> Per my understanding of call_rcu, the calls will be queued until the
> >> RCU
> >> reached a threshold. We don't have many place where call_rcu is
> >> called,
> >> so reaching the threeshold may just never happen. But nothing will
> >> tell
> >> that vCPU to go in Xen and say "I am done with RCU". Did I miss
> >> anything?
> >>
> > Yeah, and in fact we added the timer _but_, in this case, it does not
> > look that the timer is firing. It looks much more like "some random
> > interrupt happens", as you're suggesting. OTOH, in the case where there
> > are no printk()s, it might be that the timer does fire, but the vcpu
> > has not gone through Xen, so the grace period is, as far as we know,
> > not expired yet (which is also in accordance with Julien's analysis, as
> > far as I understood it).
>
> The timer is only activated when sched_tick_suspend() is called. With
> vwfi=native, you will never reach the idle_loop() and therefore never
> setup a timer.
>
> Milan confirmed that guest can be destroyed with vwfi=native removed. So
> this is confirming my thinking. Trapping wfi will end up to switch to
> idle vCPU and trigger the grace period.
>
> I am not entirely sure you will be able to reproduce it on x86, but I
> don't think it is a Xen Arm specific.
>
> When I looked at the code, I don't see any grace period in other context
> than idle_loop. Rather than adding another grace period, I would just
> force quiescence for every call_rcu.
>
> This should not be have a big performance impact as we don't use much
> call_rcu and it would allow domain to be fully destroyed in timely manner.
>
> Cheers,
>
> --
> Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-25 11:15 ` Julien Grall
2018-09-25 11:51 ` Milan Boberic
@ 2018-09-25 17:49 ` Dario Faggioli
2018-09-27 9:50 ` Dario Faggioli
1 sibling, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-25 17:49 UTC (permalink / raw)
To: Julien Grall, Milan Boberic
Cc: xen-devel, Andrew Cooper, stefano, Jan Beulich, Tim Deegan
[-- Attachment #1.1: Type: text/plain, Size: 3546 bytes --]
[Adding a few people to the Cc-list. See below...]
On Tue, 2018-09-25 at 12:15 +0100, Julien Grall wrote:
> Hi Dario,
>
Hi,
> On 09/25/2018 10:02 AM, Dario Faggioli wrote:
> > On Mon, 2018-09-24 at 22:46 +0100, Julien Grall wrote:
> > >
> > > Per my understanding of call_rcu, the calls will be queued until
> > > the
> > > RCU
> > > reached a threshold. We don't have many place where call_rcu is
> > > called,
> > > so reaching the threeshold may just never happen. But nothing
> > > will
> > > tell
> > > that vCPU to go in Xen and say "I am done with RCU". Did I miss
> > > anything?
> > >
> > Yeah, and in fact we added the timer _but_, in this case, it does
> > not
> > look that the timer is firing. It looks much more like "some random
> > interrupt happens", as you're suggesting. OTOH, in the case where
> > there
> > are no printk()s, it might be that the timer does fire, but the
> > vcpu
> > has not gone through Xen, so the grace period is, as far as we
> > know,
> > not expired yet (which is also in accordance with Julien's
> > analysis, as
> > far as I understood it).
>
> The timer is only activated when sched_tick_suspend() is called.
> With
> vwfi=native, you will never reach the idle_loop() and therefore
> never
> setup a timer.
>
Right.
> Milan confirmed that guest can be destroyed with vwfi=native removed.
> So
> this is confirming my thinking. Trapping wfi will end up to switch
> to
> idle vCPU and trigger the grace period.
>
Indeed.
> I am not entirely sure you will be able to reproduce it on x86, but
> I
> don't think it is a Xen Arm specific.
>
Perhaps with something like idle=poll. But I'm not sure, as it's not
the same thing (that is for guests, while wfi=native is in Xen).
Well, IAC, I don't think we can call it an ARM issue either. It's an
issue of whatever combination of platform and configuration we support,
which we allow not to trap when the guest goes idle.
> When I looked at the code, I don't see any grace period in other
> context
> than idle_loop. Rather than adding another grace period, I would
> just
> force quiescence for every call_rcu.
>
> This should not be have a big performance impact as we don't use
> much
> call_rcu and it would allow domain to be fully destroyed in timely
> manner.
>
I see your point, but this looks a little on the heavy side to me.
I'm not speaking of direct and measurable perf impact, but I think this
would make us diverge quite a bit from the idea of RCUs...
My knowledge of RCU themselves would need refreshing, though. I managed
to getbecome reasonably familiar with how the implementation we
imported works back then, when working on the said issue, but I guess I
better go check the code again.
I'm Cc-ing the people that have reviewed the patches and helping with
the idle timer problem, in case anyone has bright ideas out of the top
of his head.
Perhaps we should "just" get away from using RCU for domain destruction
(but I'm just tossing the idea around, without much consideration about
whether it's the right solution, or about how hard/feasible it really
is).
Or maybe we can still use the timer, in some special way, if we have
wfi=native (or equivalent)...
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-25 17:49 ` Dario Faggioli
@ 2018-09-27 9:50 ` Dario Faggioli
2018-09-27 13:15 ` Milan Boberic
0 siblings, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-27 9:50 UTC (permalink / raw)
To: Julien Grall, Milan Boberic
Cc: xen-devel, Tim Deegan, stefano, Jan Beulich, Andrew Cooper
[-- Attachment #1.1.1: Type: text/plain, Size: 2615 bytes --]
On Tue, 2018-09-25 at 19:49 +0200, Dario Faggioli wrote:
> [Adding a few people to the Cc-list. See below...]
> On Tue, 2018-09-25 at 12:15 +0100, Julien Grall wrote:
> > On 09/25/2018 10:02 AM, Dario Faggioli wrote:
> > > On Mon, 2018-09-24 at 22:46 +0100, Julien Grall wrote:
> > > >
> My knowledge of RCU themselves would need refreshing, though. I
> managed
> to getbecome reasonably familiar with how the implementation we
> imported works back then, when working on the said issue, but I guess
> I
> better go check the code again.
>
> I'm Cc-ing the people that have reviewed the patches and helping with
> the idle timer problem, in case anyone has bright ideas out of the
> top
> of his head.
>
> Perhaps we should "just" get away from using RCU for domain
> destruction
> (but I'm just tossing the idea around, without much consideration
> about
> whether it's the right solution, or about how hard/feasible it really
> is).
>
> Or maybe we can still use the timer, in some special way, if we have
> wfi=native (or equivalent)...
>
So, I've had a look (but only a quick one).
If we want to do something specific within the domain destruction path,
we can add an rcu_barrier() there (I mean in domain_destroy()).
However, that does not feel right either. Also, how can we be sure that
the CPU never going through idle (as far as Xen knows, at least), isn't
going to be problem for other RCU calls as well?
Another thing that we can do is to act on the parameters that control
the threshold which decides when a quiescent state is forced. This was
basically what Julien was suggesting, but I still would avoid to do
that always.
So, basically, in this hackish patch attached, I added a new boot
command line argument, called rcu_force_quiesc. If set to true,
thresholds are set so that quiescence is always forced at each
invocation of call_rcu(). And even if the new param is not explicitly
specified, I do tweak the threshold when "wfi=native" is.
Milan, can you apply this patch, add "wfi=native" again, and re-test?
If it works, we'll decide what to do next.
E.g., we can expose the RCU threshold via the appropriate set of boot
time parameters --like Linux, from where this code comes, did/does--
and document how they should be set, if one wants to use "wfi=native".
Thanks and Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.1.2: rcu-quiesc-patch.patch --]
[-- Type: text/x-patch, Size: 1755 bytes --]
commit 0d2beb3d4125d65c415860d66974db9a5532ac84
Author: Dario Faggioli <dfaggioli@suse.com>
Date: Wed Sep 26 11:47:06 2018 +0200
xen: RCU: bootparam to force quiescence at every call.
Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 0f4b1f2a5d..536eb17017 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -110,7 +110,10 @@ static enum {
static int __init parse_vwfi(const char *s)
{
if ( !strcmp(s, "native") )
+ {
+ rcu_always_quiesc = true;
vwfi = NATIVE;
+ }
else
vwfi = TRAP;
diff --git a/xen/common/rcupdate.c b/xen/common/rcupdate.c
index 3517790913..219dd2884f 100644
--- a/xen/common/rcupdate.c
+++ b/xen/common/rcupdate.c
@@ -140,6 +140,9 @@ static int qhimark = 10000;
static int qlowmark = 100;
static int rsinterval = 1000;
+bool rcu_always_quiesc = false;
+boolean_param("rcu_force_quiesc", rcu_always_quiesc);
+
struct rcu_barrier_data {
struct rcu_head head;
atomic_t *cpu_count;
@@ -562,6 +565,13 @@ static void rcu_init_percpu_data(int cpu, struct rcu_ctrlblk *rcp,
rdp->quiescbatch = rcp->completed;
rdp->qs_pending = 0;
rdp->cpu = cpu;
+ if ( rcu_always_quiesc )
+ {
+ blimit = INT_MAX;
+ qhimark = 0;
+ qlowmark = 0;
+ //rsinterval = 0;
+ }
rdp->blimit = blimit;
init_timer(&rdp->idle_timer, rcu_idle_timer_handler, rdp, cpu);
}
diff --git a/xen/include/xen/rcupdate.h b/xen/include/xen/rcupdate.h
index 3402eb5caf..274a01acf6 100644
--- a/xen/include/xen/rcupdate.h
+++ b/xen/include/xen/rcupdate.h
@@ -56,6 +56,8 @@ struct rcu_head {
} while (0)
+extern bool rcu_always_quiesc;
+
int rcu_pending(int cpu);
int rcu_needs_cpu(int cpu);
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 9:50 ` Dario Faggioli
@ 2018-09-27 13:15 ` Milan Boberic
2018-09-27 14:23 ` Dario Faggioli
2018-09-27 14:32 ` Dario Faggioli
0 siblings, 2 replies; 26+ messages in thread
From: Milan Boberic @ 2018-09-27 13:15 UTC (permalink / raw)
To: Dario Faggioli
Cc: Andrew.Cooper3, tim, julien.grall, stefano, JBeulich, xen-devel
Hi,
I applied patch and vwfi=native and everything works fine, I can
create and destroy guest domain as many times as I want.
I have to ask, will this patch have any impact on performance (I will
test it later, but I just need your opinions)?
And what this patch exactly do? I need to fully understand it because
I need to document it in my master thesis which will be finished soon
thanks to you people :D
Thank you for taking your time to make this patch, best regards!
On Thu, Sep 27, 2018 at 11:51 AM Dario Faggioli <dfaggioli@suse.com> wrote:
>
> On Tue, 2018-09-25 at 19:49 +0200, Dario Faggioli wrote:
> > [Adding a few people to the Cc-list. See below...]
> > On Tue, 2018-09-25 at 12:15 +0100, Julien Grall wrote:
> > > On 09/25/2018 10:02 AM, Dario Faggioli wrote:
> > > > On Mon, 2018-09-24 at 22:46 +0100, Julien Grall wrote:
> > > > >
> > My knowledge of RCU themselves would need refreshing, though. I
> > managed
> > to getbecome reasonably familiar with how the implementation we
> > imported works back then, when working on the said issue, but I guess
> > I
> > better go check the code again.
> >
> > I'm Cc-ing the people that have reviewed the patches and helping with
> > the idle timer problem, in case anyone has bright ideas out of the
> > top
> > of his head.
> >
> > Perhaps we should "just" get away from using RCU for domain
> > destruction
> > (but I'm just tossing the idea around, without much consideration
> > about
> > whether it's the right solution, or about how hard/feasible it really
> > is).
> >
> > Or maybe we can still use the timer, in some special way, if we have
> > wfi=native (or equivalent)...
> >
> So, I've had a look (but only a quick one).
>
> If we want to do something specific within the domain destruction path,
> we can add an rcu_barrier() there (I mean in domain_destroy()).
> However, that does not feel right either. Also, how can we be sure that
> the CPU never going through idle (as far as Xen knows, at least), isn't
> going to be problem for other RCU calls as well?
>
> Another thing that we can do is to act on the parameters that control
> the threshold which decides when a quiescent state is forced. This was
> basically what Julien was suggesting, but I still would avoid to do
> that always.
>
> So, basically, in this hackish patch attached, I added a new boot
> command line argument, called rcu_force_quiesc. If set to true,
> thresholds are set so that quiescence is always forced at each
> invocation of call_rcu(). And even if the new param is not explicitly
> specified, I do tweak the threshold when "wfi=native" is.
>
> Milan, can you apply this patch, add "wfi=native" again, and re-test?
> If it works, we'll decide what to do next.
>
> E.g., we can expose the RCU threshold via the appropriate set of boot
> time parameters --like Linux, from where this code comes, did/does--
> and document how they should be set, if one wants to use "wfi=native".
>
> Thanks and Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Software Engineer @ SUSE https://www.suse.com/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 13:15 ` Milan Boberic
@ 2018-09-27 14:23 ` Dario Faggioli
2018-09-27 14:32 ` Dario Faggioli
1 sibling, 0 replies; 26+ messages in thread
From: Dario Faggioli @ 2018-09-27 14:23 UTC (permalink / raw)
To: Milan Boberic
Cc: Andrew.Cooper3, tim, julien.grall, stefano, JBeulich, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 4707 bytes --]
On Thu, 2018-09-27 at 15:15 +0200, Milan Boberic wrote:
> Hi,
> I applied patch and vwfi=native and everything works fine, I can
> create and destroy guest domain as many times as I want.
>
> I have to ask, will this patch have any impact on performance (I will
> test it later, but I just need your opinions)?
>
Well, with a question like this, the only possible answer is "depends".
:-)
Basically, there is a little bit of overhead to be expected, with this
patch applied, every time that call_rcu() is invoked, inside Xen. Now,
you can look at when that happens, and you'll notice that this
basically never happen in an hot-path.
In your case, there is at least one call in the domain destruction
path. You can try to measure whether actually destroying the domain
takes more time _with_ "wfi=native" (plus this patch) as compared to
how long it takes _without_ "wfi=native" (and also without this patch).
I don't think you'll be able to appreciate any significant difference.
The point is more, I think, whether "wfi=native" helps your use case.
Have you measure that? I mean, have you checked what is the difference
in performance (or latency, or whatever you're interested in) between
the "wfi=native" case and the default?
If you have, and "wfi=native" helps, then you also need something like
this patch, or domain destruction won't work (in fact, I call the fact
that it takes 'around 7 seconds', not working). If "wfi=native" does
not help your use case, then you're better off not using neither it nor
this patch.
> And what this patch exactly do? I need to fully understand it because
> I need to document it in my master thesis which will be finished soon
> thanks to you people :D
>
Have you heard about RCU? It's a very clever synchronization solution,
widely used in the Linux kernel. Xen has that too, but we use an old
version of the Linux code, and we don't use it that much.
This is, IMO, some good introductory material, but, really, just google
"RCU" or "RCU linux", and you'll hit tons of articles and docs:
https://lwn.net/Articles/262464/
Well, our implementation of RCU requires that, from time to time, the
various physical CPUs of your box become idle, or get an interrupt, or
go executing inside Xen (for hypercalls, vmexits, etc). In fact, a CPU
going through Xen is what allow us to tell that it reached a so-called
'quiescent state', which in turns is necessary for declaring a so-
called 'RCU grace period' over.
Usually, as soon as a guest (or dom0) vCPU become idle, the pCPU on
which it was running does go through Xen, to figure out whether or not
there is another vCPU, from the same or from another guest, to be run.
If not, the pCPU stays idle, but it stays idle _in_Xen_, and that is
good for RCU quiescence and grace period tracking.
Now, with the combination of "sched=null" and "wfi=native", when the
guest (or dom0) vCPU becomes idle, we _stay_in_the_guest_, until
something (typically an interrupt) comes. This means that the vCPU in
question never let Xen's RCU know that he has gone through a quiescent
state, and grace periods risk lasting very long, if not forever.
In fact, the reason why everything was working again with a printk()
was, as Julien noted, that an interrupt was being injected. Check the
old discussion on xen-devel about the RCU bug that I linked to in one
of my first messages in this thread to even more insights.
https://www.mail-archive.com/xen-devel@lists.xen.org/msg105388.html
https://lists.xenproject.org/archives/html/xen-devel/2017-07/msg02770.html
https://lists.xen.org/archives/html/xen-devel/2017-09/msg01855.html
https://lists.xen.org/archives/html/xen-devel/2017-09/msg03515.html
https://lists.xenproject.org/archives/html/xen-devel/2017-09/msg01855.html
Setting the qhimark, qlowmark and blimit to the values you see in the
patch, partially defeats the purpose of RCU, as the update of the data
structure is not deferred to some future point in time, but it is
basically always performed synchronously with the modification, and
that's why I dislike just doing it all the time, and I prefer limiting
to doing it when we're using "wfi=native".
For some more details about the meaning of the qhimark, qlowmark and
blimit values, check these:
https://www.systutorials.com/linux-kernels/132439/patch-rcu-batch-tuning-linux-2-6-16/
https://lwn.net/Articles/166647/
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 13:15 ` Milan Boberic
2018-09-27 14:23 ` Dario Faggioli
@ 2018-09-27 14:32 ` Dario Faggioli
2018-09-27 15:09 ` Julien Grall
1 sibling, 1 reply; 26+ messages in thread
From: Dario Faggioli @ 2018-09-27 14:32 UTC (permalink / raw)
To: Stefano Stabellini, Julien Grall, Juergen Gross
Cc: xen-devel, tim, Milan Boberic, JBeulich, Andrew Cooper
[-- Attachment #1.1: Type: text/plain, Size: 1084 bytes --]
On Thu, 2018-09-27 at 15:15 +0200, Milan Boberic wrote:
> Hi,
> I applied patch and vwfi=native and everything works fine, I can
> create and destroy guest domain as many times as I want.
>
Ok, now that we know it works, what do you guys prefer?
Stefano? Julien? I know it's not strictly an ARM-only issue, but I'm
asking you because ARM is where it shows-up/harm the most.
I personally would be ok with:
- doing a patch adding qhimark, qlowmark and blimit boot command line
parameters;
- doing a patch (similar to this one) forcing the parameters to a
specific state (and printing a warning about that), if wfi=native is
used.
Thoughts?
Oh, and I'm adding Juergen in case we consider this relevant for the
release (and in case we want to add a Jira issue, which I don't think I
can do).
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 14:32 ` Dario Faggioli
@ 2018-09-27 15:09 ` Julien Grall
2018-09-27 17:06 ` Dario Faggioli
0 siblings, 1 reply; 26+ messages in thread
From: Julien Grall @ 2018-09-27 15:09 UTC (permalink / raw)
To: Dario Faggioli, Stefano Stabellini, Juergen Gross
Cc: xen-devel, tim, Milan Boberic, JBeulich, Andrew Cooper
Hi Dario,
On 09/27/2018 03:32 PM, Dario Faggioli wrote:
> On Thu, 2018-09-27 at 15:15 +0200, Milan Boberic wrote:
>> Hi,
>> I applied patch and vwfi=native and everything works fine, I can
>> create and destroy guest domain as many times as I want.
>>
> Ok, now that we know it works, what do you guys prefer?
>
> Stefano? Julien? I know it's not strictly an ARM-only issue, but I'm
> asking you because ARM is where it shows-up/harm the most.
>
> I personally would be ok with:
> - doing a patch adding qhimark, qlowmark and blimit boot command line
> parameters;
> - doing a patch (similar to this one) forcing the parameters to a
> specific state (and printing a warning about that), if wfi=native is
> used.
>
> Thoughts?
I know I first suggested this but I have been thinking about it and
trying to find a different approach. With NULL scheduler, you end up
partitioning your platform. I think may have have Xen to be there just
for handling hypercall, emulation and guest interrupt. So I would like
to avoid adding an interrupt when possible.
In one of your e-mail, you wrote:
"Well, our implementation of RCU requires that, from time to time, the
various physical CPUs of your box become idle, or get an interrupt, or
go executing inside Xen (for hypercalls, vmexits, etc). In fact, a CPU
going through Xen is what allow us to tell that it reached a so-called
'quiescent state', which in turns is necessary for declaring a so-
called 'RCU grace period' over."
I don't quite agree with you on the definition of "quiescent state"
here. To take the domain example, we want to wait until all the CPU has
stopped using the pointer (an hypercall could race put_domain). That
pointer will not be in-use if the CPU is in kernel-mode/user-mode or in
the idle loop. Am I correct?
So I am wondering whether we could:
- Mark any CPU in kernel-mode/user-mode quiet
- Raise a RCU_SOFTIRQ in call_rcu?
With that solution, it may even be possible to avoid the timer in the
idle loop.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 15:09 ` Julien Grall
@ 2018-09-27 17:06 ` Dario Faggioli
2018-09-28 8:18 ` Milan Boberic
2018-10-01 12:03 ` Julien Grall
0 siblings, 2 replies; 26+ messages in thread
From: Dario Faggioli @ 2018-09-27 17:06 UTC (permalink / raw)
To: Julien Grall, Stefano Stabellini, Juergen Gross
Cc: xen-devel, tim, Milan Boberic, JBeulich, Andrew Cooper
[-- Attachment #1.1: Type: text/plain, Size: 3838 bytes --]
On Thu, 2018-09-27 at 16:09 +0100, Julien Grall wrote:
> Hi Dario,
>
Hi,
> On 09/27/2018 03:32 PM, Dario Faggioli wrote:
> > On Thu, 2018-09-27 at 15:15 +0200, Milan Boberic wrote:
> > >
> In one of your e-mail, you wrote:
>
> "Well, our implementation of RCU requires that, from time to time,
> the
> various physical CPUs of your box become idle, or get an interrupt,
> or
> go executing inside Xen (for hypercalls, vmexits, etc). In fact, a
> CPU
> going through Xen is what allow us to tell that it reached a so-
> called
> 'quiescent state', which in turns is necessary for declaring a so-
> called 'RCU grace period' over."
>
> I don't quite agree with you on the definition of "quiescent state"
> here.
>
Hehe... I was trying to be both quick and accurate. It's more than
possible that I failed. :-)
> To take the domain example, we want to wait until all the CPU has
> stopped using the pointer (an hypercall could race put_domain).
>
I'm not sure what you mean with "an hypercall could race put_domain".
What we want is to wait until all the CPUs that are involved in the
grace period, have gone through rcupdate.c:cpu_quiet(), or have become
idle.
Receiving an interrupt, or experiencing a context switch, or even going
idle, it's "just" how it happens that these CPUs have their chance to
go through cpu_quiet(). It is in this sense that I meant that those
events are used as markers of a quiescent state.
And "wfi=native" (in particular in combination with the null scheduler,
but I guess also with other ones, at least to a certain extent) makes
figuring out the "or have become idle" part tricky. That is the problem
here, isn't it?
> That
> pointer will not be in-use if the CPU is in kernel-mode/user-mode or
> in
> the idle loop. Am I correct?
>
Right.
So, we want that all the CPUs that were in Xen to have either left Xen
at least once or, if they're still there and have never left, that must
be because they've become idle.
And currently we treat all the CPUs that have not told the RCU
subsystem that they're idle (via rcu_idle_enter()) as busy, without
distinguishing between the ones that are busy in Xen from the one which
are busy in guest (kernel or user) mode.
> So I am wondering whether we could:
> - Mark any CPU in kernel-mode/user-mode quiet
>
Right. We'd need something like a rcu_guest_enter()/rcu_guest_exit()
(or a rcu_xen_exit()/rcu_xen_enter()), which works for all combination
of arches and guest types.
It looks to me too that this would help in this case, as the vCPU that
stays in guest mode because of wfi=idle would be counted as quiet, and
we won't have to wait for it.
> - Raise a RCU_SOFTIRQ in call_rcu?
>
Mmm... what would be the point of this?
> With that solution, it may even be possible to avoid the timer in
> the
> idle loop.
>
Not sure. The timer is there to deal with the case when a CPU which has
a callback queued wants to go idle. It may have quiesced already, but
if there are others which have not, either:
1) we let it go idle, but then the callback will run only when it
wakes up from idle which, without the timer, could be far ahead in
time;
2) we don't let it go idle, but we waste resources;
3) we let it go idle and keep the timer. :-)
But anyway, even if it would not let us get rid of the timer, it seems
like it could be nicer than any other approaches. I accept
help/suggestions about the "let's intercept guest-Xen and Xen-guest
transitions, and track that inside RCU code.
Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Software Engineer @ SUSE https://www.suse.com/
[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 17:06 ` Dario Faggioli
@ 2018-09-28 8:18 ` Milan Boberic
2018-10-01 12:03 ` Julien Grall
1 sibling, 0 replies; 26+ messages in thread
From: Milan Boberic @ 2018-09-28 8:18 UTC (permalink / raw)
To: Dario Faggioli
Cc: jgross, Andrew.Cooper3, tim, julien.grall, stefano, Jan Beulich,
xen-devel
Hi,
thank you for explanation, links and advices. I'm gonna go through all
that literature.
Best regards!
On Thu, Sep 27, 2018 at 7:06 PM Dario Faggioli <dfaggioli@suse.com> wrote:
>
> On Thu, 2018-09-27 at 16:09 +0100, Julien Grall wrote:
> > Hi Dario,
> >
> Hi,
>
> > On 09/27/2018 03:32 PM, Dario Faggioli wrote:
> > > On Thu, 2018-09-27 at 15:15 +0200, Milan Boberic wrote:
> > > >
> > In one of your e-mail, you wrote:
> >
> > "Well, our implementation of RCU requires that, from time to time,
> > the
> > various physical CPUs of your box become idle, or get an interrupt,
> > or
> > go executing inside Xen (for hypercalls, vmexits, etc). In fact, a
> > CPU
> > going through Xen is what allow us to tell that it reached a so-
> > called
> > 'quiescent state', which in turns is necessary for declaring a so-
> > called 'RCU grace period' over."
> >
> > I don't quite agree with you on the definition of "quiescent state"
> > here.
> >
> Hehe... I was trying to be both quick and accurate. It's more than
> possible that I failed. :-)
>
> > To take the domain example, we want to wait until all the CPU has
> > stopped using the pointer (an hypercall could race put_domain).
> >
> I'm not sure what you mean with "an hypercall could race put_domain".
> What we want is to wait until all the CPUs that are involved in the
> grace period, have gone through rcupdate.c:cpu_quiet(), or have become
> idle.
>
> Receiving an interrupt, or experiencing a context switch, or even going
> idle, it's "just" how it happens that these CPUs have their chance to
> go through cpu_quiet(). It is in this sense that I meant that those
> events are used as markers of a quiescent state.
>
> And "wfi=native" (in particular in combination with the null scheduler,
> but I guess also with other ones, at least to a certain extent) makes
> figuring out the "or have become idle" part tricky. That is the problem
> here, isn't it?
>
> > That
> > pointer will not be in-use if the CPU is in kernel-mode/user-mode or
> > in
> > the idle loop. Am I correct?
> >
> Right.
>
> So, we want that all the CPUs that were in Xen to have either left Xen
> at least once or, if they're still there and have never left, that must
> be because they've become idle.
>
> And currently we treat all the CPUs that have not told the RCU
> subsystem that they're idle (via rcu_idle_enter()) as busy, without
> distinguishing between the ones that are busy in Xen from the one which
> are busy in guest (kernel or user) mode.
>
> > So I am wondering whether we could:
> > - Mark any CPU in kernel-mode/user-mode quiet
> >
> Right. We'd need something like a rcu_guest_enter()/rcu_guest_exit()
> (or a rcu_xen_exit()/rcu_xen_enter()), which works for all combination
> of arches and guest types.
>
> It looks to me too that this would help in this case, as the vCPU that
> stays in guest mode because of wfi=idle would be counted as quiet, and
> we won't have to wait for it.
>
> > - Raise a RCU_SOFTIRQ in call_rcu?
> >
> Mmm... what would be the point of this?
>
> > With that solution, it may even be possible to avoid the timer in
> > the
> > idle loop.
> >
> Not sure. The timer is there to deal with the case when a CPU which has
> a callback queued wants to go idle. It may have quiesced already, but
> if there are others which have not, either:
> 1) we let it go idle, but then the callback will run only when it
> wakes up from idle which, without the timer, could be far ahead in
> time;
> 2) we don't let it go idle, but we waste resources;
> 3) we let it go idle and keep the timer. :-)
>
> But anyway, even if it would not let us get rid of the timer, it seems
> like it could be nicer than any other approaches. I accept
> help/suggestions about the "let's intercept guest-Xen and Xen-guest
> transitions, and track that inside RCU code.
>
> Regards,
> Dario
> --
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Software Engineer @ SUSE https://www.suse.com/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: null scheduler bug
2018-09-27 17:06 ` Dario Faggioli
2018-09-28 8:18 ` Milan Boberic
@ 2018-10-01 12:03 ` Julien Grall
1 sibling, 0 replies; 26+ messages in thread
From: Julien Grall @ 2018-10-01 12:03 UTC (permalink / raw)
To: Dario Faggioli, Stefano Stabellini, Juergen Gross
Cc: xen-devel, tim, Milan Boberic, JBeulich, Andrew Cooper
On 09/27/2018 06:06 PM, Dario Faggioli wrote:
> On Thu, 2018-09-27 at 16:09 +0100, Julien Grall wrote:
> Hi,
Hi Dario,
>> On 09/27/2018 03:32 PM, Dario Faggioli wrote:
>>> On Thu, 2018-09-27 at 15:15 +0200, Milan Boberic wrote:
>>>>
>> In one of your e-mail, you wrote:
>>
>> "Well, our implementation of RCU requires that, from time to time,
>> the
>> various physical CPUs of your box become idle, or get an interrupt,
>> or
>> go executing inside Xen (for hypercalls, vmexits, etc). In fact, a
>> CPU
>> going through Xen is what allow us to tell that it reached a so-
>> called
>> 'quiescent state', which in turns is necessary for declaring a so-
>> called 'RCU grace period' over."
>>
>> I don't quite agree with you on the definition of "quiescent state"
>> here.
>>
> Hehe... I was trying to be both quick and accurate. It's more than
> possible that I failed. :-)
>
>> To take the domain example, we want to wait until all the CPU has
>> stopped using the pointer (an hypercall could race put_domain).
>>
> I'm not sure what you mean with "an hypercall could race put_domain".
I meant that another CPU get a pointer on the domain until the domain is
effectively removed from the list by _domain_destroy.
> What we want is to wait until all the CPUs that are involved in the
> grace period, have gone through rcupdate.c:cpu_quiet(), or have become
> idle.
Which is what I meant but in a more convoluted way.
>
> Receiving an interrupt, or experiencing a context switch, or even going
> idle, it's "just" how it happens that these CPUs have their chance to
> go through cpu_quiet(). It is in this sense that I meant that those
> events are used as markers of a quiescent state.
>
> And "wfi=native" (in particular in combination with the null scheduler,
> but I guess also with other ones, at least to a certain extent) makes
> figuring out the "or have become idle" part tricky. That is the problem
> here, isn't it?
That's correct.
>
>> That
>> pointer will not be in-use if the CPU is in kernel-mode/user-mode or
>> in
>> the idle loop. Am I correct?
>>
> Right.
>
> So, we want that all the CPUs that were in Xen to have either left Xen
> at least once or, if they're still there and have never left, that must
> be because they've become idle.
>
> And currently we treat all the CPUs that have not told the RCU
> subsystem that they're idle (via rcu_idle_enter()) as busy, without
> distinguishing between the ones that are busy in Xen from the one which
> are busy in guest (kernel or user) mode.
>
>> So I am wondering whether we could:
>> - Mark any CPU in kernel-mode/user-mode quiet
>>
> Right. We'd need something like a rcu_guest_enter()/rcu_guest_exit()
> (or a rcu_xen_exit()/rcu_xen_enter()), which works for all combination
> of arches and guest types.
>
> It looks to me too that this would help in this case, as the vCPU that
> stays in guest mode because of wfi=idle would be counted as quiet, and
> we won't have to wait for it.
>
>> - Raise a RCU_SOFTIRQ in call_rcu?
>>
> Mmm... what would be the point of this?
To check if the RCU has work to do. You may call_rcu with all the other
CPUs quiet. Or do you expect this to be done in rcu_guest_{enter,exit}()?
>
>> With that solution, it may even be possible to avoid the timer in
>> the
>> idle loop.
>>
> Not sure. The timer is there to deal with the case when a CPU which has
> a callback queued wants to go idle. It may have quiesced already, but
> if there are others which have not, either:
> 1) we let it go idle, but then the callback will run only when it
> wakes up from idle which, without the timer, could be far ahead in
> time;
> 2) we don't let it go idle, but we waste resources;
> 3) we let it go idle and keep the timer. :-)
Oh right.
>
> But anyway, even if it would not let us get rid of the timer, it seems
> like it could be nicer than any other approaches. I accept
> help/suggestions about the "let's intercept guest-Xen and Xen-guest
> transitions, and track that inside RCU code.
The best place for Arm to have those calls would be in:
- enter_hypervisor_head(): This is called after exiting the guest and
before doing any other work
- leave_hypervisor_head(): This is called just before returning to the
guest.
I am CCing Stefano work e-mail. When I spoke with him he was interested
to put some effort towards fixing the bug.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
* null scheduler bug
@ 2018-09-13 8:15 Milan Boberic
0 siblings, 0 replies; 26+ messages in thread
From: Milan Boberic @ 2018-09-13 8:15 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 1902 bytes --]
Hi,
I implemented Xen Hypervisor 4.9.2 on UltraZed-EG board with carrier card
following these steps:
1.) installed petalinux on Ubuntu 16.04
2.) dowloaded UltraZed-EG IO Carrier Card - PetaLinux 2018.2 Standard BSP
3.) created project: petalinux-create -t project –s <path_to_BSP>
4.) copied xen-overlay.dtsi from ZCU102 project (also made with BSP) to my
project
5.) built xen hypervisor following this guide link
<http://www.wiki.xilinx.com/Building%20Xen%20Hypervisor%20with%20Petalinux%202018.1>
(booting
with SD card)
6.) made baremetal application that blinks PS LED with Vivado
7.) measured signals jitted when application is on board with and with out
Xen
I menaged to decrease jitter adding following in xen-overlay.dtsi file in
xen-bootargs: sched=null vwfi=native.
The problem is when I add sched=null vwfi=native I can create my bare-metal
application with xl create and destroy it with xl destroy (disapears from
xl list) but when I want to start it again (xl create) this error pops:
root@uz3eg-iocc-2018-2:~# xl create bm1.cfg
Parsing config from bm1.cfg
(XEN) IRQ 48 is already used by domain 1
libxl: error: libxl_create.c:1278:domcreate_launch_dm: Domain 2:failed give
domain access to irq 48: Device or resource busy
libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain
2:Non-existant domain
libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain 2:Unable
to destroy guest
libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain 2:Destruction of
domain failed
When I remove sched=null vwfi=native and build project again everything
works fine, I can create and destroy bm app as many times as i want.
Added in attachment bare-metal application's configuration file, xl dmesg,
dmesg and xl -v create message, xen-overlay.dtsi file and system-user.dtsi
file.
Thank you in adance, best regards, Milan Boberic.
[-- Attachment #1.2: Type: text/html, Size: 3177 bytes --]
[-- Attachment #2: bare-metal application's configuration file.txt --]
[-- Type: text/plain, Size: 119 bytes --]
name = "bm1"
kernel = "ultraled.bin"
memory = 8
vcpus = 1
cpus = [2]
irqs = [ 48 ]
iomem = [ "0xff0a0,1" ]
hap=0
[-- Attachment #3: dmesg.txt --]
[-- Type: text/plain, Size: 24485 bytes --]
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.0-xilinx-v2018.2 (oe-user@oe-host) (gcc version 7.2.0 (GCC)) #1 SMP Wed Sep 12 13:32:35 CEST 2018
[ 0.000000] Boot CPU: AArch64 Processor [410fd034]
[ 0.000000] Machine model: xlnx,zynqmp
[ 0.000000] Xen 4.9 support found
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 256 MiB at 0x0000000060000000
[ 0.000000] On node 0 totalpages: 196608
[ 0.000000] DMA zone: 2688 pages used for memmap
[ 0.000000] DMA zone: 0 pages reserved
[ 0.000000] DMA zone: 196608 pages, LIFO batch:31
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv0.2 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: Trusted OS migration not required
[ 0.000000] random: fast init done
[ 0.000000] percpu: Embedded 21 pages/cpu @ffffffc03ffb7000 s46488 r8192 d31336 u86016
[ 0.000000] pcpu-alloc: s46488 r8192 d31336 u86016 alloc=21*4096
[ 0.000000] pcpu-alloc: [0] 0
[ 0.000000] Detected VIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for ARM erratum 845719
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 193920
[ 0.000000] Kernel command line: console=hvc0 earlycon=xen earlyprintk=xen maxcpus=1 clk_ignore_unused
[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
[ 0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.000000] Memory: 423276K/786432K available (9980K kernel code, 644K rwdata, 3132K rodata, 512K init, 2168K bss, 101012K reserved, 262144K cma-reserved)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] modules : 0xffffff8000000000 - 0xffffff8008000000 ( 128 MB)
[ 0.000000] vmalloc : 0xffffff8008000000 - 0xffffffbebfff0000 ( 250 GB)
[ 0.000000] .text : 0xffffff8008080000 - 0xffffff8008a40000 ( 9984 KB)
[ 0.000000] .rodata : 0xffffff8008a40000 - 0xffffff8008d60000 ( 3200 KB)
[ 0.000000] .init : 0xffffff8008d60000 - 0xffffff8008de0000 ( 512 KB)
[ 0.000000] .data : 0xffffff8008de0000 - 0xffffff8008e81200 ( 645 KB)
[ 0.000000] .bss : 0xffffff8008e81200 - 0xffffff800909f2b0 ( 2169 KB)
[ 0.000000] fixed : 0xffffffbefe7fd000 - 0xffffffbefec00000 ( 4108 KB)
[ 0.000000] PCI I/O : 0xffffffbefee00000 - 0xffffffbeffe00000 ( 16 MB)
[ 0.000000] vmemmap : 0xffffffbf00000000 - 0xffffffc000000000 ( 4 GB maximum)
[ 0.000000] 0xffffffbf00700000 - 0xffffffbf01880000 ( 17 MB actual)
[ 0.000000] memory : 0xffffffc020000000 - 0xffffffc070000000 ( 1280 MB)
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU event tracing is enabled.
[ 0.000000] RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] arch_timer: cp15 timer(s) running at 99.99MHz (virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x171015c90f, max_idle_ns: 440795203080 ns
[ 0.000003] sched_clock: 56 bits at 99MHz, resolution 10ns, wraps every 4398046511101ns
[ 0.000288] Console: colour dummy device 80x25
[ 0.283045] console [hvc0] enabled
[ 0.286517] Calibrating delay loop (skipped), value calculated using timer frequency.. 199.99 BogoMIPS (lpj=399996)
[ 0.296973] pid_max: default: 32768 minimum: 301
[ 0.301734] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.308397] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.316322] ASID allocator initialised with 65536 entries
[ 0.321335] xen:grant_table: Grant tables using version 1 layout
[ 0.327096] Grant table initialized
[ 0.330640] xen:events: Using FIFO-based ABI
[ 0.334966] Xen: initializing cpu0
[ 0.338472] Hierarchical SRCU implementation.
[ 0.343135] EFI services will not be available.
[ 0.347426] zynqmp_plat_init Platform Management API v1.0
[ 0.352856] zynqmp_plat_init Trustzone version v1.0
[ 0.357830] smp: Bringing up secondary CPUs ...
[ 0.362370] smp: Brought up 1 node, 1 CPU
[ 0.366434] SMP: Total of 1 processors activated.
[ 0.371194] CPU features: detected feature: 32-bit EL0 Support
[ 0.377078] CPU: All CPU(s) started at EL1
[ 0.381235] alternatives: patching kernel code
[ 0.386139] devtmpfs: initialized
[ 0.392769] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.398879] futex hash table entries: 256 (order: 3, 32768 bytes)
[ 0.410951] xor: measuring software checksum speed
[ 0.449190] 8regs : 2111.000 MB/sec
[ 0.489247] 8regs_prefetch: 1882.000 MB/sec
[ 0.529301] 32regs : 2594.000 MB/sec
[ 0.569360] 32regs_prefetch: 2181.000 MB/sec
[ 0.569398] xor: using function: 32regs (2594.000 MB/sec)
[ 0.573961] pinctrl core: initialized pinctrl subsystem
[ 0.580445] NET: Registered protocol family 16
[ 0.585024] vdso: 2 pages (1 code @ ffffff8008a46000, 1 data @ ffffff8008de4000)
[ 0.591107] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.598979] DMA: preallocated 256 KiB pool for atomic allocations
[ 0.604134] xen:swiotlb_xen: Warning: only able to allocate 4 MB for software IO TLB
[ 0.612986] software IO TLB [mem 0x3d400000-0x3d800000] (4MB) mapped at [ffffffc03d400000-ffffffc03d7fffff]
[ 0.654599] reset_zynqmp reset-controller: Xilinx zynqmp reset driver probed
[ 0.656585] ARM CCI_400_r1 PMU driver probed
[ 0.661854] zynqmp-pinctrl ff180000.pinctrl: zynqmp pinctrl initialized
[ 0.697873] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.763762] raid6: int64x1 gen() 369 MB/s
[ 0.831786] raid6: int64x1 xor() 408 MB/s
[ 0.899934] raid6: int64x2 gen() 631 MB/s
[ 0.967976] raid6: int64x2 xor() 552 MB/s
[ 1.036132] raid6: int64x4 gen() 957 MB/s
[ 1.104181] raid6: int64x4 xor() 679 MB/s
[ 1.172324] raid6: int64x8 gen() 898 MB/s
[ 1.240426] raid6: int64x8 xor() 683 MB/s
[ 1.308592] raid6: neonx1 gen() 666 MB/s
[ 1.376645] raid6: neonx1 xor() 782 MB/s
[ 1.444750] raid6: neonx2 gen() 1071 MB/s
[ 1.512884] raid6: neonx2 xor() 1102 MB/s
[ 1.580975] raid6: neonx4 gen() 1377 MB/s
[ 1.649094] raid6: neonx4 xor() 1317 MB/s
[ 1.717204] raid6: neonx8 gen() 1510 MB/s
[ 1.785318] raid6: neonx8 xor() 1398 MB/s
[ 1.785354] raid6: using algorithm neonx8 gen() 1510 MB/s
[ 1.789486] raid6: .... xor() 1398 MB/s, rmw enabled
[ 1.794502] raid6: using neon recovery algorithm
[ 1.800431] XGpio: /amba_pl@0/gpio@80000000: registered, base is 504
[ 1.806398] XGpio: /amba_pl@0/gpio@80001000: registered, base is 496
[ 1.812352] XGpio: /amba_pl@0/gpio@80002000: registered, base is 493
[ 1.818780] xen:balloon: Initialising balloon driver
[ 1.823815] SCSI subsystem initialized
[ 1.827229] libata version 3.00 loaded.
[ 1.827357] usbcore: registered new interface driver usbfs
[ 1.832766] usbcore: registered new interface driver hub
[ 1.838125] usbcore: registered new device driver usb
[ 1.843264] media: Linux media interface: v0.10
[ 1.847806] Linux video capture interface: v2.00
[ 1.852489] pps_core: LinuxPPS API ver. 1 registered
[ 1.857475] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 1.866655] PTP clock support registered
[ 1.870636] EDAC MC: Ver: 3.0.0
[ 1.876209] zynqmp-ipi ff9905c0.mailbox: Probed ZynqMP IPI Mailbox driver.
[ 1.880915] FPGA manager framework
[ 1.884301] fpga-region fpga-full: FPGA Region probed
[ 1.889393] Advanced Linux Sound Architecture Driver Initialized.
[ 1.895713] Bluetooth: Core ver 2.22
[ 1.899099] NET: Registered protocol family 31
[ 1.903581] Bluetooth: HCI device and connection manager initialized
[ 1.909984] Bluetooth: HCI socket layer initialized
[ 1.914913] Bluetooth: L2CAP socket layer initialized
[ 1.920026] Bluetooth: SCO socket layer initialized
[ 1.926783] clocksource: Switched to clocksource arch_sys_counter
[ 1.931188] VFS: Disk quotas dquot_6.6.0
[ 1.935113] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 1.949643] NET: Registered protocol family 2
[ 1.950022] TCP established hash table entries: 8192 (order: 4, 65536 bytes)
[ 1.955627] TCP bind hash table entries: 8192 (order: 5, 131072 bytes)
[ 1.962264] TCP: Hash tables configured (established 8192 bind 8192)
[ 1.968610] UDP hash table entries: 512 (order: 2, 16384 bytes)
[ 1.974525] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[ 1.980998] NET: Registered protocol family 1
[ 1.986218] RPC: Registered named UNIX socket transport module.
[ 1.991291] RPC: Registered udp transport module.
[ 1.996042] RPC: Registered tcp transport module.
[ 2.000798] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 2.007291] PCI: CLS 0 bytes, default 128
[ 2.007402] Trying to unpack rootfs image as initramfs...
[ 4.342876] Freeing initrd memory: 53916K
[ 4.343827] audit: initializing netlink subsys (disabled)
[ 4.347696] audit: type=2000 audit(4.112:1): state=initialized audit_enabled=0 res=1
[ 4.354839] workingset: timestamp_bits=62 max_order=18 bucket_order=0
[ 4.361860] NFS: Registering the id_resolver key type
[ 4.366205] Key type id_resolver registered
[ 4.370423] Key type id_legacy registered
[ 4.374493] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 4.381249] jffs2: version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc.
[ 4.417470] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[ 4.419325] io scheduler noop registered
[ 4.423302] io scheduler deadline registered
[ 4.427642] io scheduler cfq registered (default)
[ 4.432382] io scheduler mq-deadline registered
[ 4.436966] io scheduler kyber registered
[ 4.441996] OF: /amba/dma@fd500000: could not find phandle
[ 4.446894] xilinx-zynqmp-dma fd500000.dma: ZynqMP DMA driver Probe success
[ 4.453614] OF: /amba/dma@fd510000: could not find phandle
[ 4.459236] xilinx-zynqmp-dma fd510000.dma: ZynqMP DMA driver Probe success
[ 4.466150] OF: /amba/dma@fd520000: could not find phandle
[ 4.471768] xilinx-zynqmp-dma fd520000.dma: ZynqMP DMA driver Probe success
[ 4.478696] OF: /amba/dma@fd530000: could not find phandle
[ 4.484312] xilinx-zynqmp-dma fd530000.dma: ZynqMP DMA driver Probe success
[ 4.491235] OF: /amba/dma@fd540000: could not find phandle
[ 4.496859] xilinx-zynqmp-dma fd540000.dma: ZynqMP DMA driver Probe success
[ 4.503777] OF: /amba/dma@fd550000: could not find phandle
[ 4.509405] xilinx-zynqmp-dma fd550000.dma: ZynqMP DMA driver Probe success
[ 4.516319] OF: /amba/dma@fd560000: could not find phandle
[ 4.521940] xilinx-zynqmp-dma fd560000.dma: ZynqMP DMA driver Probe success
[ 4.528863] OF: /amba/dma@fd570000: could not find phandle
[ 4.534482] xilinx-zynqmp-dma fd570000.dma: ZynqMP DMA driver Probe success
[ 4.541598] xilinx-zynqmp-dma ffa80000.dma: ZynqMP DMA driver Probe success
[ 4.548525] xilinx-zynqmp-dma ffa90000.dma: ZynqMP DMA driver Probe success
[ 4.555537] xilinx-zynqmp-dma ffaa0000.dma: ZynqMP DMA driver Probe success
[ 4.562540] xilinx-zynqmp-dma ffab0000.dma: ZynqMP DMA driver Probe success
[ 4.569548] xilinx-zynqmp-dma ffac0000.dma: ZynqMP DMA driver Probe success
[ 4.576551] xilinx-zynqmp-dma ffad0000.dma: ZynqMP DMA driver Probe success
[ 4.583559] xilinx-zynqmp-dma ffae0000.dma: ZynqMP DMA driver Probe success
[ 4.590567] xilinx-zynqmp-dma ffaf0000.dma: ZynqMP DMA driver Probe success
[ 4.600586] xen:xen_evtchn: Event-channel device installed
[ 4.654825] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 4.659875] cacheinfo: Unable to detect cache hierarchy for CPU 0
[ 4.668302] brd: module loaded
[ 4.672780] loop: module loaded
[ 4.672824] Invalid max_queues (4), will use default max: 1.
[ 4.676838] OF: /amba/ahci@fd0c0000: could not find phandle
[ 4.682023] ahci-ceva fd0c0000.ahci: AHCI 0001.0301 32 slots 2 ports 6 Gbps 0x3 impl platform mode
[ 4.690724] ahci-ceva fd0c0000.ahci: flags: 64bit ncq sntf pm clo only pmp fbs pio slum part ccc sds apst
[ 4.702555] scsi host0: ahci-ceva
[ 4.704090] scsi host1: ahci-ceva
[ 4.707272] ata1: SATA max UDMA/133 mmio [mem 0xfd0c0000-0xfd0c1fff] port 0x100 irq 32
[ 4.715113] ata2: SATA max UDMA/133 mmio [mem 0xfd0c0000-0xfd0c1fff] port 0x180 irq 32
[ 4.723218] mtdoops: mtd device (mtddev=name/number) must be supplied
[ 4.729875] OF: /amba/spi@ff0f0000: could not find phandle
[ 4.736377] m25p80 spi0.0: found n25q256a, expected m25p80
[ 4.740974] m25p80 spi0.0: n25q256a (65536 Kbytes)
[ 4.745496] 3 ofpart partitions found on MTD device spi0.0
[ 4.751008] Creating 3 MTD partitions on "spi0.0":
[ 4.755856] 0x000000000000-0x000001360000 : "boot"
[ 4.761240] 0x000001360000-0x0000013a0000 : "bootenv"
[ 4.766378] 0x0000013a0000-0x000002aa0000 : "kernel"
[ 4.772281] libphy: Fixed MDIO Bus: probed
[ 4.776036] tun: Universal TUN/TAP device driver, 1.6
[ 4.786614] CAN device driver interface
[ 4.787120] OF: /amba/ethernet@ff0e0000: could not find phandle
[ 4.791221] macb ff0e0000.ethernet: Not enabling partial store and forward
[ 4.798312] libphy: MACB_mii_bus: probed
[ 4.806311] macb ff0e0000.ethernet eth0: Cadence GEM rev 0x50070106 at 0xff0e0000 irq 25 (00:0a:35:00:22:01)
[ 4.811677] TI DP83867 ff0e0000.ethernet-ffffffff:09: attached PHY driver [TI DP83867] (mii_bus:phy_addr=ff0e0000.ethernet-ffffffff:09, irq=POLL)
[ 4.825270] xen_netfront: Initialising Xen virtual ethernet driver
[ 4.831015] usbcore: registered new interface driver asix
[ 4.836423] usbcore: registered new interface driver ax88179_178a
[ 4.842557] usbcore: registered new interface driver cdc_ether
[ 4.848440] usbcore: registered new interface driver net1080
[ 4.854148] usbcore: registered new interface driver cdc_subset
[ 4.860116] usbcore: registered new interface driver zaurus
[ 4.865748] usbcore: registered new interface driver cdc_ncm
[ 4.872748] xilinx-axipmon ffa00000.perf-monitor: Probed Xilinx APM
[ 4.878124] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 4.884316] ehci-pci: EHCI PCI platform driver
[ 4.889081] usbcore: registered new interface driver cdc_acm
[ 4.894521] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
[ 4.902590] usbcore: registered new interface driver uas
[ 4.907959] usbcore: registered new interface driver usb-storage
[ 4.915300] rtc_zynqmp ffa60000.rtc: rtc core: registered ffa60000.rtc as rtc0
[ 4.921306] i2c /dev entries driver
[ 4.925549] cdns-i2c ff030000.i2c: 400 kHz mmio ff030000 irq 26
[ 4.959403] i2c i2c-0: Added multiplexed i2c bus 1
[ 4.959654] i2c i2c-0: Added multiplexed i2c bus 2
[ 4.963499] pca954x 0-0070: registered 2 multiplexed busses for I2C mux pca9542
[ 4.970929] IR NEC protocol handler initialized
[ 4.975426] IR RC5(x/sz) protocol handler initialized
[ 4.980528] IR RC6 protocol handler initialized
[ 4.985112] IR JVC protocol handler initialized
[ 4.989697] IR Sony protocol handler initialized
[ 4.994368] IR SANYO protocol handler initialized
[ 4.999125] IR Sharp protocol handler initialized
[ 5.003883] IR MCE Keyboard/mouse protocol handler initialized
[ 5.009765] IR XMP protocol handler initialized
[ 5.015351] usbcore: registered new interface driver uvcvideo
[ 5.020147] USB Video Class driver (1.1.1)
[ 5.026313] cdns-wdt fd4d0000.watchdog: Xilinx Watchdog Timer at ffffff800915d000 with timeout 10s
[ 5.033721] Bluetooth: HCI UART driver ver 2.3
[ 5.037800] Bluetooth: HCI UART protocol H4 registered
[ 5.042988] Bluetooth: HCI UART protocol BCSP registered
[ 5.048366] Bluetooth: HCI UART protocol LL registered
[ 5.053537] Bluetooth: HCI UART protocol ATH3K registered
[ 5.058985] Bluetooth: HCI UART protocol Three-wire (H5) registered
[ 5.065336] Bluetooth: HCI UART protocol Intel registered
[ 5.070758] Bluetooth: HCI UART protocol QCA registered
[ 5.076059] usbcore: registered new interface driver bcm203x
[ 5.081764] usbcore: registered new interface driver bpa10x
[ 5.087385] usbcore: registered new interface driver bfusb
[ 5.092921] usbcore: registered new interface driver btusb
[ 5.098430] Bluetooth: Generic Bluetooth SDIO driver ver 0.1
[ 5.104185] usbcore: registered new interface driver ath3k
[ 5.109804] EDAC MC: ECC not enabled
[ 5.113462] EDAC DEVICE0: Giving out device to module zynqmp-ocm-edac controller zynqmp_ocm: DEV ff960000.memory-controller (INTERRUPT)
[ 5.125769] cpu cpu0: failed to get clock: -2
[ 5.129923] cpufreq-dt: probe of cpufreq-dt failed with error -2
[ 5.136111] sdhci: Secure Digital Host Controller Interface driver
[ 5.142200] sdhci: Copyright(c) Pierre Ossman
[ 5.146610] sdhci-pltfm: SDHCI platform and OF driver helper
[ 5.152424] OF: /amba/sdhci@ff160000: could not find phandle
[ 5.158829] ata2: SATA link down (SStatus 0 SControl 330)
[ 5.163514] ata1: SATA link down (SStatus 0 SControl 330)
[ 5.214769] mmc0: SDHCI controller on ff160000.sdhci [ff160000.sdhci] using ADMA 64-bit
[ 5.217259] PLL: shutdown
[ 5.219989] PLL: enable
[ 5.228137] OF: /amba/sdhci@ff170000: could not find phandle
[ 5.270766] mmc1: SDHCI controller on ff170000.sdhci [ff170000.sdhci] using ADMA 64-bit
[ 5.279165] ledtrig-cpu: registered to indicate activity on CPUs
[ 5.279794] usbcore: registered new interface driver usbhid
[ 5.285238] usbhid: USB HID core driver
[ 5.297942] fpga_manager fpga0: Xilinx ZynqMP FPGA Manager registered
[ 5.300304] pktgen: Packet Generator for packet performance testing. Version: 2.75
[ 5.307120] Netfilter messages via NETLINK v0.30.
[ 5.311328] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 5.316597] Initializing XFRM netlink socket
[ 5.320944] NET: Registered protocol family 10
[ 5.325997] Segment Routing with IPv6
[ 5.329152] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 5.334581] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 5.340840] NET: Registered protocol family 17
[ 5.345027] NET: Registered protocol family 15
[ 5.349527] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 5.362491] Ebtables v2.0 registered
[ 5.371640] can: controller area network core (rev 20170425 abi 9)
[ 5.372387] NET: Registered protocol family 29
[ 5.376860] can: raw protocol (rev 20170425)
[ 5.381175] can: broadcast manager protocol (rev 20170425 t)
[ 5.386886] can: netlink gateway (rev 20170425) max_hops=1
[ 5.392508] Bluetooth: RFCOMM TTY layer initialized
[ 5.397362] Bluetooth: RFCOMM socket layer initialized
[ 5.402547] Bluetooth: RFCOMM ver 1.11
[ 5.406350] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 5.411709] Bluetooth: BNEP filters: protocol multicast
[ 5.416988] Bluetooth: BNEP socket layer initialized
[ 5.422003] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[ 5.427974] Bluetooth: HIDP socket layer initialized
[ 5.433105] 9pnet: Installing 9P2000 support
[ 5.437329] Key type dns_resolver registered
[ 5.441917] mmc0: new HS200 MMC card at address 0001
[ 5.447024] mmcblk0: mmc0:0001 Q2J55L 7.09 GiB
[ 5.451319] mmcblk0boot0: mmc0:0001 Q2J55L partition 1 16.0 MiB
[ 5.457296] mmcblk0boot1: mmc0:0001 Q2J55L partition 2 16.0 MiB
[ 5.463278] mmcblk0rpmb: mmc0:0001 Q2J55L partition 3 4.00 MiB
[ 5.469747] registered taskstats version 1
[ 5.473635] mmcblk0: p1
[ 5.477361] Btrfs loaded, crc32c=crc32c-generic
[ 5.487234] dwc3-of-simple ff9d0000.usb0: dwc3_simple_set_phydata: Can't find usb3-phy
[ 5.490013] OF: /amba/usb0@ff9d0000/dwc3@fe200000: could not find phandle
[ 5.497385] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[ 5.502001] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 1
[ 5.510060] xhci-hcd xhci-hcd.0.auto: hcc params 0x0238f625 hci version 0x100 quirks 0x22010010
[ 5.518477] xhci-hcd xhci-hcd.0.auto: irq 57, io mem 0xfe200000
[ 5.524525] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[ 5.531227] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 5.538479] usb usb1: Product: xHCI Host Controller
[ 5.543408] usb usb1: Manufacturer: Linux 4.14.0-xilinx-v2018.2 xhci-hcd
[ 5.550155] usb usb1: SerialNumber: xhci-hcd.0.auto
[ 5.556069] mmc1: new high speed SDHC card at address 1234
[ 5.560951] hub 1-0:1.0: USB hub found
[ 5.564561] mmcblk1: mmc1:1234 SA08G 7.21 GiB (ro)
[ 5.569448] hub 1-0:1.0: 1 port detected
[ 5.573630] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[ 5.578808] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 2
[ 5.586602] mmcblk1: p1
[ 5.589567] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[ 5.597310] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[ 5.604052] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 5.611314] usb usb2: Product: xHCI Host Controller
[ 5.616243] usb usb2: Manufacturer: Linux 4.14.0-xilinx-v2018.2 xhci-hcd
[ 5.622990] usb usb2: SerialNumber: xhci-hcd.0.auto
[ 5.628190] hub 2-0:1.0: USB hub found
[ 5.631832] hub 2-0:1.0: 1 port detected
[ 5.636827] rtc_zynqmp ffa60000.rtc: setting system clock to 1970-01-01 00:01:05 UTC (65)
[ 5.644033] clk: Not disabling unused clocks
[ 5.648291] ALSA device list:
[ 5.651274] No soundcards found.
[ 5.655498] Freeing unused kernel memory: 512K
[ 5.734925] udevd[1716]: starting version 3.2.2
[ 5.742492] udevd[1717]: starting eudev-3.2.2
[ 6.771074] export_store: invalid GPIO 350
[ 6.771291] blinky[1980]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006, in libc-2.26.so[7f992e2000+138000]
[ 6.783194] CPU: 0 PID: 1980 Comm: blinky Not tainted 4.14.0-xilinx-v2018.2 #1
[ 6.790438] Hardware name: xlnx,zynqmp (DT)
[ 6.794677] task: ffffffc03c9e8080 task.stack: ffffff800b2d0000
[ 6.800646] PC is at 0x7f99343abc
[ 6.804018] LR is at 0x400dfc
[ 6.807046] pc : [<0000007f99343abc>] lr : [<0000000000400dfc>] pstate: 60000000
[ 6.814484] sp : 0000007fc4068f90
[ 6.817857] x29: 0000007fc4068f90 x28: 0000000000000000
[ 6.823220] x27: 0000000000000000 x26: 0000007fc4069edf
[ 6.828584] x25: 0000007fc40690a4 x24: 0000000000401140
[ 6.833947] x23: 0000000000412000 x22: 0000000000000003
[ 6.839309] x21: 0000000000000001 x20: 0000007fc4068ff8
[ 6.844673] x19: 0000000000000000 x18: 0000007fc4068d3d
[ 6.850036] x17: 0000007f99343aa0 x16: 0000000000412058
[ 6.855399] x15: 000000000000000a x14: 000000000000015e
[ 6.860762] x13: 0000000000000000 x12: 0000000000000000
[ 6.866125] x11: 0000000000000020 x10: 0000007fc4068d40
[ 6.871488] x9 : 0000000000000000 x8 : 0000000000000038
[ 6.876851] x7 : 0000000000000000 x6 : 0000000000401126
[ 6.882214] x5 : 0000000000000000 x4 : 000000000d518260
[ 6.887577] x3 : 0000000000000000 x2 : 0000000000000001
[ 6.892940] x1 : 0000000000000001 x0 : 0000007fc4068ff8
[ 7.093343] pps pps0: new PPS source ptp0
[ 7.093400] macb ff0e0000.ethernet: gem-ptp-timer ptp clock registered.
[ 7.098572] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 9.131043] macb ff0e0000.ethernet eth0: link up (1000/Full)
[ 9.131170] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 229.609858] random: crng init done
[-- Attachment #4: system-user.dtsi.txt --]
[-- Type: text/plain, Size: 1828 bytes --]
/include/ "system-conf.dtsi"
/include/ "xen-overlay.dtsi"
/ {
};
&gem3 {
status = "okay";
local-mac-address = [00 0a 35 00 02 90];
phy-mode = "rgmii-id";
phy-handle = <&phy0>;
phy0: phy@9 {
reg = <0x9>;
ti,rx-internal-delay = <0x5>;
ti,tx-internal-delay = <0x5>;
ti,fifo-depth = <0x1>;
};
};
&i2c1 {
status = "okay";
clock-frequency = <400000>;
i2cswitch@70 { /* U7 on UZ3EG SOM */
compatible = "nxp,pca9542";
#address-cells = <1>;
#size-cells = <0>;
reg = <0x70>;
i2c@0 { /* i2c mw 70 0 1 */
#address-cells = <1>;
#size-cells = <0>;
reg = <0>;
/* IIC_EEPROM */
eeprom@51 { /* U5 on UZ3EG IOCC and U7 on the UZ7EV EVCC*/
compatible = "at,24c08";
reg = <0x51>;
};
};
};
};
&qspi {
#address-cells = <1>;
#size-cells = <0>;
status = "okay";
is-dual = <1>; /* Set for dual-parallel QSPI config */
num-cs = <2>;
xlnx,fb-clk = <0x1>;
flash0: flash@0 {
/* The Flash described below doesn't match our board ("micron,n25qu256a"), but is needed */
/* so the Flash MTD partitions are correctly identified in /proc/mtd */
compatible = "micron,m25p80"; /* 32MB */
#address-cells = <1>;
#size-cells = <1>;
reg = <0x0>;
spi-tx-bus-width = <1>;
spi-rx-bus-width = <4>; /* FIXME also DUAL configuration possible */
spi-max-frequency = <108000000>; /* Set to 108000000 Based on DC1 spec */
};
};
/* SD0 eMMC, 8-bit wide data bus */
&sdhci0 {
status = "okay";
bus-width = <8>;
max-frequency = <50000000>;
};
/* SD1 with level shifter */
&sdhci1 {
status = "okay";
max-frequency = <50000000>;
no-1-8-v; /* for 1.0 silicon */
};
/* ULPI SMSC USB3320 */
&usb0 {
status = "okay";
};
&dwc3_0 {
status = "okay";
dr_mode = "host";
phy-names = "usb3-phy";
};
[-- Attachment #5: xen-overlay.dtsi.txt --]
[-- Type: text/plain, Size: 1213 bytes --]
/ {
chosen {
#address-cells = <2>;
#size-cells = <1>;
xen,xen-bootargs = "console=dtuart dtuart=serial0 dom0_mem=768M bootscrub=0 maxcpus=1 dom0_max_vcpus=1 dom0_vcpus_pin=true timer_slop=0 core_parking=performance cpufreq=xen:performance sched=null vwfi=native";
xen,dom0-bootargs = "console=hvc0 earlycon=xen earlyprintk=xen maxcpus=1 clk_ignore_unused";
dom0 {
compatible = "xen,linux-zimage", "xen,multiboot-module";
reg = <0x0 0x80000 0x3100000>;
};
};
};
&smmu {
status = "okay";
mmu-masters = < &gem0 0x874
&gem1 0x875
&gem2 0x876
&gem3 0x877
&dwc3_0 0x860
&dwc3_1 0x861
&qspi 0x873
&lpd_dma_chan1 0x868
&lpd_dma_chan2 0x869
&lpd_dma_chan3 0x86a
&lpd_dma_chan4 0x86b
&lpd_dma_chan5 0x86c
&lpd_dma_chan6 0x86d
&lpd_dma_chan7 0x86e
&lpd_dma_chan8 0x86f
&fpd_dma_chan1 0x14e8
&fpd_dma_chan2 0x14e9
&fpd_dma_chan3 0x14ea
&fpd_dma_chan4 0x14eb
&fpd_dma_chan5 0x14ec
&fpd_dma_chan6 0x14ed
&fpd_dma_chan7 0x14ee
&fpd_dma_chan8 0x14ef
&sdhci0 0x870
&sdhci1 0x871
&nand0 0x872>;
};
&uart1 {
xen,passthrough = <0x1>;
};
&gpio {
xen,passthrough = <0x1>;
};
[-- Attachment #6: xl dmesg after trying to create bm app again.txt --]
[-- Type: text/plain, Size: 4377 bytes --]
(XEN) Checking for initrd in /chosen
(XEN) Initrd 0000000002b58000-0000000005ffffc9
(XEN) RAM: 0000000000000000 - 000000007fefffff
(XEN)
(XEN) MODULE[0]: 0000000007ff4000 - 0000000007ffc080 Device Tree
(XEN) MODULE[1]: 0000000002b58000 - 0000000005ffffc9 Ramdisk
(XEN) MODULE[2]: 0000000000080000 - 0000000003180000 Kernel
(XEN) RESVD[0]: 0000000007ff4000 - 0000000007ffc000
(XEN) RESVD[1]: 0000000002b58000 - 0000000005ffffc9
(XEN)
(XEN) Command line: console=dtuart dtuart=serial0 dom0_mem=768M bootscrub=0 maxcpus=1 dom0_max_vcpus=1 dom0_vcpus_pin=true timer_slop=0 core_parking=performance cpufreq=xen:performance sched=null vwfi=native
(XEN) Placing Xen at 0x000000007fc00000-0x000000007fe00000
(XEN) Update BOOTMOD_XEN from 0000000006000000-0000000006100d81 => 000000007fc00000-000000007fd00d81
(XEN) Domain heap initialised
(XEN) Booting using Device Tree
(XEN) Looking for dtuart at "serial0", options ""
Xen 4.9.2-pre
(XEN) Xen version 4.9.2-pre (@) (aarch64-xilinx-linux-gcc (GCC) 7.2.0) debug=n Mon Jun 11 18:56:51 EDT 2018
(XEN) Latest ChangeSet: Thu Mar 22 22:02:18 2018 +0100 git:c227fe6-dirty
(XEN) Processor: 410fd034: "ARM Limited", variant: 0x0, part 0xd03, rev 0x4
(XEN) 64-bit Execution:
(XEN) Processor Features: 0000000000002222 0000000000000000
(XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN) Extensions: FloatingPoint AdvancedSIMD
(XEN) Debug Features: 0000000010305106 0000000000000000
(XEN) Auxiliary Features: 0000000000000000 0000000000000000
(XEN) Memory Model Features: 0000000000001122 0000000000000000
(XEN) ISA Features: 0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN) Processor Features: 00000131:00011011
(XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN) Extensions: GenericTimer Security
(XEN) Debug Features: 03010066
(XEN) Auxiliary Features: 00000000
(XEN) Memory Model Features: 10201105 40000000 01260000 02102211
(XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 99999 KHz
(XEN) GICv2 initialization:
(XEN) gic_dist_addr=00000000f9010000
(XEN) gic_cpu_addr=00000000f9020000
(XEN) gic_hyp_addr=00000000f9040000
(XEN) gic_vcpu_addr=00000000f9060000
(XEN) gic_maintenance_irq=25
(XEN) GICv2: Adjusting CPU interface base to 0xf902f000
(XEN) GICv2: 192 lines, 4 cpus, secure (IID 0200143b).
(XEN) Using scheduler: null Scheduler (null)
(XEN) Initializing null scheduler
(XEN) WARNING: This is experimental software in development.
(XEN) Use at your own risk.
(XEN) Allocated console ring of 16 KiB.
(XEN) Bringing up CPU1
(XEN) Bringing up CPU2
(XEN) Bringing up CPU3
(XEN) Brought up 4 CPUs
(XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID
(XEN) P2M: 3 levels with order-1 root, VTCR 0x80023558
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 0000000000080000
(XEN) Loading ramdisk from boot module @ 0000000002b58000
(XEN) Allocating 1:1 mappings totalling 768MB for dom0:
(XEN) BANK[0] 0x00000020000000-0x00000040000000 (512MB)
(XEN) BANK[1] 0x00000060000000-0x00000070000000 (256MB)
(XEN) Grant table range: 0x0000007fc00000-0x0000007fc5b000
(XEN) Loading zImage from 0000000000080000 to 0000000020080000-0000000023180000
(XEN) Loading dom0 initrd from 0000000002b58000 to 0x0000000028200000-0x000000002b6a7fc9
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading dom0 DTB to 0x0000000028000000-0x0000000028006e66
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 280kB init memory.
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER4
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER8
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER12
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER16
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER20
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER0
(XEN) IRQ 48 is already used by domain 1
(XEN) IRQ 48 is already used by domain 1
[-- Attachment #7: xl -v create.txt --]
[-- Type: text/plain, Size: 3915 bytes --]
domainbuilder: detail: xc_dom_allocate: cmdline="(null)", features="(null)"
domainbuilder: detail: xc_dom_kernel_file: filename="ultraled.bin"
domainbuilder: detail: xc_dom_boot_xen_init: ver 4.9, caps xen-3.0-aarch64 xen-3.0-armv7l
domainbuilder: detail: xc_dom_rambase_init: RAM starts at 40000
domainbuilder: detail: xc_dom_parse_image: called
domainbuilder: detail: xc_dom_find_loader: trying multiboot-binary loader ...
domainbuilder: detail: loader probe failed
domainbuilder: detail: xc_dom_find_loader: trying Linux zImage (ARM64) loader ...
domainbuilder: detail: loader probe OK
domainbuilder: detail: xc_dom_parse_zimage64_kernel: called
domainbuilder: detail: xc_dom_parse_zimage64_kernel: xen-3.0-aarch64: 0x40000000 -> 0x4000b080
domainbuilder: detail: xc_dom_devicetree_mem: called
domainbuilder: detail: xc_dom_mem_init: mem 8 MB, pages 0x800 pages, 4k each
domainbuilder: detail: xc_dom_mem_init: 0x800 pages
domainbuilder: detail: xc_dom_boot_mem_init: called
domainbuilder: detail: set_mode: guest xen-3.0-aarch64, address size 64
domainbuilder: detail: populate_guest_memory: populating RAM @ 0000000040000000-0000000040800000 (8MB)
domainbuilder: detail: populate_one_size: populated 0x4/0x4 entries with shift 9
domainbuilder: detail: meminit: placing boot modules at 0x407ff000
domainbuilder: detail: meminit: devicetree: 0x407ff000 -> 0x40800000
domainbuilder: detail: xc_dom_build_image: called
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x40000+0xc at 0x7f91b8d000
domainbuilder: detail: xc_dom_alloc_segment: kernel : 0x40000000 -> 0x4000c000 (pfn 0x40000 + 0xc pages)
domainbuilder: detail: xc_dom_load_zimage_kernel: called
domainbuilder: detail: xc_dom_load_zimage_kernel: kernel seg 0x40000000-0x4000c000
domainbuilder: detail: xc_dom_load_zimage_kernel: copy 45184 bytes from blob 0x7f91b99000 to dst 0x7f91b8d000
domainbuilder: detail: xc_dom_pfn_to_ptr_retcount: domU mapping: pfn 0x407ff+0x1 at 0x7f92186000
domainbuilder: detail: xc_dom_alloc_segment: devicetree : 0x407ff000 -> 0x40800000 (pfn 0x407ff + 0x1 pages)
domainbuilder: detail: alloc_magic_pages: called
domainbuilder: detail: alloc_pgtables_arm: called
domainbuilder: detail: xc_dom_build_image : virt_alloc_end : 0x40800000
domainbuilder: detail: xc_dom_build_image : virt_pgtab_end : 0x0
domainbuilder: detail: xc_dom_boot_image: called
domainbuilder: detail: bootearly: doing nothing
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-aarch64 <= matches
domainbuilder: detail: xc_dom_compat_check: supported guest type: xen-3.0-armv7l
domainbuilder: detail: setup_pgtables_arm: called
domainbuilder: detail: clear_page: pfn 0x39000, mfn 0x39000
domainbuilder: detail: clear_page: pfn 0x39001, mfn 0x39001
domainbuilder: detail: start_info_arm: called
domainbuilder: detail: domain builder memory footprint
domainbuilder: detail: allocated
domainbuilder: detail: malloc : 19448 bytes
domainbuilder: detail: anon mmap : 0 bytes
domainbuilder: detail: mapped
domainbuilder: detail: file mmap : 44 kB
domainbuilder: detail: domU mmap : 52 kB
domainbuilder: detail: vcpu_arm64: called
domainbuilder: detail: DTB 407ff000
domainbuilder: detail: Initial state CPSR 0x1c5 PC 0x40000000
domainbuilder: detail: xc_dom_gnttab_hvm_seed: called, pfn=0x38000
domainbuilder: detail: xc_dom_release: called
(XEN) IRQ 48 is already used by domain 1
libxl: error: libxl_create.c:1278:domcreate_launch_dm: Domain 4:failed give domain access to irq 48: Device or resource busy
libxl: error: libxl_domain.c:1003:libxl__destroy_domid: Domain 4:Non-existant domain
libxl: error: libxl_domain.c:962:domain_destroy_callback: Domain 4:Unable to destroy guest
libxl: error: libxl_domain.c:889:domain_destroy_cb: Domain 4:Destruction of domain failed
[-- Attachment #8: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2018-10-01 12:03 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-12 23:03 null scheduler bug Stefano Stabellini
2018-09-13 7:04 ` Dario Faggioli
[not found] ` <CADJ6SV2G=51BK2p-bouNGfiS5sP2tiV6ztZZ7PFGjiptRw_P3w@mail.gmail.com>
2018-09-13 8:24 ` Dario Faggioli
2018-09-13 15:18 ` Milan Boberic
2018-09-13 17:39 ` Dario Faggioli
2018-09-14 9:21 ` Milan Boberic
2018-09-20 13:04 ` Milan Boberic
2018-09-20 16:09 ` Dario Faggioli
2018-09-21 14:14 ` Milan Boberic
2018-09-21 16:20 ` Dario Faggioli
2018-09-24 21:46 ` Julien Grall
2018-09-25 9:02 ` Dario Faggioli
2018-09-25 10:12 ` Milan Boberic
2018-09-25 10:56 ` Julien Grall
2018-09-25 11:15 ` Julien Grall
2018-09-25 11:51 ` Milan Boberic
2018-09-25 17:49 ` Dario Faggioli
2018-09-27 9:50 ` Dario Faggioli
2018-09-27 13:15 ` Milan Boberic
2018-09-27 14:23 ` Dario Faggioli
2018-09-27 14:32 ` Dario Faggioli
2018-09-27 15:09 ` Julien Grall
2018-09-27 17:06 ` Dario Faggioli
2018-09-28 8:18 ` Milan Boberic
2018-10-01 12:03 ` Julien Grall
2018-09-13 8:15 Milan Boberic
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.