Re: kexec not working in xen domU?

Re: kexec not working in xen domU?

[Guilherme G. Piccoli]

On Wed, Dec 9, 2020 at 4:13 PM Phillip Susi <phill@thesusis.net> wrote:
>
> Whenever I try to use kexec in a xen domU, the domain just reboots all
> the way through the bios rather than loading the kexec'ed kernel with
> the given command line.  Is this a known issue?  I've tried with both
> systemctl kexec and kexec -e.
>

Can you capture the serial console in a pastebin? Maybe add something
like "earlyprintk=ttySX", where ttySX is your known-to-work serial
console output. This helps to determine if it's a shutdown issue or an
early boot problem.
Also, worth to CC Xen mailing-list in this discussion I guess, I'm CCing them.
Cheers,


Guilherme

Re: kexec not working in xen domU?

[Phillip Susi]

Guilherme G. Piccoli writes:

> Can you capture the serial console in a pastebin? Maybe add something
> like "earlyprintk=ttySX", where ttySX is your known-to-work serial
> console output. This helps to determine if it's a shutdown issue or an
> early boot problem.

The regular xen cosole should work for this shouldn't it?  So
earlyprintk=hvc0 I guess?  I also threw in console=hvc0 and loglevel=7:

[  184.734810] systemd-shutdown[1]: Syncing filesystems and block
devices.
[  185.772511] systemd-shutdown[1]: Sending SIGTERM to remaining
processes...
[  185.896957] systemd-shutdown[1]: Sending SIGKILL to remaining
processes...
[  185.901111] systemd-shutdown[1]: Unmounting file systems.
[  185.902180] [1035]: Remounting '/' read-only in with options
'errors=remount-ro'.
[  185.990634] EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro
[  186.002373] systemd-shutdown[1]: All filesystems unmounted.
[  186.002411] systemd-shutdown[1]: Deactivating swaps.
[  186.002502] systemd-shutdown[1]: All swaps deactivated.
[  186.002529] systemd-shutdown[1]: Detaching loop devices.
[  186.002699] systemd-shutdown[1]: All loop devices detached.
[  186.002727] systemd-shutdown[1]: Stopping MD devices.
[  186.002814] systemd-shutdown[1]: All MD devices stopped.
[  186.002840] systemd-shutdown[1]: Detaching DM devices.
[  186.002974] systemd-shutdown[1]: All DM devices detached.
[  186.003017] systemd-shutdown[1]: All filesystems, swaps, loop
devices, MD devices and DM devices detached.
[  186.168475] systemd-shutdown[1]: Syncing filesystems and block
devices.
[  186.169150] systemd-shutdown[1]: Rebooting with kexec.
[  186.418653] xenbus_probe_frontend: xenbus_frontend_dev_shutdown:
device/vbd/5632: Initialising != Connected, skipping
[  186.427377] kexec_core: Starting new kernel

Re: kexec not working in xen domU?

[Guilherme G. Piccoli]

On Mon, Dec 14, 2020 at 5:25 PM Phillip Susi <phill@thesusis.net> wrote:
> The regular xen cosole should work for this shouldn't it?  So
> earlyprintk=hvc0 I guess?  I also threw in console=hvc0 and loglevel=7:
>
> [  184.734810] systemd-shutdown[1]: Syncing filesystems and block
> devices.
> [  185.772511] systemd-shutdown[1]: Sending SIGTERM to remaining
> processes...
> [  185.896957] systemd-shutdown[1]: Sending SIGKILL to remaining
> processes...
> [  185.901111] systemd-shutdown[1]: Unmounting file systems.
> [  185.902180] [1035]: Remounting '/' read-only in with options
> 'errors=remount-ro'.
> [  185.990634] EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro
> [  186.002373] systemd-shutdown[1]: All filesystems unmounted.
> [  186.002411] systemd-shutdown[1]: Deactivating swaps.
> [  186.002502] systemd-shutdown[1]: All swaps deactivated.
> [  186.002529] systemd-shutdown[1]: Detaching loop devices.
> [  186.002699] systemd-shutdown[1]: All loop devices detached.
> [  186.002727] systemd-shutdown[1]: Stopping MD devices.
> [  186.002814] systemd-shutdown[1]: All MD devices stopped.
> [  186.002840] systemd-shutdown[1]: Detaching DM devices.
> [  186.002974] systemd-shutdown[1]: All DM devices detached.
> [  186.003017] systemd-shutdown[1]: All filesystems, swaps, loop
> devices, MD devices and DM devices detached.
> [  186.168475] systemd-shutdown[1]: Syncing filesystems and block
> devices.
> [  186.169150] systemd-shutdown[1]: Rebooting with kexec.
> [  186.418653] xenbus_probe_frontend: xenbus_frontend_dev_shutdown:
> device/vbd/5632: Initialising != Connected, skipping
> [  186.427377] kexec_core: Starting new kernel
>

Hm..not many prints, either earlyprintk didn't work, or it's a really
early boot issue. Might worth to investigate if it's not a purgatory
issue too - did you try to use the ""new"" kexec syscall, by running
"kexec -s -l" instead of just "kexec -l" ?
Also, worth to try that with upstream kernel and kexec-tools - I
assume you're doing that already?

Cheers,


Guilherme

Re: kexec not working in xen domU?

[Phillip Susi]

Guilherme G. Piccoli writes:

> Hm..not many prints, either earlyprintk didn't work, or it's a really
> early boot issue. Might worth to investigate if it's not a purgatory
> issue too - did you try to use the ""new"" kexec syscall, by running
> "kexec -s -l" instead of just "kexec -l" ?
> Also, worth to try that with upstream kernel and kexec-tools - I
> assume you're doing that already?

I tried with -s and it didn't help.  So far I tried it originally on my
Ubuntu 20.04 amazon vps, then on my debian testing ( linux 5.9.0 ) on my
local xen server.  I'll try building the latest upstream kernel and
kexec tomorrow.

Re: kexec not working in xen domU?

[Phillip Susi]

Phillip Susi writes:

> I tried with -s and it didn't help.  So far I tried it originally on my
> Ubuntu 20.04 amazon vps, then on my debian testing ( linux 5.9.0 ) on my
> local xen server.  I'll try building the latest upstream kernel and
> kexec tomorrow.

Built the latest kexec-tools and linux kernel from git today and get
the same results.  Is there a minimum version of xen required for this
to work?  I have no idea what Amazon is running but my server has 4.9.