[RFD] efi assisted kdump

[RFD] efi assisted kdump

[Dave Young]

Hi,

Kdump has several limitations currently such as kdump kernel reboot will bypass
device shutdown path so device drivers should reset during initialization.

* One of such problem we encounter now is the iommu
issue, 1st kernel's on fly DMA requests cause 2nd kernel hang. There's some effort
on this area, Zhenhua Li posted patches to resolve the intel iommu issue which is
under review. But that is just one case there's other possible problems in the future.

* There's no serial console on most machines in the market especially for desktop
machines and laptops. kms enabled kernel need drm layer driver for framebuffer
console, after kernel crashing we need a console to see the 2nd kernel output
ideally a serial console because we can not switch back to VGA mode.

ppc64 has a feature "firmware assisted kdump", see below documentation:
Documentation/powerpc/firmware-assisted-dump.txt

In case UEFI machines I wonder if we can do similar things, basic idea is doing
minimum thing based on original kdump process.

kernel reserve crashkernel memory during early boot
 -> user (kdump service) save necessary informations in some way so that second
    kernel boot can access ie. efi runtime variables or in reserved memory
    * crashkernel memory ranges infomation
    * collect infomations and save elf notes for 2nd kernel vmcore initialization
    -> crashing
       * call efi warm reboot
       -> 2nd kernel boot
          -> efi stub checking if this is a warm reboot and if this is a crashed boot
             -> if this is not a crashed boot then
                    do nothing more than original logic
                else
                    boot kernel with limited memory which was reserved in previous kernel
                    -> 2nd kernel boot
                       vmcore intializing

There's several things I need to ask for help from EFI gurus to see if it is doable:

* Make sure in case EFI warm reboot the memory of previous kernel can be retained.

* How to determine if kernel is boot with EFI warm reboot in stub

* What is the right way to pass informations from 1st kernel to 2nd kernel.
  Is it ok for saving something with efi variables? 

* Other possible problems what I missed

Thanks
Dave

Re: [RFD] efi assisted kdump

[Vivek Goyal]

On Sat, Jan 24, 2015 at 09:26:37PM +0800, Dave Young wrote:
> Hi,
> 
> Kdump has several limitations currently such as kdump kernel reboot will bypass
> device shutdown path so device drivers should reset during initialization.
> 
> * One of such problem we encounter now is the iommu
> issue, 1st kernel's on fly DMA requests cause 2nd kernel hang. There's some effort
> on this area, Zhenhua Li posted patches to resolve the intel iommu issue which is
> under review. But that is just one case there's other possible problems in the future.
> 
> * There's no serial console on most machines in the market especially for desktop
> machines and laptops. kms enabled kernel need drm layer driver for framebuffer
> console, after kernel crashing we need a console to see the 2nd kernel output
> ideally a serial console because we can not switch back to VGA mode.
> 
> ppc64 has a feature "firmware assisted kdump", see below documentation:
> Documentation/powerpc/firmware-assisted-dump.txt
> 
> In case UEFI machines I wonder if we can do similar things, basic idea is doing
> minimum thing based on original kdump process.
> 
> kernel reserve crashkernel memory during early boot
>  -> user (kdump service) save necessary informations in some way so that second
>     kernel boot can access ie. efi runtime variables or in reserved memory
>     * crashkernel memory ranges infomation
>     * collect infomations and save elf notes for 2nd kernel vmcore initialization

I think anything related to vmcore initilization can be in memory
somewhere and should not be a problem.

But we do have to think about anything which is passed in bootparams or
command line to second kenrnel with current mechanism, how will it be
passed to second kernel.

Thanks
Vivek

Re: [RFD] efi assisted kdump

[Matt Fleming]

On Sat, 24 Jan, at 09:26:37PM, Dave Young wrote:
> 
> There's several things I need to ask for help from EFI gurus to see if it is doable:
> 
> * Make sure in case EFI warm reboot the memory of previous kernel can be retained.
 
The only way to reserve memory across a reboot is with EFI capsules.

> * How to determine if kernel is boot with EFI warm reboot in stub

The presence of a capsule would indicate whether we were invoked from a
crash handler.
 
> * What is the right way to pass informations from 1st kernel to 2nd kernel.
>   Is it ok for saving something with efi variables? 

EFI capsules would the be most natural mechanism, but you could
conceivably do it with EFI variables (there's an upper limit on the size
of variable data, though).

> * Other possible problems what I missed

Support for runtime EFI capsules was rather spotty last time I looked.
Things may have moved on, but it's definitely something to watch out
for.

There have been patches on linux-efi in recent months for making use of
EFI capsules,

  https://lkml.kernel.org/r/1412692886-25306-1-git-send-email-matt-HNK1S37rvNbeXh+fF434Mdi2O/JbrIOy@public.gmane.org
  https://lkml.kernel.org/r/1414984030-13859-1-git-send-email-hock.leong.kweh-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org

I could take a look at resubmitting the first series above if that would
be useful for doing kexec across a reset.

-- 
Matt Fleming, Intel Open Source Technology Center

Re: [RFD] efi assisted kdump

[Dave Young]

On 01/26/15 at 09:08am, Vivek Goyal wrote:
> On Sat, Jan 24, 2015 at 09:26:37PM +0800, Dave Young wrote:
> > Hi,
> > 
> > Kdump has several limitations currently such as kdump kernel reboot will bypass
> > device shutdown path so device drivers should reset during initialization.
> > 
> > * One of such problem we encounter now is the iommu
> > issue, 1st kernel's on fly DMA requests cause 2nd kernel hang. There's some effort
> > on this area, Zhenhua Li posted patches to resolve the intel iommu issue which is
> > under review. But that is just one case there's other possible problems in the future.
> > 
> > * There's no serial console on most machines in the market especially for desktop
> > machines and laptops. kms enabled kernel need drm layer driver for framebuffer
> > console, after kernel crashing we need a console to see the 2nd kernel output
> > ideally a serial console because we can not switch back to VGA mode.
> > 
> > ppc64 has a feature "firmware assisted kdump", see below documentation:
> > Documentation/powerpc/firmware-assisted-dump.txt
> > 
> > In case UEFI machines I wonder if we can do similar things, basic idea is doing
> > minimum thing based on original kdump process.
> > 
> > kernel reserve crashkernel memory during early boot
> >  -> user (kdump service) save necessary informations in some way so that second
> >     kernel boot can access ie. efi runtime variables or in reserved memory
> >     * crashkernel memory ranges infomation
> >     * collect infomations and save elf notes for 2nd kernel vmcore initialization
> 
> I think anything related to vmcore initilization can be in memory
> somewhere and should not be a problem.
> 
> But we do have to think about anything which is passed in bootparams or
> command line to second kenrnel with current mechanism, how will it be
> passed to second kernel.

Hi, Vivek,

Since we are rebooting via efi maybe we can just choose another boot entry
in grub.cfg with the proper cmdline?

Thanks
Dave 

Re: [RFD] efi assisted kdump

[Dave Young]

Hi, Matt

Thanks for the feedback.

On 01/26/15 at 04:39pm, Matt Fleming wrote:
> On Sat, 24 Jan, at 09:26:37PM, Dave Young wrote:
> > 
> > There's several things I need to ask for help from EFI gurus to see if it is doable:
> > 
> > * Make sure in case EFI warm reboot the memory of previous kernel can be retained.
>  
> The only way to reserve memory across a reboot is with EFI capsules.

I know your capsules patchset, but I did not look into the details.

Do you means the old memory will be not reliable in case booting without capsules
even for warm reboot?

> 
> > * How to determine if kernel is boot with EFI warm reboot in stub
> 
> The presence of a capsule would indicate whether we were invoked from a
> crash handler.

Ok, another problem is for efi capsule handling, can we do it in efi stub, or I
in a efi application, which stage is the doable?

>  
> > * What is the right way to pass informations from 1st kernel to 2nd kernel.
> >   Is it ok for saving something with efi variables? 
> 
> EFI capsules would the be most natural mechanism, but you could
> conceivably do it with EFI variables (there's an upper limit on the size
> of variable data, though).

Will think about it after I checking the capsule machanisms details.

> 
> > * Other possible problems what I missed
> 
> Support for runtime EFI capsules was rather spotty last time I looked.
> Things may have moved on, but it's definitely something to watch out
> for.
> 
> There have been patches on linux-efi in recent months for making use of
> EFI capsules,
> 
>   https://lkml.kernel.org/r/1412692886-25306-1-git-send-email-matt-HNK1S37rvNbeXh+fF434Mdi2O/JbrIOy@public.gmane.org
>   https://lkml.kernel.org/r/1414984030-13859-1-git-send-email-hock.leong.kweh-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org
> 
> I could take a look at resubmitting the first series above if that would
> be useful for doing kexec across a reset.

Thanks a lot. I will take a look at how capsules works firstly.
Dave