[PATCH v2] xen,kdump: handle pv domain in paddr_vmcoreinfo_note()

[PATCH v2] xen,kdump: handle pv domain in paddr_vmcoreinfo_note()

[Juergen Gross]

For kdump to work correctly it needs the physical address of
vmcoreinfo_note. When running as dom0 this means the virtual address
has to be translated to the related machine address.

paddr_vmcoreinfo_note() is meant to do the translation via
__pa_symbol() only, but being attributed "weak" it can be replaced
easily in Xen case.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
Changes in V2:
- use __pa_symbol() (Boris Ostrovsky)
- remove unneeded casts (Jan Beulich)

This patch needs to be rebased on top of Vitaly's series to split
pv- and hvm-code. I'll do this as soon as his series is in the Xen
tree in its final form.
---
 arch/x86/xen/mmu.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
index 37cb5aa..33ab96c 100644
--- a/arch/x86/xen/mmu.c
+++ b/arch/x86/xen/mmu.c
@@ -49,6 +49,9 @@
 #include <linux/memblock.h>
 #include <linux/seq_file.h>
 #include <linux/crash_dump.h>
+#ifdef CONFIG_KEXEC_CORE
+#include <linux/kexec.h>
+#endif
 
 #include <trace/events/xen.h>
 
@@ -2903,3 +2906,13 @@ int xen_unmap_domain_gfn_range(struct vm_area_struct *vma,
 	return -EINVAL;
 }
 EXPORT_SYMBOL_GPL(xen_unmap_domain_gfn_range);
+
+#ifdef CONFIG_KEXEC_CORE
+phys_addr_t paddr_vmcoreinfo_note(void)
+{
+	if (xen_pv_domain())
+		return virt_to_machine(&vmcoreinfo_note).maddr;
+	else
+		return __pa_symbol(&vmcoreinfo_note);
+}
+#endif /* CONFIG_KEXEC_CORE */
-- 
2.10.2

Re: [PATCH v2] xen,kdump: handle pv domain in paddr_vmcoreinfo_note()

[Boris Ostrovsky]

On 03/31/2017 06:14 AM, Juergen Gross wrote:
> For kdump to work correctly it needs the physical address of
> vmcoreinfo_note. When running as dom0 this means the virtual address
> has to be translated to the related machine address.
>
> paddr_vmcoreinfo_note() is meant to do the translation via
> __pa_symbol() only, but being attributed "weak" it can be replaced
> easily in Xen case.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Similar to Jan's concern, if bare-metal x86 people decide to have their
own paddr_vmcoreinfo_note() (and they usually build with !CONFIG_XEN) we
will have to update this again. I suppose we can deal with that if/when
it happens since we will discover this immediately.

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Boris Ostrovsky]

On 03/31/2017 06:14 AM, Juergen Gross wrote:
> For kdump to work correctly it needs the physical address of
> vmcoreinfo_note. When running as dom0 this means the virtual address
> has to be translated to the related machine address.
>
> paddr_vmcoreinfo_note() is meant to do the translation via
> __pa_symbol() only, but being attributed "weak" it can be replaced
> easily in Xen case.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Similar to Jan's concern, if bare-metal x86 people decide to have their
own paddr_vmcoreinfo_note() (and they usually build with !CONFIG_XEN) we
will have to update this again. I suppose we can deal with that if/when
it happens since we will discover this immediately.

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> For kdump to work correctly it needs the physical address of
> vmcoreinfo_note. When running as dom0 this means the virtual address
> has to be translated to the related machine address.
>
> paddr_vmcoreinfo_note() is meant to do the translation via
> __pa_symbol() only, but being attributed "weak" it can be replaced
> easily in Xen case.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Have you tested this patch with latest crash tool? Do dom0 and Xen
hypervisor analysis work without any issue (at least basic commands
like dmesg, bt, ps, etc.)? If yes for both you can add:

Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

Daniel

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> For kdump to work correctly it needs the physical address of
> vmcoreinfo_note. When running as dom0 this means the virtual address
> has to be translated to the related machine address.
>
> paddr_vmcoreinfo_note() is meant to do the translation via
> __pa_symbol() only, but being attributed "weak" it can be replaced
> easily in Xen case.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Have you tested this patch with latest crash tool? Do dom0 and Xen
hypervisor analysis work without any issue (at least basic commands
like dmesg, bt, ps, etc.)? If yes for both you can add:

Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Juergen Gross]

On 03/04/17 14:42, Daniel Kiper wrote:
> On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
>> For kdump to work correctly it needs the physical address of
>> vmcoreinfo_note. When running as dom0 this means the virtual address
>> has to be translated to the related machine address.
>>
>> paddr_vmcoreinfo_note() is meant to do the translation via
>> __pa_symbol() only, but being attributed "weak" it can be replaced
>> easily in Xen case.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Have you tested this patch with latest crash tool? Do dom0 and Xen
> hypervisor analysis work without any issue (at least basic commands
> like dmesg, bt, ps, etc.)? If yes for both you can add:
> 
> Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

This patch isn't for dump analysis, but for dump creation. Petr has
verified that the dump is in the expected format. Please ask Petr
for further details, e.g. user side modifications being necessary.


Juergen

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Juergen Gross]

On 03/04/17 14:42, Daniel Kiper wrote:
> On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
>> For kdump to work correctly it needs the physical address of
>> vmcoreinfo_note. When running as dom0 this means the virtual address
>> has to be translated to the related machine address.
>>
>> paddr_vmcoreinfo_note() is meant to do the translation via
>> __pa_symbol() only, but being attributed "weak" it can be replaced
>> easily in Xen case.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Have you tested this patch with latest crash tool? Do dom0 and Xen
> hypervisor analysis work without any issue (at least basic commands
> like dmesg, bt, ps, etc.)? If yes for both you can add:
> 
> Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

This patch isn't for dump analysis, but for dump creation. Petr has
verified that the dump is in the expected format. Please ask Petr
for further details, e.g. user side modifications being necessary.


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Juergen Gross]

On 03/04/17 14:42, Daniel Kiper wrote:
> On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
>> For kdump to work correctly it needs the physical address of
>> vmcoreinfo_note. When running as dom0 this means the virtual address
>> has to be translated to the related machine address.
>>
>> paddr_vmcoreinfo_note() is meant to do the translation via
>> __pa_symbol() only, but being attributed "weak" it can be replaced
>> easily in Xen case.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Have you tested this patch with latest crash tool? Do dom0 and Xen
> hypervisor analysis work without any issue (at least basic commands
> like dmesg, bt, ps, etc.)? If yes for both you can add:
> 
> Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

So can I add your R-b: now?


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Juergen Gross]

On 03/04/17 14:42, Daniel Kiper wrote:
> On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
>> For kdump to work correctly it needs the physical address of
>> vmcoreinfo_note. When running as dom0 this means the virtual address
>> has to be translated to the related machine address.
>>
>> paddr_vmcoreinfo_note() is meant to do the translation via
>> __pa_symbol() only, but being attributed "weak" it can be replaced
>> easily in Xen case.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
> 
> Have you tested this patch with latest crash tool? Do dom0 and Xen
> hypervisor analysis work without any issue (at least basic commands
> like dmesg, bt, ps, etc.)? If yes for both you can add:
> 
> Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>

So can I add your R-b: now?


Juergen

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Tue, Apr 11, 2017 at 02:45:43PM +0200, Juergen Gross wrote:
> On 03/04/17 14:42, Daniel Kiper wrote:
> > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> >> For kdump to work correctly it needs the physical address of
> >> vmcoreinfo_note. When running as dom0 this means the virtual address
> >> has to be translated to the related machine address.
> >>
> >> paddr_vmcoreinfo_note() is meant to do the translation via
> >> __pa_symbol() only, but being attributed "weak" it can be replaced
> >> easily in Xen case.
> >>
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> >
> > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > hypervisor analysis work without any issue (at least basic commands
> > like dmesg, bt, ps, etc.)? If yes for both you can add:
> >
> > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
>
> So can I add your R-b: now?

My R-b is still valid. Though, let's wait for Petr's Tested-by. He
did makedumpfile tests but I asked him to do crash tool tests too.
I think it is important.

Daniel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Tue, 11 Apr 2017 15:00:58 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Tue, Apr 11, 2017 at 02:45:43PM +0200, Juergen Gross wrote:
> > On 03/04/17 14:42, Daniel Kiper wrote:
> > > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> > >> For kdump to work correctly it needs the physical address of
> > >> vmcoreinfo_note. When running as dom0 this means the virtual address
> > >> has to be translated to the related machine address.
> > >>
> > >> paddr_vmcoreinfo_note() is meant to do the translation via
> > >> __pa_symbol() only, but being attributed "weak" it can be replaced
> > >> easily in Xen case.
> > >>
> > >> Signed-off-by: Juergen Gross <jgross@suse.com>
> > >
> > > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > > hypervisor analysis work without any issue (at least basic commands
> > > like dmesg, bt, ps, etc.)? If yes for both you can add:
> > >
> > > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
> >
> > So can I add your R-b: now?
> 
> My R-b is still valid. Though, let's wait for Petr's Tested-by. He
> did makedumpfile tests but I asked him to do crash tool tests too.
> I think it is important.

Tested-by: Petr Tesarik <ptesarik@suse.com>

I copied the complete /proc/vmcore to a directory on disk. Exactly
as expected, crash works both without the patch and with the patch, as
it does not use VMCOREINFO at all (instead, crash obtains the
information from kernel debuginfo directly).

Without the patch:

morricone:/abuild/dumps/2017-04-11-16:38/:[0]# crash vmlinux vmcore 

crash 7.1.8
Copyright (C) 2002-2016  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
crash: vmlinux: no .gnu_debuglink section
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL: vmlinux                           
    DUMPFILE: vmcore
        CPUS: 12 [OFFLINE: 11]
        DATE: Tue Apr 11 16:37:34 2017
      UPTIME: 00:11:51
LOAD AVERAGE: 0.04, 0.08, 0.08
       TASKS: 269
    NODENAME: morricone
     RELEASE: 4.11.0-rc5-default-pt+
     VERSION: #1 SMP PREEMPT Thu Apr 6 17:40:53 CEST 2017
     MACHINE: x86_64  (2400 Mhz)
      MEMORY: 16 GB
       PANIC: "sysrq: SysRq : Trigger a crash"
         PID: 12009
     COMMAND: "bash"
        TASK: ffff8804065d2080  [THREAD_INFO: ffff8804065d2080]
         CPU: 7
       STATE: TASK_RUNNING (SYSRQ)

crash> 


With the patch included:

morricone:/abuild/dumps/2017-04-11-12:52/:[0]# crash vmlinux vmcore  

crash 7.1.8
Copyright (C) 2002-2016  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
crash: vmlinux: no .gnu_debuglink section
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL: vmlinux                           
    DUMPFILE: vmcore
        CPUS: 12 [OFFLINE: 11]
        DATE: Tue Apr 11 12:51:15 2017
      UPTIME: 4 days, 01:31:42
LOAD AVERAGE: 0.04, 0.01, 0.00
       TASKS: 634
    NODENAME: morricone
     RELEASE: 4.11.0-rc5-default-pt+
     VERSION: #2 SMP PREEMPT Fri Apr 7 09:34:10 CEST 2017
     MACHINE: x86_64  (2400 Mhz)
      MEMORY: 16 GB
       PANIC: "sysrq: SysRq : Trigger a crash"
         PID: 19503
     COMMAND: "bash"
        TASK: ffff8803367762c0  [THREAD_INFO: ffff8803367762c0]
         CPU: 3
       STATE: TASK_RUNNING (SYSRQ)

crash> 

Petr T

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Tue, Apr 11, 2017 at 04:59:16PM +0200, Petr Tesarik wrote:
> On Tue, 11 Apr 2017 15:00:58 +0200
> Daniel Kiper <daniel.kiper@oracle.com> wrote:
>
> > On Tue, Apr 11, 2017 at 02:45:43PM +0200, Juergen Gross wrote:
> > > On 03/04/17 14:42, Daniel Kiper wrote:
> > > > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> > > >> For kdump to work correctly it needs the physical address of
> > > >> vmcoreinfo_note. When running as dom0 this means the virtual address
> > > >> has to be translated to the related machine address.
> > > >>
> > > >> paddr_vmcoreinfo_note() is meant to do the translation via
> > > >> __pa_symbol() only, but being attributed "weak" it can be replaced
> > > >> easily in Xen case.
> > > >>
> > > >> Signed-off-by: Juergen Gross <jgross@suse.com>
> > > >
> > > > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > > > hypervisor analysis work without any issue (at least basic commands
> > > > like dmesg, bt, ps, etc.)? If yes for both you can add:
> > > >
> > > > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
> > >
> > > So can I add your R-b: now?
> >
> > My R-b is still valid. Though, let's wait for Petr's Tested-by. He
> > did makedumpfile tests but I asked him to do crash tool tests too.
> > I think it is important.
>
> Tested-by: Petr Tesarik <ptesarik@suse.com>
>
> I copied the complete /proc/vmcore to a directory on disk. Exactly
> as expected, crash works both without the patch and with the patch, as
> it does not use VMCOREINFO at all (instead, crash obtains the
> information from kernel debuginfo directly).

Thanks for doing the tests. I suppose that you have tested HVM guests.
IIRC, PV guests are not supported by crash right now due to p2m VMA
mapping. At least it was an issue some time ago. Is it still valid?
Anyway, one guy in Oracle works on fix for that issue and I do review.
We are going to post it in 2-3 weeks.

Juergen, I do not have any objections any longer. You can go ahead.
Ahhh... I see. You have posted v3. Good!

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Tue, Apr 11, 2017 at 04:59:16PM +0200, Petr Tesarik wrote:
> On Tue, 11 Apr 2017 15:00:58 +0200
> Daniel Kiper <daniel.kiper@oracle.com> wrote:
>
> > On Tue, Apr 11, 2017 at 02:45:43PM +0200, Juergen Gross wrote:
> > > On 03/04/17 14:42, Daniel Kiper wrote:
> > > > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> > > >> For kdump to work correctly it needs the physical address of
> > > >> vmcoreinfo_note. When running as dom0 this means the virtual address
> > > >> has to be translated to the related machine address.
> > > >>
> > > >> paddr_vmcoreinfo_note() is meant to do the translation via
> > > >> __pa_symbol() only, but being attributed "weak" it can be replaced
> > > >> easily in Xen case.
> > > >>
> > > >> Signed-off-by: Juergen Gross <jgross@suse.com>
> > > >
> > > > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > > > hypervisor analysis work without any issue (at least basic commands
> > > > like dmesg, bt, ps, etc.)? If yes for both you can add:
> > > >
> > > > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
> > >
> > > So can I add your R-b: now?
> >
> > My R-b is still valid. Though, let's wait for Petr's Tested-by. He
> > did makedumpfile tests but I asked him to do crash tool tests too.
> > I think it is important.
>
> Tested-by: Petr Tesarik <ptesarik@suse.com>
>
> I copied the complete /proc/vmcore to a directory on disk. Exactly
> as expected, crash works both without the patch and with the patch, as
> it does not use VMCOREINFO at all (instead, crash obtains the
> information from kernel debuginfo directly).

Thanks for doing the tests. I suppose that you have tested HVM guests.
IIRC, PV guests are not supported by crash right now due to p2m VMA
mapping. At least it was an issue some time ago. Is it still valid?
Anyway, one guy in Oracle works on fix for that issue and I do review.
We are going to post it in 2-3 weeks.

Juergen, I do not have any objections any longer. You can go ahead.
Ahhh... I see. You have posted v3. Good!

Daniel

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Tue, 11 Apr 2017 19:20:08 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Tue, Apr 11, 2017 at 04:59:16PM +0200, Petr Tesarik wrote:
>[...]
> > Tested-by: Petr Tesarik <ptesarik@suse.com>
> >
> > I copied the complete /proc/vmcore to a directory on disk. Exactly
> > as expected, crash works both without the patch and with the patch, as
> > it does not use VMCOREINFO at all (instead, crash obtains the
> > information from kernel debuginfo directly).
> 
> Thanks for doing the tests. I suppose that you have tested HVM guests.

Not really. I crashed Dom0, which is in turn sent to the hypervisor, so
the result is a complete host dump, including Xen hypervisor data and
all domains.

> IIRC, PV guests are not supported by crash right now due to p2m VMA
> mapping. At least it was an issue some time ago. Is it still valid?

Yes, this is correct. I tested this behaviour a few weeks ago.

> Anyway, one guy in Oracle works on fix for that issue and I do review.
> We are going to post it in 2-3 weeks.

All right. FYI I do not plan to put much effort into it, as my focus has
shifted towards libkdumpfile (https://github.com/ptesarik/libkdumpfile),
and this library can open PV guest dump files without any issues.

Petr T

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Tue, 11 Apr 2017 19:20:08 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Tue, Apr 11, 2017 at 04:59:16PM +0200, Petr Tesarik wrote:
>[...]
> > Tested-by: Petr Tesarik <ptesarik@suse.com>
> >
> > I copied the complete /proc/vmcore to a directory on disk. Exactly
> > as expected, crash works both without the patch and with the patch, as
> > it does not use VMCOREINFO at all (instead, crash obtains the
> > information from kernel debuginfo directly).
> 
> Thanks for doing the tests. I suppose that you have tested HVM guests.

Not really. I crashed Dom0, which is in turn sent to the hypervisor, so
the result is a complete host dump, including Xen hypervisor data and
all domains.

> IIRC, PV guests are not supported by crash right now due to p2m VMA
> mapping. At least it was an issue some time ago. Is it still valid?

Yes, this is correct. I tested this behaviour a few weeks ago.

> Anyway, one guy in Oracle works on fix for that issue and I do review.
> We are going to post it in 2-3 weeks.

All right. FYI I do not plan to put much effort into it, as my focus has
shifted towards libkdumpfile (https://github.com/ptesarik/libkdumpfile),
and this library can open PV guest dump files without any issues.

Petr T

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Fri, Apr 14, 2017 at 06:53:36PM +0200, Petr Tesarik wrote:
> On Tue, 11 Apr 2017 19:20:08 +0200
> Daniel Kiper <daniel.kiper@oracle.com> wrote:
> > On Tue, Apr 11, 2017 at 04:59:16PM +0200, Petr Tesarik wrote:
> >[...]
> > > Tested-by: Petr Tesarik <ptesarik@suse.com>
> > >
> > > I copied the complete /proc/vmcore to a directory on disk. Exactly
> > > as expected, crash works both without the patch and with the patch, as
> > > it does not use VMCOREINFO at all (instead, crash obtains the
> > > information from kernel debuginfo directly).
> >
> > Thanks for doing the tests. I suppose that you have tested HVM guests.
>
> Not really. I crashed Dom0, which is in turn sent to the hypervisor, so
> the result is a complete host dump, including Xen hypervisor data and
> all domains.

OK.

> > IIRC, PV guests are not supported by crash right now due to p2m VMA
> > mapping. At least it was an issue some time ago. Is it still valid?
>
> Yes, this is correct. I tested this behaviour a few weeks ago.

Thanks for update.

> > Anyway, one guy in Oracle works on fix for that issue and I do review.
> > We are going to post it in 2-3 weeks.
>
> All right. FYI I do not plan to put much effort into it, as my focus has

OK.

> shifted towards libkdumpfile (https://github.com/ptesarik/libkdumpfile),
> and this library can open PV guest dump files without any issues.

Great! AIUI, it reminds my idea to make such think. However, I have not
have time to make it happen. Is it based on makedumpfile or written from
scratch? Do you plan support for Linux kernel dumps and/or Xen ones?

Daniel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Sat, 15 Apr 2017 00:26:05 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Fri, Apr 14, 2017 at 06:53:36PM +0200, Petr Tesarik wrote:
>[...]
> > shifted towards libkdumpfile (https://github.com/ptesarik/libkdumpfile),
> > and this library can open PV guest dump files without any issues.
> 
> Great! AIUI, it reminds my idea to make such think. However, I have not
> have time to make it happen. Is it based on makedumpfile or written from
> scratch? Do you plan support for Linux kernel dumps and/or Xen ones?

Some ideas are borrowed from existing tools (makedumpfile, crash). All
code is written from scratch, however.

The kdumpfile library itself is designed for use with any platform and
operating system. Xen is treated as just another type of operating
system. Based on which OS type is initialized, the library is able to
provide a hypervisor view or a Dom0 view.

There is another project led by Jeff Mahoney, which extends standard
gdb with semantic commands. This project supports only x86_64 Linux
right now. See https://github.com/jeffmahoney/crash-python.

Petr T

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Sat, 15 Apr 2017 00:26:05 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Fri, Apr 14, 2017 at 06:53:36PM +0200, Petr Tesarik wrote:
>[...]
> > shifted towards libkdumpfile (https://github.com/ptesarik/libkdumpfile),
> > and this library can open PV guest dump files without any issues.
> 
> Great! AIUI, it reminds my idea to make such think. However, I have not
> have time to make it happen. Is it based on makedumpfile or written from
> scratch? Do you plan support for Linux kernel dumps and/or Xen ones?

Some ideas are borrowed from existing tools (makedumpfile, crash). All
code is written from scratch, however.

The kdumpfile library itself is designed for use with any platform and
operating system. Xen is treated as just another type of operating
system. Based on which OS type is initialized, the library is able to
provide a hypervisor view or a Dom0 view.

There is another project led by Jeff Mahoney, which extends standard
gdb with semantic commands. This project supports only x86_64 Linux
right now. See https://github.com/jeffmahoney/crash-python.

Petr T

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Fri, Apr 14, 2017 at 06:53:36PM +0200, Petr Tesarik wrote:
> On Tue, 11 Apr 2017 19:20:08 +0200
> Daniel Kiper <daniel.kiper@oracle.com> wrote:
> > On Tue, Apr 11, 2017 at 04:59:16PM +0200, Petr Tesarik wrote:
> >[...]
> > > Tested-by: Petr Tesarik <ptesarik@suse.com>
> > >
> > > I copied the complete /proc/vmcore to a directory on disk. Exactly
> > > as expected, crash works both without the patch and with the patch, as
> > > it does not use VMCOREINFO at all (instead, crash obtains the
> > > information from kernel debuginfo directly).
> >
> > Thanks for doing the tests. I suppose that you have tested HVM guests.
>
> Not really. I crashed Dom0, which is in turn sent to the hypervisor, so
> the result is a complete host dump, including Xen hypervisor data and
> all domains.

OK.

> > IIRC, PV guests are not supported by crash right now due to p2m VMA
> > mapping. At least it was an issue some time ago. Is it still valid?
>
> Yes, this is correct. I tested this behaviour a few weeks ago.

Thanks for update.

> > Anyway, one guy in Oracle works on fix for that issue and I do review.
> > We are going to post it in 2-3 weeks.
>
> All right. FYI I do not plan to put much effort into it, as my focus has

OK.

> shifted towards libkdumpfile (https://github.com/ptesarik/libkdumpfile),
> and this library can open PV guest dump files without any issues.

Great! AIUI, it reminds my idea to make such think. However, I have not
have time to make it happen. Is it based on makedumpfile or written from
scratch? Do you plan support for Linux kernel dumps and/or Xen ones?

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Tue, 11 Apr 2017 15:00:58 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Tue, Apr 11, 2017 at 02:45:43PM +0200, Juergen Gross wrote:
> > On 03/04/17 14:42, Daniel Kiper wrote:
> > > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> > >> For kdump to work correctly it needs the physical address of
> > >> vmcoreinfo_note. When running as dom0 this means the virtual address
> > >> has to be translated to the related machine address.
> > >>
> > >> paddr_vmcoreinfo_note() is meant to do the translation via
> > >> __pa_symbol() only, but being attributed "weak" it can be replaced
> > >> easily in Xen case.
> > >>
> > >> Signed-off-by: Juergen Gross <jgross@suse.com>
> > >
> > > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > > hypervisor analysis work without any issue (at least basic commands
> > > like dmesg, bt, ps, etc.)? If yes for both you can add:
> > >
> > > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
> >
> > So can I add your R-b: now?
> 
> My R-b is still valid. Though, let's wait for Petr's Tested-by. He
> did makedumpfile tests but I asked him to do crash tool tests too.
> I think it is important.

Tested-by: Petr Tesarik <ptesarik@suse.com>

I copied the complete /proc/vmcore to a directory on disk. Exactly
as expected, crash works both without the patch and with the patch, as
it does not use VMCOREINFO at all (instead, crash obtains the
information from kernel debuginfo directly).

Without the patch:

morricone:/abuild/dumps/2017-04-11-16:38/:[0]# crash vmlinux vmcore 

crash 7.1.8
Copyright (C) 2002-2016  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
crash: vmlinux: no .gnu_debuglink section
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL: vmlinux                           
    DUMPFILE: vmcore
        CPUS: 12 [OFFLINE: 11]
        DATE: Tue Apr 11 16:37:34 2017
      UPTIME: 00:11:51
LOAD AVERAGE: 0.04, 0.08, 0.08
       TASKS: 269
    NODENAME: morricone
     RELEASE: 4.11.0-rc5-default-pt+
     VERSION: #1 SMP PREEMPT Thu Apr 6 17:40:53 CEST 2017
     MACHINE: x86_64  (2400 Mhz)
      MEMORY: 16 GB
       PANIC: "sysrq: SysRq : Trigger a crash"
         PID: 12009
     COMMAND: "bash"
        TASK: ffff8804065d2080  [THREAD_INFO: ffff8804065d2080]
         CPU: 7
       STATE: TASK_RUNNING (SYSRQ)

crash> 


With the patch included:

morricone:/abuild/dumps/2017-04-11-12:52/:[0]# crash vmlinux vmcore  

crash 7.1.8
Copyright (C) 2002-2016  Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010  IBM Corporation
Copyright (C) 1999-2006  Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012  Fujitsu Limited
Copyright (C) 2006, 2007  VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011  NEC Corporation
Copyright (C) 1999, 2002, 2007  Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002  Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions.  Enter "help copying" to see the conditions.
This program has absolutely no warranty.  Enter "help warranty" for details.
 
crash: vmlinux: no .gnu_debuglink section
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...

      KERNEL: vmlinux                           
    DUMPFILE: vmcore
        CPUS: 12 [OFFLINE: 11]
        DATE: Tue Apr 11 12:51:15 2017
      UPTIME: 4 days, 01:31:42
LOAD AVERAGE: 0.04, 0.01, 0.00
       TASKS: 634
    NODENAME: morricone
     RELEASE: 4.11.0-rc5-default-pt+
     VERSION: #2 SMP PREEMPT Fri Apr 7 09:34:10 CEST 2017
     MACHINE: x86_64  (2400 Mhz)
      MEMORY: 16 GB
       PANIC: "sysrq: SysRq : Trigger a crash"
         PID: 19503
     COMMAND: "bash"
        TASK: ffff8803367762c0  [THREAD_INFO: ffff8803367762c0]
         CPU: 3
       STATE: TASK_RUNNING (SYSRQ)

crash> 

Petr T

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Tue, Apr 11, 2017 at 02:45:43PM +0200, Juergen Gross wrote:
> On 03/04/17 14:42, Daniel Kiper wrote:
> > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> >> For kdump to work correctly it needs the physical address of
> >> vmcoreinfo_note. When running as dom0 this means the virtual address
> >> has to be translated to the related machine address.
> >>
> >> paddr_vmcoreinfo_note() is meant to do the translation via
> >> __pa_symbol() only, but being attributed "weak" it can be replaced
> >> easily in Xen case.
> >>
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> >
> > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > hypervisor analysis work without any issue (at least basic commands
> > like dmesg, bt, ps, etc.)? If yes for both you can add:
> >
> > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
>
> So can I add your R-b: now?

My R-b is still valid. Though, let's wait for Petr's Tested-by. He
did makedumpfile tests but I asked him to do crash tool tests too.
I think it is important.

Daniel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

> On 03/04/17 14:42, Daniel Kiper wrote:
> > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> >> For kdump to work correctly it needs the physical address of
> >> vmcoreinfo_note. When running as dom0 this means the virtual address
> >> has to be translated to the related machine address.
> >>
> >> paddr_vmcoreinfo_note() is meant to do the translation via
> >> __pa_symbol() only, but being attributed "weak" it can be replaced
> >> easily in Xen case.
> >>
> >> Signed-off-by: Juergen Gross <jgross@suse.com>
> >
> > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > hypervisor analysis work without any issue (at least basic commands
> > like dmesg, bt, ps, etc.)? If yes for both you can add:
> >
> > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
>
> This patch isn't for dump analysis, but for dump creation. Petr has

I know that. However, it may have impact on crash analysis. So,
I would expect that you or anybody else in your behalf will do
at least minimal crash tool tests.

> verified that the dump is in the expected format. Please ask Petr
> for further details, e.g. user side modifications being necessary.

So, if Petr did relevant tests that is nice. However, then, IMO, this
patch begs Petr Tested-by.

Daniel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Tue, 4 Apr 2017 12:42:53 -0700 (PDT)
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> > On 03/04/17 14:42, Daniel Kiper wrote:
> > > On Fri, Mar 31, 2017 at 12:14:38PM +0200, Juergen Gross wrote:
> > >> For kdump to work correctly it needs the physical address of
> > >> vmcoreinfo_note. When running as dom0 this means the virtual address
> > >> has to be translated to the related machine address.
> > >>
> > >> paddr_vmcoreinfo_note() is meant to do the translation via
> > >> __pa_symbol() only, but being attributed "weak" it can be replaced
> > >> easily in Xen case.
> > >>
> > >> Signed-off-by: Juergen Gross <jgross@suse.com>
> > >
> > > Have you tested this patch with latest crash tool? Do dom0 and Xen
> > > hypervisor analysis work without any issue (at least basic commands
> > > like dmesg, bt, ps, etc.)? If yes for both you can add:
> > >
> > > Reviewed-by: Daniel Kiper <daniel.kiper@oracle.com>
> >
> > This patch isn't for dump analysis, but for dump creation. Petr has
> 
> I know that. However, it may have impact on crash analysis. So,
> I would expect that you or anybody else in your behalf will do
> at least minimal crash tool tests.
> 
> > verified that the dump is in the expected format. Please ask Petr
> > for further details, e.g. user side modifications being necessary.
> 
> So, if Petr did relevant tests that is nice. However, then, IMO, this
> patch begs Petr Tested-by.

Actually, I tested with this patch applied on top of kernel 4.4 (SLES
12 SP2). It matches what traditional Xen had always done, so I am quite
confident it will work with a later kernel, but to give my Tested-by,
let me first re-run the test on master, hopefully until today EOB.

HTH,
Petr T

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Wed, 5 Apr 2017 13:13:00 +0200
Petr Tesarik <ptesarik@suse.com> wrote:

> On Tue, 4 Apr 2017 12:42:53 -0700 (PDT)
> Daniel Kiper <daniel.kiper@oracle.com> wrote:
> 
>[...]
> > So, if Petr did relevant tests that is nice. However, then, IMO, this
> > patch begs Petr Tested-by.
> 
> Actually, I tested with this patch applied on top of kernel 4.4 (SLES
> 12 SP2). It matches what traditional Xen had always done, so I am quite
> confident it will work with a later kernel, but to give my Tested-by,
> let me first re-run the test on master, hopefully until today EOB.

It took me much longer than anticipated (I had some trouble setting up
the host again), but I can confirm that the patch works as expected on
top of 4.11-rc5.

Without the patch, makedumpfile in the crash kernel complains:

/proc/vmcore doesn't contain vmcoreinfo.

With the patch applied, dumping still fails later because of an
unrelated bug in makedumpfile, but I was able to extract the kernel
message buffer with "makedumpfile --dump-dmesg". This already confirms
VMCOREINFO presence and usability.

So finally,

Tested-by: Petr Tesarik <ptesarik@suse.com>

Petr Tesarik

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Daniel Kiper]

On Fri, Apr 07, 2017 at 11:16:22AM +0200, Petr Tesarik wrote:
> On Wed, 5 Apr 2017 13:13:00 +0200
> Petr Tesarik <ptesarik@suse.com> wrote:
>
> > On Tue, 4 Apr 2017 12:42:53 -0700 (PDT)
> > Daniel Kiper <daniel.kiper@oracle.com> wrote:
> >
> >[...]
> > > So, if Petr did relevant tests that is nice. However, then, IMO, this
> > > patch begs Petr Tested-by.
> >
> > Actually, I tested with this patch applied on top of kernel 4.4 (SLES
> > 12 SP2). It matches what traditional Xen had always done, so I am quite
> > confident it will work with a later kernel, but to give my Tested-by,
> > let me first re-run the test on master, hopefully until today EOB.
>
> It took me much longer than anticipated (I had some trouble setting up
> the host again), but I can confirm that the patch works as expected on

No problem. I know how it works.

> top of 4.11-rc5.

Great!

> Without the patch, makedumpfile in the crash kernel complains:
>
> /proc/vmcore doesn't contain vmcoreinfo.

Though, I would like to ask you to do crash tool tests too.
Could you do that?

> With the patch applied, dumping still fails later because of an
> unrelated bug in makedumpfile, but I was able to extract the kernel
> message buffer with "makedumpfile --dump-dmesg". This already confirms
> VMCOREINFO presence and usability.

Is it Xen specific issue or more generic one?

Daniel

Re: [Xen-devel] [PATCH v2] xen, kdump: handle pv domain in paddr_vmcoreinfo_note()

[Petr Tesarik]

On Mon, 10 Apr 2017 22:49:33 +0200
Daniel Kiper <daniel.kiper@oracle.com> wrote:

> On Fri, Apr 07, 2017 at 11:16:22AM +0200, Petr Tesarik wrote:
> > On Wed, 5 Apr 2017 13:13:00 +0200
> > Petr Tesarik <ptesarik@suse.com> wrote:
> >
> > > On Tue, 4 Apr 2017 12:42:53 -0700 (PDT)
> > > Daniel Kiper <daniel.kiper@oracle.com> wrote:
> > >
> > >[...]
> > > > So, if Petr did relevant tests that is nice. However, then, IMO, this
> > > > patch begs Petr Tested-by.
> > >
> > > Actually, I tested with this patch applied on top of kernel 4.4 (SLES
> > > 12 SP2). It matches what traditional Xen had always done, so I am quite
> > > confident it will work with a later kernel, but to give my Tested-by,
> > > let me first re-run the test on master, hopefully until today EOB.
> >
> > It took me much longer than anticipated (I had some trouble setting up
> > the host again), but I can confirm that the patch works as expected on
> 
> No problem. I know how it works.
> 
> > top of 4.11-rc5.
> 
> Great!
> 
> > Without the patch, makedumpfile in the crash kernel complains:
> >
> > /proc/vmcore doesn't contain vmcoreinfo.
> 
> Though, I would like to ask you to do crash tool tests too.
> Could you do that?

I'm not sure what kind of tests you want me to perform. I'm not able to
save a dump file without the patch.

Well ... I could reserve enough memory to launch a crash session
directly in the kdump environment. I'll get to you later.

> > With the patch applied, dumping still fails later because of an
> > unrelated bug in makedumpfile, but I was able to extract the kernel
> > message buffer with "makedumpfile --dump-dmesg". This already confirms
> > VMCOREINFO presence and usability.
> 
> Is it Xen specific issue or more generic one?

It is Xen-specific, and I even have a patch now, about to be sent to
the kexec ML.

Petr T