Dmesg not being dumped

Dmesg not being dumped

[Nikolay Borisov]

Hello,

I've recently noticed that when creating crashdumps the dmesg is not
being saved. The error reported is this: "Missing the struct log size
export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
been compiled with debugging info so the respective symbol should be
exported but apparently it is not. Any ideas how to debug that?

[Please CC me as I'm not subscribed to the mailing list.]

Regards,
Nikolay

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Baoquan He]

On 08/19/15 at 09:21am, Nikolay Borisov wrote:
> Hello,
> 
> I've recently noticed that when creating crashdumps the dmesg is not
> being saved. The error reported is this: "Missing the struct log size
> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
> been compiled with debugging info so the respective symbol should be
> exported but apparently it is not. Any ideas how to debug that?

That would be more helpful if you can attach the 1st kernel and kdump
kernel console log.

> 
> [Please CC me as I'm not subscribed to the mailing list.]
> 
> Regards,
> Nikolay
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Nikolay Borisov]

Hi,

On 08/19/2015 12:02 PM, Baoquan He wrote:
> On 08/19/15 at 09:21am, Nikolay Borisov wrote:
>> Hello,
>>
>> I've recently noticed that when creating crashdumps the dmesg is not
>> being saved. The error reported is this: "Missing the struct log size
>> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
>> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
>> been compiled with debugging info so the respective symbol should be
>> exported but apparently it is not. Any ideas how to debug that?
> 
> That would be more helpful if you can attach the 1st kernel and kdump
> kernel console log.

I just tested with 4.1.6 with the same result. I'm sending you a link to
the bzImage in question
http://georgi.unixsol.org/cruft/tmp/bzImage-4.1.6-clouder1 ( I assume
you meant the bzImage, if you need I can provide the vmlinux as well).
Unfortunately, I couldn't figure how to obtain the log of kdump in
textual format (and didn't want to send pictures). Here are the 3
relevant lines (from memory):

Saving vmcore-dmesg.txt
Missing the struct log size export
Error saving vmcore-dmesg.txt.

Yet, when I load the crashdump inside the 'crash' utility and invoke the
'log' command the dmesg log is there. Whereas the
vmcore-dmesg-incomplete.txt is empty.


> 
>>
>> [Please CC me as I'm not subscribed to the mailing list.]
>>
>> Regards,
>> Nikolay
>>
>> _______________________________________________
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Minfei Huang]

Hi. Nikolay.

Does kexec-tools work for original rhel kernel, like 3.10.0-xxx. Also do
you specify the config to build the kernel? It is appropriate if you can
paste the config related to the kexec.

Thanks
Minfei

On 08/19/15 at 09:21am, Nikolay Borisov wrote:
> Hello,
> 
> I've recently noticed that when creating crashdumps the dmesg is not
> being saved. The error reported is this: "Missing the struct log size
> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
> been compiled with debugging info so the respective symbol should be
> exported but apparently it is not. Any ideas how to debug that?
> 
> [Please CC me as I'm not subscribed to the mailing list.]
> 
> Regards,
> Nikolay
> 
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Nikolay Borisov]

On 08/19/2015 04:13 PM, Minfei Huang wrote:
> Hi. Nikolay.
Hi
> 
> Does kexec-tools work for original rhel kernel, like 3.10.0-xxx. Also do

Unfortunately I canno test with an official rhel kernel since we are not
using those. Also, the problem is not that kexec is not working at all,
but just that the dmesg log is not being saved in the vmcore-dmesg.txt
file. Here are the relevant portion of the my .config which deal with
debug info:

CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
# CONFIG_DEBUG_INFO_DWARF4 is not set

The failures is happening (according to my reading of the vmcore-dmesg.c
file) here:

+	if (!log_sz) {
+		fprintf(stderr, "Missing the struct log size export\n");
+		exit(64);
+	}

And log_size is being set here:

		str = "SIZE(printk_log)=";
		if (memcmp(str, pos, strlen(str)) == 0)
			log_sz = strtoull(pos + strlen(str), NULL, 10);

The pertinent question I guess is why this string cannot be found in the
resulting vmcore image. I just tried with kexec 2.0.10 and the result is
the same.

> you specify the config to build the kernel? It is appropriate if you can
> paste the config related to the kexec.
> 
> Thanks
> Minfei
> 
> On 08/19/15 at 09:21am, Nikolay Borisov wrote:
>> Hello,
>>
>> I've recently noticed that when creating crashdumps the dmesg is not
>> being saved. The error reported is this: "Missing the struct log size
>> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
>> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
>> been compiled with debugging info so the respective symbol should be
>> exported but apparently it is not. Any ideas how to debug that?
>>
>> [Please CC me as I'm not subscribed to the mailing list.]
>>
>> Regards,
>> Nikolay
>>
>> _______________________________________________
>> kexec mailing list
>> kexec@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Minfei Huang]

On 08/19/15 at 04:40pm, Nikolay Borisov wrote:
> On 08/19/2015 04:13 PM, Minfei Huang wrote:
> > Hi. Nikolay.
> > Does kexec-tools work for original rhel kernel, like 3.10.0-xxx. Also do
> 
> Unfortunately I canno test with an official rhel kernel since we are not
> using those. Also, the problem is not that kexec is not working at all,
> but just that the dmesg log is not being saved in the vmcore-dmesg.txt
> file. Here are the relevant portion of the my .config which deal with
> debug info:
> 
> CONFIG_DEBUG_INFO=y
> # CONFIG_DEBUG_INFO_REDUCED is not set
> # CONFIG_DEBUG_INFO_SPLIT is not set
> # CONFIG_DEBUG_INFO_DWARF4 is not set

I ran the kexec-tools-2.0.7-35 on kernel 4.2.0-rc5+ and kdump works
successfully to dump the vmcore. The kernel is built by the default config
(generated by make defconfig). Following is the relevant portions of the
config. 

 # cat .config | grep KEXEC
CONFIG_KEXEC=y
CONFIG_KEXEC_FILE=y
CONFIG_KEXEC_VERIFY_SIG=y
CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y
CONFIG_KEXEC_JUMP=y
 # cat .config | grep DEBUG_INFO
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
# CONFIG_DEBUG_INFO_DWARF4 is not set

Thanks
Minfei

> 
> The failures is happening (according to my reading of the vmcore-dmesg.c
> file) here:
> 
> +	if (!log_sz) {
> +		fprintf(stderr, "Missing the struct log size export\n");
> +		exit(64);
> +	}
> 
> And log_size is being set here:
> 
> 		str = "SIZE(printk_log)=";
> 		if (memcmp(str, pos, strlen(str)) == 0)
> 			log_sz = strtoull(pos + strlen(str), NULL, 10);
> 
> The pertinent question I guess is why this string cannot be found in the
> resulting vmcore image. I just tried with kexec 2.0.10 and the result is
> the same.

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Nikolay Borisov]

On 08/19/2015 05:08 PM, Minfei Huang wrote:
> On 08/19/15 at 04:40pm, Nikolay Borisov wrote:
>> On 08/19/2015 04:13 PM, Minfei Huang wrote:
>>> Hi. Nikolay.
>>> Does kexec-tools work for original rhel kernel, like 3.10.0-xxx. Also do
>>
>> Unfortunately I canno test with an official rhel kernel since we are not
>> using those. Also, the problem is not that kexec is not working at all,
>> but just that the dmesg log is not being saved in the vmcore-dmesg.txt
>> file. Here are the relevant portion of the my .config which deal with
>> debug info:
>>
>> CONFIG_DEBUG_INFO=y
>> # CONFIG_DEBUG_INFO_REDUCED is not set
>> # CONFIG_DEBUG_INFO_SPLIT is not set
>> # CONFIG_DEBUG_INFO_DWARF4 is not set
> 
> I ran the kexec-tools-2.0.7-35 on kernel 4.2.0-rc5+ and kdump works
> successfully to dump the vmcore. The kernel is built by the default config
> (generated by make defconfig). Following is the relevant portions of the
> config. 
> 
>  # cat .config | grep KEXEC
> CONFIG_KEXEC=y
> CONFIG_KEXEC_FILE=y
> CONFIG_KEXEC_VERIFY_SIG=y
> CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y
> CONFIG_KEXEC_JUMP=y
>  # cat .config | grep DEBUG_INFO
> CONFIG_DEBUG_INFO=y
> # CONFIG_DEBUG_INFO_REDUCED is not set
> # CONFIG_DEBUG_INFO_SPLIT is not set
> # CONFIG_DEBUG_INFO_DWARF4 is not set
> 
> Thanks
> Minfei

I only have CONFIG_KEXEC=y but the others are not necessary as far as I
can tell. I don't do any verification  so the VERIFY_SIG are disabled.
KEXEC_JUMP is also n, since I haven't enabled hibernation and KEXEC_FILE
is also not critical.  My debug info is the same as yours.


> 
>>
>> The failures is happening (according to my reading of the vmcore-dmesg.c
>> file) here:
>>
>> +	if (!log_sz) {
>> +		fprintf(stderr, "Missing the struct log size export\n");
>> +		exit(64);
>> +	}
>>
>> And log_size is being set here:
>>
>> 		str = "SIZE(printk_log)=";
>> 		if (memcmp(str, pos, strlen(str)) == 0)
>> 			log_sz = strtoull(pos + strlen(str), NULL, 10);
>>
>> The pertinent question I guess is why this string cannot be found in the
>> resulting vmcore image. I just tried with kexec 2.0.10 and the result is
>> the same.

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Pratyush Anand]

Hi Nikolay,

On 19/08/2015:02:10:21 PM, Nikolay Borisov wrote:
> Hi,
> 
> On 08/19/2015 12:02 PM, Baoquan He wrote:
> > On 08/19/15 at 09:21am, Nikolay Borisov wrote:
> >> Hello,
> >>
> >> I've recently noticed that when creating crashdumps the dmesg is not
> >> being saved. The error reported is this: "Missing the struct log size
> >> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
> >> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has

Are you sure if you have 2.0.9. I see 2.0.9 is dated 09 Feb 2015.

commit 8e762175d295aad08a1cc62d6394e8285f225a40
Author: Simon Horman <horms@verge.net.au>
Date:   Mon Feb 9 14:33:24 2015 +0900

    kexec-tools 2.0.9


> >> been compiled with debugging info so the respective symbol should be
> >> exported but apparently it is not. Any ideas how to debug that?

Can you please cross check that you have following patch in your kexec-tool. If
you have that patch, then its really strange that you see this issue.

commit c96b723e17eb4af689de494b636d8bded4b98c88
Author: Vivek Goyal <vgoyal@redhat.com>
Date:   Thu May 22 10:43:14 2014 -0400

    vmcore-dmesg: Handle struct log to struct printk_log renaming

~Pratyush    

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Dave Young]

On 08/21/15 at 10:19am, Pratyush Anand wrote:
> Hi Nikolay,
> 
> On 19/08/2015:02:10:21 PM, Nikolay Borisov wrote:
> > Hi,
> > 
> > On 08/19/2015 12:02 PM, Baoquan He wrote:
> > > On 08/19/15 at 09:21am, Nikolay Borisov wrote:
> > >> Hello,
> > >>
> > >> I've recently noticed that when creating crashdumps the dmesg is not
> > >> being saved. The error reported is this: "Missing the struct log size
> > >> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
> > >> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
> 
> Are you sure if you have 2.0.9. I see 2.0.9 is dated 09 Feb 2015.


I  believe the release date is not the real "release date" if one use
kexec --version

Because for kexec --version the date is created while building, it is
actually the build date instead of release date.

Thanks
Dave

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Nikolay Borisov]

On 08/24/2015 04:03 PM, Dave Anderson wrote:
> 
> 
> ----- Original Message -----
>>
>>
>> On 08/20/2015 04:04 PM, Dave Anderson wrote:
>>> The vmcoreinfo data strings were initially located in an ELF note in /proc/vmcore.
>>> When makedumpfile -c was run on /proc/vmcore, it copied those ELF notes into the
>>> compressed kdump header, and you have dumped them above.
>>>
>>> So it seems to be an issue with vmcore-dmesg.  If you change the core_collector
>>> variable to "cp" or "scp", it will copy /proc/vmcore unmodified to the target
>>> location.  Then you can run vmcore-dmesg on that file to debug it.
>>
>> This is very, very odd. Obtaining a raw vmcore with cp and then running
>> vmcore-dmesg does show the dmesg log. But at the same time, the
>> vmcore-dmesg.txt file is empty and its name is
>> vmcore-dmesg-incomplete.txt...
> 
> I'm not sure what you mean.  Similar to the saving of the vmcore file, the
> script directs the output of vmcore-dmesg into vmcore-dmesg-incomplete.txt.
> Then, only if vmcore-dmesg returns an exit status of 0, the script will 
> rename the file to vmcore-dmesg.txt.

vmcore-dmesg vmcore
<dmesg log is printed"
echo $?
0

so vmcore-dmesg succeeds but for some reason the dmesg is not being
redirrected to the txt file. Yes the vmcore-dmesg-incomplete.txt is
empty when the crash dump was invoked.

So to sum up my findings so far:

* When a crash dump is invoked the with the -c option of makedumpfile I
get an empty vmcore-dmesg-incomplete.txt. And the error string printed
on screen while taking the dump is: "struct log size not exported".
Looking at the code this can happen if the "printk_log" symbol cannot be
found. However, when I load the resulting vmcore image in crash and
invoke help -n I see that the correct symbol is there.

* When a raw coredump is taken (via cp as a collector) I still have an
empty vmcore-dmesg-incomplete.txt file, yet when I manually run
vmcore-dmesg on the resulting vmcore the dmesg is extracted and the
return code of the vmcore-dmesg script is 0 (success).

Having those facts in mind and what you said I do not see a logical
reason why dmesg is not being stored in the text file.

So either I'm missing something or the error message that is being
printed is very misleading, even though the code that involves is not
complicated it at all. Hm....

> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Dave Anderson]

----- Original Message -----
> 
> 
> On 08/20/2015 04:04 PM, Dave Anderson wrote:
> > The vmcoreinfo data strings were initially located in an ELF note in /proc/vmcore.
> > When makedumpfile -c was run on /proc/vmcore, it copied those ELF notes into the
> > compressed kdump header, and you have dumped them above.
> > 
> > So it seems to be an issue with vmcore-dmesg.  If you change the core_collector
> > variable to "cp" or "scp", it will copy /proc/vmcore unmodified to the target
> > location.  Then you can run vmcore-dmesg on that file to debug it.
> 
> This is very, very odd. Obtaining a raw vmcore with cp and then running
> vmcore-dmesg does show the dmesg log. But at the same time, the
> vmcore-dmesg.txt file is empty and its name is
> vmcore-dmesg-incomplete.txt...

I'm not sure what you mean.  Similar to the saving of the vmcore file, the
script directs the output of vmcore-dmesg into vmcore-dmesg-incomplete.txt.
Then, only if vmcore-dmesg returns an exit status of 0, the script will 
rename the file to vmcore-dmesg.txt.

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Nikolay Borisov]

On 08/20/2015 04:04 PM, Dave Anderson wrote:
> The vmcoreinfo data strings were initially located in an ELF note in /proc/vmcore.
> When makedumpfile -c was run on /proc/vmcore, it copied those ELF notes into the
> compressed kdump header, and you have dumped them above.
> 
> So it seems to be an issue with vmcore-dmesg.  If you change the core_collector
> variable to "cp" or "scp", it will copy /proc/vmcore unmodified to the target
> location.  Then you can run vmcore-dmesg on that file to debug it.

This is very, very odd. Obtaining a raw vmcore with cp and then running
vmcore-dmesg does show the dmesg log. But at the same time, the
vmcore-dmesg.txt file is empty and its name is
vmcore-dmesg-incomplete.txt...

> 
> Dave
>  
> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Dave Anderson]

----- Original Message -----
> 
> 
> On 08/19/2015 11:38 PM, Dave Anderson wrote:
> > 
> > 
> > ----- Original Message -----
> >>
> >> Hi,
> >>
> >> On 08/19/2015 12:02 PM, Baoquan He wrote:
> >>> On 08/19/15 at 09:21am, Nikolay Borisov wrote:
> >>>> Hello,
> >>>>
> >>>> I've recently noticed that when creating crashdumps the dmesg is not
> >>>> being saved. The error reported is this: "Missing the struct log size
> >>>> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
> >>>> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
> >>>> been compiled with debugging info so the respective symbol should be
> >>>> exported but apparently it is not. Any ideas how to debug that?
> >>>
> >>> That would be more helpful if you can attach the 1st kernel and kdump
> >>> kernel console log.
> >>
> >> I just tested with 4.1.6 with the same result. I'm sending you a link to
> >> the bzImage in question
> >> http://georgi.unixsol.org/cruft/tmp/bzImage-4.1.6-clouder1 ( I assume
> >> you meant the bzImage, if you need I can provide the vmlinux as well).
> >> Unfortunately, I couldn't figure how to obtain the log of kdump in
> >> textual format (and didn't want to send pictures). Here are the 3
> >> relevant lines (from memory):
> >>
> >> Saving vmcore-dmesg.txt
> >> Missing the struct log size export
> >> Error saving vmcore-dmesg.txt.
> >>
> >> Yet, when I load the crashdump inside the 'crash' utility and invoke the
> >> 'log' command the dmesg log is there. Whereas the vmcore-dmesg-incomplete.txt is empty.
> > 
> > FYI -- the crash utility doesn't use the vmcoreinfo data for its "log" command,
> > but you can dump the vmcore's header contents, including the vmcoreinfo strings,
> > with the "help -n" command.  For example:
> 
> Thanks for the tip, using that I can confirm that the actual symbols are
> in the resulting vmcoreinfo:
> 
> 		      SIZE(printk_log)=16
> 		      OFFSET(printk_log.ts_nsec)=0
>                       OFFSET(printk_log.len)=8
>                       OFFSET(printk_log.text_len)=10
>                       OFFSET(printk_log.dict_len)=12
> 
> So the required symbol is there (printk_log) but for some reason kexec
> cannot read it... I tried creating an uncompressed crash (removing the
> -c option from the makedumpfile command to no avail)..

The vmcoreinfo data strings were initially located in an ELF note in /proc/vmcore.
When makedumpfile -c was run on /proc/vmcore, it copied those ELF notes into the
compressed kdump header, and you have dumped them above.

So it seems to be an issue with vmcore-dmesg.  If you change the core_collector
variable to "cp" or "scp", it will copy /proc/vmcore unmodified to the target
location.  Then you can run vmcore-dmesg on that file to debug it.

Dave
 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: Dmesg not being dumped

[Nikolay Borisov]

On 08/19/2015 11:38 PM, Dave Anderson wrote:
> 
> 
> ----- Original Message -----
>>
>> Hi,
>>
>> On 08/19/2015 12:02 PM, Baoquan He wrote:
>>> On 08/19/15 at 09:21am, Nikolay Borisov wrote:
>>>> Hello,
>>>>
>>>> I've recently noticed that when creating crashdumps the dmesg is not
>>>> being saved. The error reported is this: "Missing the struct log size
>>>> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
>>>> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
>>>> been compiled with debugging info so the respective symbol should be
>>>> exported but apparently it is not. Any ideas how to debug that?
>>>
>>> That would be more helpful if you can attach the 1st kernel and kdump
>>> kernel console log.
>>
>> I just tested with 4.1.6 with the same result. I'm sending you a link to
>> the bzImage in question
>> http://georgi.unixsol.org/cruft/tmp/bzImage-4.1.6-clouder1 ( I assume
>> you meant the bzImage, if you need I can provide the vmlinux as well).
>> Unfortunately, I couldn't figure how to obtain the log of kdump in
>> textual format (and didn't want to send pictures). Here are the 3
>> relevant lines (from memory):
>>
>> Saving vmcore-dmesg.txt
>> Missing the struct log size export   
>> Error saving vmcore-dmesg.txt.
>>
>> Yet, when I load the crashdump inside the 'crash' utility and invoke the
>> 'log' command the dmesg log is there. Whereas the vmcore-dmesg-incomplete.txt is empty.
> 
> FYI -- the crash utility doesn't use the vmcoreinfo data for its "log" command, 
> but you can dump the vmcore's header contents, including the vmcoreinfo strings,
> with the "help -n" command.  For example:

Thanks for the tip, using that I can confirm that the actual symbols are
in the resulting vmcoreinfo:

		      SIZE(printk_log)=16
		      OFFSET(printk_log.ts_nsec)=0
                      OFFSET(printk_log.len)=8
                      OFFSET(printk_log.text_len)=10
                      OFFSET(printk_log.dict_len)=12

So the required symbol is there (printk_log) but for some reason kexec
cannot read it... I tried creating an uncompressed crash (removing the
-c option from the makedumpfile command to no avail)..


> 
> crash> help -n
> ... [ cut ] ...
>                       OSRELEASE=3.18.0-0.rc2.git2.1.slub1.fc21.x86_64
>                       PAGESIZE=4096
>                       SYMBOL(init_uts_ns)=ffffffff81e1b300
>                       SYMBOL(node_online_map)=ffffffff81fe8980
>                       SYMBOL(swapper_pg_dir)=ffffffff81e14000
>                       SYMBOL(_stext)=ffffffff810001c8
>                       SYMBOL(vmap_area_list)=ffffffff81efce50
>                       SYMBOL(mem_section)=ffffffff832eb180
>                       LENGTH(mem_section)=4096
>                       SIZE(mem_section)=32
>                       OFFSET(mem_section.section_mem_map)=0
>                       SIZE(page)=64
>                       SIZE(pglist_data)=156928
>                       SIZE(zone)=1984
>                       SIZE(free_area)=88
>                       SIZE(list_head)=16
>                       SIZE(nodemask_t)=128
>                       OFFSET(page.flags)=0
>                       OFFSET(page._count)=28
>                       OFFSET(page.mapping)=8
>                       OFFSET(page.lru)=32
>                       OFFSET(page._mapcount)=24
>                       OFFSET(page.private)=48
>                       OFFSET(pglist_data.node_zones)=0
>                       OFFSET(pglist_data.nr_zones)=156480
>                       OFFSET(pglist_data.node_start_pfn)=156560
>                       OFFSET(pglist_data.node_spanned_pages)=156576
>                       OFFSET(pglist_data.node_id)=156584
>                       OFFSET(zone.free_area)=392
>                       OFFSET(zone.vm_stat)=1664
>                       OFFSET(zone.spanned_pages)=120
>                       OFFSET(free_area.free_list)=0
>                       OFFSET(list_head.next)=0
>                       OFFSET(list_head.prev)=8
>                       OFFSET(vmap_area.va_start)=0
>                       OFFSET(vmap_area.list)=48
>                       LENGTH(zone.free_area)=11
>                       SYMBOL(log_buf)=ffffffff81e5f0c0
>                       SYMBOL(log_buf_len)=ffffffff81e5f0bc
>                       SYMBOL(log_first_idx)=ffffffff831bee88
>                       SYMBOL(log_next_idx)=ffffffff831bee78
>                       SIZE(printk_log)=16
>                       OFFSET(printk_log.ts_nsec)=0
>                       OFFSET(printk_log.len)=8
>                       OFFSET(printk_log.text_len)=10
>                       OFFSET(printk_log.dict_len)=12
>                       LENGTH(free_area.free_list)=5
>                       NUMBER(NR_FREE_PAGES)=0
>                       NUMBER(PG_lru)=5
>                       NUMBER(PG_private)=11
>                       NUMBER(PG_swapcache)=16
>                       NUMBER(PG_slab)=7
>                       NUMBER(PG_hwpoison)=23
>                       NUMBER(PG_head_mask)=16384
>                       NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128
>                       SYMBOL(free_huge_page)=ffffffff81239430
>                       SYMBOL(phys_base)=ffffffff81e1b010
>                       SYMBOL(init_level4_pgt)=ffffffff81e14000
>                       SYMBOL(node_data)=ffffffff81fe2d40
>                       LENGTH(node_data)=1024
>                       KERNELOFFSET=0
>                       CRASHTIME=141512763
> ...
> 
> Dave
> 

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Dmesg not being dumped

[Dave Anderson]

----- Original Message -----
> 
> Hi,
> 
> On 08/19/2015 12:02 PM, Baoquan He wrote:
> > On 08/19/15 at 09:21am, Nikolay Borisov wrote:
> >> Hello,
> >>
> >> I've recently noticed that when creating crashdumps the dmesg is not
> >> being saved. The error reported is this: "Missing the struct log size
> >> export". I've tested with both kernel 4.1.1 and 3.12.28. My version of
> >> kexec tools is : kexec-tools 2.0.9 released 04 June 2015. The kernel has
> >> been compiled with debugging info so the respective symbol should be
> >> exported but apparently it is not. Any ideas how to debug that?
> > 
> > That would be more helpful if you can attach the 1st kernel and kdump
> > kernel console log.
> 
> I just tested with 4.1.6 with the same result. I'm sending you a link to
> the bzImage in question
> http://georgi.unixsol.org/cruft/tmp/bzImage-4.1.6-clouder1 ( I assume
> you meant the bzImage, if you need I can provide the vmlinux as well).
> Unfortunately, I couldn't figure how to obtain the log of kdump in
> textual format (and didn't want to send pictures). Here are the 3
> relevant lines (from memory):
> 
> Saving vmcore-dmesg.txt
> Missing the struct log size export   
> Error saving vmcore-dmesg.txt.
> 
> Yet, when I load the crashdump inside the 'crash' utility and invoke the
> 'log' command the dmesg log is there. Whereas the vmcore-dmesg-incomplete.txt is empty.

FYI -- the crash utility doesn't use the vmcoreinfo data for its "log" command, 
but you can dump the vmcore's header contents, including the vmcoreinfo strings,
with the "help -n" command.  For example:

crash> help -n
... [ cut ] ...
                      OSRELEASE=3.18.0-0.rc2.git2.1.slub1.fc21.x86_64
                      PAGESIZE=4096
                      SYMBOL(init_uts_ns)=ffffffff81e1b300
                      SYMBOL(node_online_map)=ffffffff81fe8980
                      SYMBOL(swapper_pg_dir)=ffffffff81e14000
                      SYMBOL(_stext)=ffffffff810001c8
                      SYMBOL(vmap_area_list)=ffffffff81efce50
                      SYMBOL(mem_section)=ffffffff832eb180
                      LENGTH(mem_section)=4096
                      SIZE(mem_section)=32
                      OFFSET(mem_section.section_mem_map)=0
                      SIZE(page)=64
                      SIZE(pglist_data)=156928
                      SIZE(zone)=1984
                      SIZE(free_area)=88
                      SIZE(list_head)=16
                      SIZE(nodemask_t)=128
                      OFFSET(page.flags)=0
                      OFFSET(page._count)=28
                      OFFSET(page.mapping)=8
                      OFFSET(page.lru)=32
                      OFFSET(page._mapcount)=24
                      OFFSET(page.private)=48
                      OFFSET(pglist_data.node_zones)=0
                      OFFSET(pglist_data.nr_zones)=156480
                      OFFSET(pglist_data.node_start_pfn)=156560
                      OFFSET(pglist_data.node_spanned_pages)=156576
                      OFFSET(pglist_data.node_id)=156584
                      OFFSET(zone.free_area)=392
                      OFFSET(zone.vm_stat)=1664
                      OFFSET(zone.spanned_pages)=120
                      OFFSET(free_area.free_list)=0
                      OFFSET(list_head.next)=0
                      OFFSET(list_head.prev)=8
                      OFFSET(vmap_area.va_start)=0
                      OFFSET(vmap_area.list)=48
                      LENGTH(zone.free_area)=11
                      SYMBOL(log_buf)=ffffffff81e5f0c0
                      SYMBOL(log_buf_len)=ffffffff81e5f0bc
                      SYMBOL(log_first_idx)=ffffffff831bee88
                      SYMBOL(log_next_idx)=ffffffff831bee78
                      SIZE(printk_log)=16
                      OFFSET(printk_log.ts_nsec)=0
                      OFFSET(printk_log.len)=8
                      OFFSET(printk_log.text_len)=10
                      OFFSET(printk_log.dict_len)=12
                      LENGTH(free_area.free_list)=5
                      NUMBER(NR_FREE_PAGES)=0
                      NUMBER(PG_lru)=5
                      NUMBER(PG_private)=11
                      NUMBER(PG_swapcache)=16
                      NUMBER(PG_slab)=7
                      NUMBER(PG_hwpoison)=23
                      NUMBER(PG_head_mask)=16384
                      NUMBER(PAGE_BUDDY_MAPCOUNT_VALUE)=-128
                      SYMBOL(free_huge_page)=ffffffff81239430
                      SYMBOL(phys_base)=ffffffff81e1b010
                      SYMBOL(init_level4_pgt)=ffffffff81e14000
                      SYMBOL(node_data)=ffffffff81fe2d40
                      LENGTH(node_data)=1024
                      KERNELOFFSET=0
                      CRASHTIME=141512763
...

Dave


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec