[PATCH resend v2 1/3] makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public

[PATCH resend v2 1/3] makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public

[Wang Xiao]

Move the following two functions from internal function to external
function.

get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr)
get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr)

Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
---
 elf_info.c | 71 +++++++++++++++++++++++++++++++-------------------------------
 elf_info.h |  2 ++
 2 files changed, 37 insertions(+), 36 deletions(-)

diff --git a/elf_info.c b/elf_info.c
index 5be1990..24936d4 100644
--- a/elf_info.c
+++ b/elf_info.c
@@ -95,42 +95,6 @@ static unsigned long		size_xen_crash_info;
  * Internal functions.
  */
 static int
-get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr)
-{
-	off_t offset;
-
-	offset = sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) * index;
-
-	if (lseek(fd, offset, SEEK_SET) < 0) {
-		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	if (read(fd, phdr, sizeof(Elf64_Phdr)) != sizeof(Elf64_Phdr)) {
-		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	return TRUE;
-}
-
-static int
-get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr)
-{
-	off_t offset;
-
-	offset = sizeof(Elf32_Ehdr) + sizeof(Elf32_Phdr) * index;
-
-	if (lseek(fd, offset, SEEK_SET) < 0) {
-		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	if (read(fd, phdr, sizeof(Elf32_Phdr)) != sizeof(Elf32_Phdr)) {
-		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
-		return FALSE;
-	}
-	return TRUE;
-}
-
-static int
 check_elf_format(int fd, char *filename, int *phnum, unsigned int *num_load)
 {
 	int i;
@@ -446,6 +410,41 @@ int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len)
 /*
  * External functions.
  */
+int
+get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr)
+{
+	off_t offset;
+
+	offset = sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) * index;
+
+	if (lseek(fd, offset, SEEK_SET) < 0) {
+		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	if (read(fd, phdr, sizeof(Elf64_Phdr)) != sizeof(Elf64_Phdr)) {
+		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	return TRUE;
+}
+
+int
+get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr)
+{
+	off_t offset;
+
+	offset = sizeof(Elf32_Ehdr) + sizeof(Elf32_Phdr) * index;
+
+	if (lseek(fd, offset, SEEK_SET) < 0) {
+		ERRMSG("Can't seek %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	if (read(fd, phdr, sizeof(Elf32_Phdr)) != sizeof(Elf32_Phdr)) {
+		ERRMSG("Can't read %s. %s\n", filename, strerror(errno));
+		return FALSE;
+	}
+	return TRUE;
+}
 
 /*
  * Convert Physical Address to File Offset.
diff --git a/elf_info.h b/elf_info.h
index cbabf8a..e712253 100644
--- a/elf_info.h
+++ b/elf_info.h
@@ -27,6 +27,8 @@
 
 #define MAX_SIZE_NHDR	MAX(sizeof(Elf64_Nhdr), sizeof(Elf32_Nhdr))
 
+int get_elf64_phdr(int fd, char *filename, int index, Elf64_Phdr *phdr);
+int get_elf32_phdr(int fd, char *filename, int index, Elf32_Phdr *phdr);
 
 off_t paddr_to_offset(unsigned long long paddr);
 off_t paddr_to_offset2(unsigned long long paddr, off_t hint);
-- 
1.8.3.1


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH resend v2 2/3] makedumpfile: make the incomplete dumpfile generated by ENOSPC error analyzable

[Wang Xiao]

Since the incomplete dumpfile generated by ENOSPC error can't be anylyzed
by crash utility, but sometimes this file may contain important information
and the panic problem won't be reproduced, then we came up with an idea to
modify the exist data of the incomplete dumpfile to make it analyzable by
crash utility. And each of those dumpfiles has a flag to indicate that it
has been modified. As there are two formats of these dumpfiles, different
methods and flags are needed to deal with each of them,

elf:
Modify the value of the PT_LOAD program header to reflect the actual size
of the incomplete dumpfile. This method can't be used to modify the dumpfile
written in flattened mode. This format use the "e_flags" of "elf header"
to indicate that it has been modified.

The method of dealing with "kdump-compressed" format dumpfiles is implemented
in the next patch.

Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
---
 makedumpfile.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 175 insertions(+), 4 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index b4d43d8..0c84b35 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -24,6 +24,7 @@
 #include <ctype.h>
 #include <sys/time.h>
 #include <limits.h>
+#include <assert.h>
 
 struct symbol_table	symbol_table;
 struct size_table	size_table;
@@ -3768,6 +3769,155 @@ read_flat_data_header(struct makedumpfile_data_header *fdh)
 }
 
 int
+reserve_diskspace(int fd, off_t start_offset, off_t end_offset, char *file_name)
+{
+	size_t buf_size;
+	char *buf = NULL;
+
+	int ret = FALSE;
+
+	assert(start_offset < end_offset);
+	buf_size = end_offset - start_offset;
+
+	if ((buf = malloc(buf_size)) == NULL) {
+		ERRMSG("Can't allocate memory for the size of reserved diskspace. %s\n",
+		       strerror(errno));
+		return FALSE;
+	}
+
+	memset(buf, 0, buf_size);
+	if (!write_buffer(fd, start_offset, buf, buf_size, file_name))
+		goto out;
+
+	ret = TRUE;
+out:
+	if (buf != NULL) {
+		free(buf);
+	}
+
+	return ret;
+}
+
+#define DUMP_ELF_INCOMPLETE 0x1
+int
+check_and_modify_elf_headers(char *filename) {
+	int fd, i, phnum, ret = FALSE;
+	off_t file_end, offset, end_offset;
+	Elf64_Ehdr ehdr64;
+	Elf32_Ehdr ehdr32;
+
+	if ((fd = open(filename, O_RDWR)) < 0) {
+		ERRMSG("Can't open the dump file(%s). %s\n",
+		       filename, strerror(errno));
+		return FALSE;
+	}
+
+	/*
+	 * the is_elf64_memory() function still can be used.
+	 */
+	if (is_elf64_memory()) { /* ELF64 */
+		if (!get_elf64_ehdr(fd, filename, &ehdr64)) {
+			ERRMSG("Can't get ehdr64.\n");
+			goto out_close_file;
+		}
+		phnum = ehdr64.e_phnum;
+	} else { /* ELF32 */
+		if (!get_elf32_ehdr(fd, filename, &ehdr32)) {
+			ERRMSG("Can't get ehdr32.\n");
+			goto out_close_file;
+		}
+		phnum = ehdr32.e_phnum;
+	}
+
+	file_end = lseek(fd, 0, SEEK_END);
+	if (file_end < 0) {
+		ERRMSG("Can't detect the size of %s. %s\n",
+		       filename, strerror(errno));
+		goto out_close_file;
+	}
+
+	for (i = 0; i < phnum; i++) {
+		if (is_elf64_memory()) {
+			Elf64_Phdr phdr64;
+
+			if (!get_elf64_phdr(fd, filename, i, &phdr64)) {
+				ERRMSG("Can't find Phdr %d.\n", i);
+				goto out_close_file;
+			}
+
+			offset = sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) * i;
+			end_offset = phdr64.p_offset + phdr64.p_filesz;
+
+			/*
+			 * Check the program header and modify its value according
+			 * to the actual written size.
+			 */
+			if (file_end >= end_offset)
+				continue;
+			/*
+			 * This is the last PT_LOAD that still has some data, but
+			 * not complete.
+			 */
+			else if (file_end >= phdr64.p_offset && file_end < end_offset)
+				phdr64.p_filesz = file_end - phdr64.p_offset;
+			else if (file_end < phdr64.p_offset) {
+				phdr64.p_offset = 0;
+				phdr64.p_filesz = 0;
+			}
+
+			if (!write_buffer(fd, offset, &phdr64,
+			                  sizeof(Elf64_Phdr), filename))
+				goto out_close_file;
+		} else {
+			Elf32_Phdr phdr32;
+
+			if (!get_elf32_phdr(fd, filename, i, &phdr32)) {
+				ERRMSG("Can't find Phdr %d.\n", i);
+				goto out_close_file;
+			}
+
+			offset = sizeof(Elf32_Ehdr) + sizeof(Elf32_Phdr) * i;
+			end_offset = phdr32.p_offset + phdr32.p_filesz;
+
+			if (file_end >= end_offset)
+				continue;
+			else if (file_end >= phdr32.p_offset && file_end < end_offset)
+				phdr32.p_filesz = file_end - phdr32.p_offset;
+			else if (file_end < phdr32.p_offset) {
+				phdr32.p_offset = 0;
+				phdr32.p_filesz = 0;
+			}
+
+			if (!write_buffer(fd, offset, &phdr32,
+			                  sizeof(Elf32_Phdr), filename))
+				goto out_close_file;
+		}
+	}
+
+	/*
+	 * Set the incomplete flag to the e_flags of elf header.
+	 */
+	if (is_elf64_memory()) {
+		ehdr64.e_flags |= DUMP_ELF_INCOMPLETE;
+		if (!write_buffer(fd, 0, &ehdr64, sizeof(Elf64_Ehdr), filename))
+			goto out_close_file;
+	} else {
+		ehdr32.e_flags |= DUMP_ELF_INCOMPLETE;
+		if (!write_buffer(fd, 0, &ehdr32, sizeof(Elf32_Ehdr), filename))
+			goto out_close_file;
+	}
+
+	ret = TRUE;
+out_close_file:
+	if (close(fd) < 0) {
+		ERRMSG("Can't close the dump file(%s). %s\n",
+		       filename, strerror(errno));
+	}
+
+	return ret;
+}
+
+int
 rearrange_dumpdata(void)
 {
 	int read_size, tmp_read_size;
@@ -5498,6 +5648,13 @@ write_elf_header(struct cache_data *cd_header)
 	size_note          = note.p_filesz;
 
 	/*
+	 * Reserve a space to store the whole program headers.
+	 */
+	if (!reserve_diskspace(cd_header->fd, cd_header->offset,
+				offset_note_dumpfile, cd_header->file_name))
+		goto out;
+
+	/*
 	 * Modify the note size in PT_NOTE header to accomodate eraseinfo data.
 	 * Eraseinfo will be written later.
 	 */
@@ -8015,10 +8172,10 @@ writeout_dumpfile(void)
 			goto out;
 		if (info->flag_cyclic) {
 			if (!write_elf_pages_cyclic(&cd_header, &cd_page))
-				goto out;
+				goto write_cache_enospc;
 		} else {
 			if (!write_elf_pages(&cd_header, &cd_page))
-				goto out;
+				goto write_cache_enospc;
 		}
 		if (!write_elf_eraseinfo(&cd_header))
 			goto out;
@@ -8045,6 +8202,11 @@ writeout_dumpfile(void)
 	}
 
 	ret = TRUE;
+write_cache_enospc:
+	if ((ret == FALSE) && info->flag_nospace && !info->flag_flatten) {
+		if (!write_cache_bufsz(&cd_header))
+			ERRMSG("This dumpfile may lost some important data.\n");
+	}
 out:
 	free_cache_data(&cd_header);
 	free_cache_data(&cd_page);
@@ -8237,8 +8399,17 @@ retry:
 		 * to create a dumpfile with it again.
 		 */
 		num_retry++;
-		if ((info->dump_level = get_next_dump_level(num_retry)) < 0)
- 			return FALSE;
+		if ((info->dump_level = get_next_dump_level(num_retry)) < 0) {
+			if (!info->flag_flatten) {
+				if (info->flag_elf_dumpfile) {
+					if (check_and_modify_elf_headers(info->name_dumpfile))
+						MSG("This is an incomplete dumpfile,"
+						    " but might analyzable.\n");
+				}
+			}
+
+			return FALSE;
+		}
 		MSG("Retry to create a dumpfile by dump_level(%d).\n",
 		    info->dump_level);
 		if (!delete_dumpfile())
-- 
1.8.3.1


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

[PATCH resend v2 3/3] makedumpfile: implementation of dealing with kdump-compressed dumpfile with ENOSPC error

[Wang Xiao]

kdump-compressed:
Dump the bitmap before any page header and page data. This format use
"status" of "disk_dump_header" to indicate that it has been modified.

Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
---
 diskdump_mod.h |  2 ++
 makedumpfile.c | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 82 insertions(+), 5 deletions(-)

diff --git a/diskdump_mod.h b/diskdump_mod.h
index dd24eb2..275ed83 100644
--- a/diskdump_mod.h
+++ b/diskdump_mod.h
@@ -95,6 +95,8 @@ struct kdump_sub_header {
 #define DUMP_DH_COMPRESSED_LZO	0x2	/* paged is compressed with lzo */
 #define DUMP_DH_COMPRESSED_SNAPPY	0x4
 					/* paged is compressed with snappy */
+#define DUMP_DH_COMPRESSED_INCOMPLETE	0x8
+					/* indicate an incomplete dumpfile */
 
 /* descriptor of each page for vmcore */
 typedef struct page_desc {
diff --git a/makedumpfile.c b/makedumpfile.c
index 0c84b35..44da9f2 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -3918,6 +3918,71 @@ out_close_file:
 }
 
 int
+check_and_modify_kdump_headers(char *filename) {
+	int fd, ret = FALSE;
+	struct disk_dump_header dh;
+
+	if (!read_disk_dump_header(&dh, filename))
+		return FALSE;
+
+	if ((fd = open(filename, O_RDWR)) < 0) {
+		ERRMSG("Can't open the dump file(%s). %s\n",
+		       filename, strerror(errno));
+		return FALSE;
+	}
+
+	/*
+	 * Set the incomplete flag to the status of disk_dump_header.
+	 */
+	dh.status |= DUMP_DH_COMPRESSED_INCOMPLETE;
+
+	/*
+	 * It's safe to overwrite the disk_dump_header.
+	 */
+	if (!write_buffer(fd, 0, &dh, sizeof(struct disk_dump_header), filename))
+		goto out_close_file;
+
+	ret = TRUE;
+out_close_file:
+	if (close(fd) < 0) {
+		ERRMSG("Can't close the dump file(%s). %s\n",
+		       filename, strerror(errno));
+	}
+
+	return ret;
+}
+
+int
+check_and_modify_multiple_kdump_headers() {
+	int i, status, ret = TRUE;
+	pid_t pid;
+	pid_t array_pid[info->num_dumpfile];
+
+	for (i = 0; i < info->num_dumpfile; i++) {
+		if ((pid = fork()) < 0) {
+			return FALSE;
+
+		} else if (pid == 0) { /* Child */
+			if (!check_and_modify_kdump_headers(SPLITTING_DUMPFILE(i)))
+				exit(1);
+			exit(0);
+		}
+		array_pid[i] = pid;
+	}
+
+	for (i = 0; i < info->num_dumpfile; i++) {
+		waitpid(array_pid[i], &status, WUNTRACED);
+		if (!WIFEXITED(status) || WEXITSTATUS(status) == 1) {
+			ERRMSG("Check and modify the incomplete dumpfile(%s) failed.\n",
+			       SPLITTING_DUMPFILE(i));
+			ret = FALSE;
+		}
+	}
+
+	return ret;
+}
+
+int
 rearrange_dumpdata(void)
 {
 	int read_size, tmp_read_size;
@@ -7185,11 +7250,11 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
 		if (!exclude_unnecessary_pages_cyclic(&cycle))
 			return FALSE;
 
-		if (!write_kdump_pages_cyclic(cd_header, cd_page, &pd_zero,
-					&offset_data, &cycle))
+		if (!write_kdump_bitmap2_cyclic(&cycle))
 			return FALSE;
 
-		if (!write_kdump_bitmap2_cyclic(&cycle))
+		if (!write_kdump_pages_cyclic(cd_header, cd_page, &pd_zero,
+					&offset_data, &cycle))
 			return FALSE;
 	}
 
@@ -8189,12 +8254,12 @@ writeout_dumpfile(void)
 	} else {
 		if (!write_kdump_header())
 			goto out;
+		if (!write_kdump_bitmap())
+			goto out;
 		if (!write_kdump_pages(&cd_header, &cd_page))
 			goto out;
 		if (!write_kdump_eraseinfo(&cd_page))
 			goto out;
-		if (!write_kdump_bitmap())
-			goto out;
 	}
 	if (info->flag_flatten) {
 		if (!write_end_flat_header())
@@ -8405,6 +8470,16 @@ retry:
 					if (check_and_modify_elf_headers(info->name_dumpfile))
 						MSG("This is an incomplete dumpfile,"
 						    " but might analyzable.\n");
+				} else {
+					if (info->flag_split) {
+						if (check_and_modify_multiple_kdump_headers())
+							MSG("The splited dumpfiles are incomplete,"
+							    " but might analyzable.\n");
+					} else {
+						if (check_and_modify_kdump_headers(info->name_dumpfile))
+							MSG("This is an incomplete dumpfile,"
+							    " but might analyzable.\n");
+					}
 				}
 			}
 
-- 
1.8.3.1


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH resend v2 1/3] makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public

[Wang, Xiao/Wang Xiao]

Hello,

These patches have been rebased, please review, thanks.

-- 
Regards
Wang Xiao

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

RE: [PATCH resend v2 1/3] makedumpfile: make get_elf64_phdr()/get_elf32_phdr() public

[Atsushi Kumagai]

Hello Wang,

>Hello,
>
>These patches have been rebased, please review, thanks.

Thanks for your reposting, but I haven't gotten a chance to
review them yet.

Unluckily, I'll be on vacation from tomorrow till Oct 5, sorry
that I have to keep you waiting for a while.
At least, I also like the basic idea.


Thanks
Atsushi Kumagai

>--
>Regards
>Wang Xiao
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

RE: [PATCH resend v2 3/3] makedumpfile: implementation of dealing with kdump-compressed dumpfile with ENOSPC error

[Atsushi Kumagai]

>kdump-compressed:
>Dump the bitmap before any page header and page data. This format use
>"status" of "disk_dump_header" to indicate that it has been modified.
>
>Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
>---
> diskdump_mod.h |  2 ++
> makedumpfile.c | 85 ++++++++++++++++++++++++++++++++++++++++++++++++++++++----
> 2 files changed, 82 insertions(+), 5 deletions(-)
>

[snip]

>+int
>+check_and_modify_multiple_kdump_headers() {
>+	int i, status, ret = TRUE;
>+	pid_t pid;
>+	pid_t array_pid[info->num_dumpfile];
>+
>+	for (i = 0; i < info->num_dumpfile; i++) {
>+		if ((pid = fork()) < 0) {
>+			return FALSE;
>+
>+		} else if (pid == 0) { /* Child */
>+			if (!check_and_modify_kdump_headers(SPLITTING_DUMPFILE(i)))
>+				exit(1);

reassemble_kdump_header() adopts only the dh.status of SPLITTING_DUMPFILE(0),
we should fix it to take over incomplete flags of the other dump files.


Thanks
Atsushi Kumagai

>+			exit(0);
>+		}
>+		array_pid[i] = pid;
>+	}
>+
>+	for (i = 0; i < info->num_dumpfile; i++) {
>+		waitpid(array_pid[i], &status, WUNTRACED);
>+		if (!WIFEXITED(status) || WEXITSTATUS(status) == 1) {
>+			ERRMSG("Check and modify the incomplete dumpfile(%s) failed.\n",
>+			       SPLITTING_DUMPFILE(i));
>+			ret = FALSE;
>+		}
>+	}
>+
>+	return ret;
>+}
>+
>+int
> rearrange_dumpdata(void)
> {
> 	int read_size, tmp_read_size;
>@@ -7185,11 +7250,11 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d
> 		if (!exclude_unnecessary_pages_cyclic(&cycle))
> 			return FALSE;
>
>-		if (!write_kdump_pages_cyclic(cd_header, cd_page, &pd_zero,
>-					&offset_data, &cycle))
>+		if (!write_kdump_bitmap2_cyclic(&cycle))
> 			return FALSE;
>
>-		if (!write_kdump_bitmap2_cyclic(&cycle))
>+		if (!write_kdump_pages_cyclic(cd_header, cd_page, &pd_zero,
>+					&offset_data, &cycle))
> 			return FALSE;
> 	}
>
>@@ -8189,12 +8254,12 @@ writeout_dumpfile(void)
> 	} else {
> 		if (!write_kdump_header())
> 			goto out;
>+		if (!write_kdump_bitmap())
>+			goto out;
> 		if (!write_kdump_pages(&cd_header, &cd_page))
> 			goto out;
> 		if (!write_kdump_eraseinfo(&cd_page))
> 			goto out;
>-		if (!write_kdump_bitmap())
>-			goto out;
> 	}
> 	if (info->flag_flatten) {
> 		if (!write_end_flat_header())
>@@ -8405,6 +8470,16 @@ retry:
> 					if (check_and_modify_elf_headers(info->name_dumpfile))
> 						MSG("This is an incomplete dumpfile,"
> 						    " but might analyzable.\n");
>+				} else {
>+					if (info->flag_split) {
>+						if (check_and_modify_multiple_kdump_headers())
>+							MSG("The splited dumpfiles are incomplete,"
>+							    " but might analyzable.\n");
>+					} else {
>+						if (check_and_modify_kdump_headers(info->name_dumpfile))
>+							MSG("This is an incomplete dumpfile,"
>+							    " but might analyzable.\n");
>+					}
> 				}
> 			}
>
>--
>1.8.3.1
>
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

RE: [PATCH resend v2 2/3] makedumpfile: make the incomplete dumpfile generated by ENOSPC error analyzable

[Atsushi Kumagai]

>Since the incomplete dumpfile generated by ENOSPC error can't be anylyzed
>by crash utility, but sometimes this file may contain important information
>and the panic problem won't be reproduced, then we came up with an idea to
>modify the exist data of the incomplete dumpfile to make it analyzable by
>crash utility. And each of those dumpfiles has a flag to indicate that it
>has been modified. As there are two formats of these dumpfiles, different
>methods and flags are needed to deal with each of them,
>
>elf:
>Modify the value of the PT_LOAD program header to reflect the actual size
>of the incomplete dumpfile. This method can't be used to modify the dumpfile
>written in flattened mode. This format use the "e_flags" of "elf header"
>to indicate that it has been modified.

Is it necessary to change the PT_LOAD headers in makedumpfile ?
It sounds too much work for makedumpfile, I think it's better to
avoid such an irreversible and more than enough change in capturing
phase. Are there some reasons why you don't do the same thing on the
fly in crash ?

OTOH, setting the incomplete flag is certainly makedumpfile's task
since crash can't detect ENOSPC.


Thanks
Atsushi Kumagai

>The method of dealing with "kdump-compressed" format dumpfiles is implemented
>in the next patch.
>
>Signed-of-by: Wang Xiao <wangx.fnst@cn.fujitsu.com>
>---
> makedumpfile.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 175 insertions(+), 4 deletions(-)
>
>diff --git a/makedumpfile.c b/makedumpfile.c
>index b4d43d8..0c84b35 100644
>--- a/makedumpfile.c
>+++ b/makedumpfile.c
>@@ -24,6 +24,7 @@
> #include <ctype.h>
> #include <sys/time.h>
> #include <limits.h>
>+#include <assert.h>
>
> struct symbol_table	symbol_table;
> struct size_table	size_table;
>@@ -3768,6 +3769,155 @@ read_flat_data_header(struct makedumpfile_data_header *fdh)
> }
>
> int
>+reserve_diskspace(int fd, off_t start_offset, off_t end_offset, char *file_name)
>+{
>+	size_t buf_size;
>+	char *buf = NULL;
>+
>+	int ret = FALSE;
>+
>+	assert(start_offset < end_offset);
>+	buf_size = end_offset - start_offset;
>+
>+	if ((buf = malloc(buf_size)) == NULL) {
>+		ERRMSG("Can't allocate memory for the size of reserved diskspace. %s\n",
>+		       strerror(errno));
>+		return FALSE;
>+	}
>+
>+	memset(buf, 0, buf_size);
>+	if (!write_buffer(fd, start_offset, buf, buf_size, file_name))
>+		goto out;
>+
>+	ret = TRUE;
>+out:
>+	if (buf != NULL) {
>+		free(buf);
>+	}
>+
>+	return ret;
>+}
>+
>+#define DUMP_ELF_INCOMPLETE 0x1
>+int
>+check_and_modify_elf_headers(char *filename) {
>+	int fd, i, phnum, ret = FALSE;
>+	off_t file_end, offset, end_offset;
>+	Elf64_Ehdr ehdr64;
>+	Elf32_Ehdr ehdr32;
>+
>+	if ((fd = open(filename, O_RDWR)) < 0) {
>+		ERRMSG("Can't open the dump file(%s). %s\n",
>+		       filename, strerror(errno));
>+		return FALSE;
>+	}
>+
>+	/*
>+	 * the is_elf64_memory() function still can be used.
>+	 */
>+	if (is_elf64_memory()) { /* ELF64 */
>+		if (!get_elf64_ehdr(fd, filename, &ehdr64)) {
>+			ERRMSG("Can't get ehdr64.\n");
>+			goto out_close_file;
>+		}
>+		phnum = ehdr64.e_phnum;
>+	} else { /* ELF32 */
>+		if (!get_elf32_ehdr(fd, filename, &ehdr32)) {
>+			ERRMSG("Can't get ehdr32.\n");
>+			goto out_close_file;
>+		}
>+		phnum = ehdr32.e_phnum;
>+	}
>+
>+	file_end = lseek(fd, 0, SEEK_END);
>+	if (file_end < 0) {
>+		ERRMSG("Can't detect the size of %s. %s\n",
>+		       filename, strerror(errno));
>+		goto out_close_file;
>+	}
>+
>+	for (i = 0; i < phnum; i++) {
>+		if (is_elf64_memory()) {
>+			Elf64_Phdr phdr64;
>+
>+			if (!get_elf64_phdr(fd, filename, i, &phdr64)) {
>+				ERRMSG("Can't find Phdr %d.\n", i);
>+				goto out_close_file;
>+			}
>+
>+			offset = sizeof(Elf64_Ehdr) + sizeof(Elf64_Phdr) * i;
>+			end_offset = phdr64.p_offset + phdr64.p_filesz;
>+
>+			/*
>+			 * Check the program header and modify its value according
>+			 * to the actual written size.
>+			 */
>+			if (file_end >= end_offset)
>+				continue;
>+			/*
>+			 * This is the last PT_LOAD that still has some data, but
>+			 * not complete.
>+			 */
>+			else if (file_end >= phdr64.p_offset && file_end < end_offset)
>+				phdr64.p_filesz = file_end - phdr64.p_offset;
>+			else if (file_end < phdr64.p_offset) {
>+				phdr64.p_offset = 0;
>+				phdr64.p_filesz = 0;
>+			}
>+
>+			if (!write_buffer(fd, offset, &phdr64,
>+			                  sizeof(Elf64_Phdr), filename))
>+				goto out_close_file;
>+		} else {
>+			Elf32_Phdr phdr32;
>+
>+			if (!get_elf32_phdr(fd, filename, i, &phdr32)) {
>+				ERRMSG("Can't find Phdr %d.\n", i);
>+				goto out_close_file;
>+			}
>+
>+			offset = sizeof(Elf32_Ehdr) + sizeof(Elf32_Phdr) * i;
>+			end_offset = phdr32.p_offset + phdr32.p_filesz;
>+
>+			if (file_end >= end_offset)
>+				continue;
>+			else if (file_end >= phdr32.p_offset && file_end < end_offset)
>+				phdr32.p_filesz = file_end - phdr32.p_offset;
>+			else if (file_end < phdr32.p_offset) {
>+				phdr32.p_offset = 0;
>+				phdr32.p_filesz = 0;
>+			}
>+
>+			if (!write_buffer(fd, offset, &phdr32,
>+			                  sizeof(Elf32_Phdr), filename))
>+				goto out_close_file;
>+		}
>+	}
>+
>+	/*
>+	 * Set the incomplete flag to the e_flags of elf header.
>+	 */
>+	if (is_elf64_memory()) {
>+		ehdr64.e_flags |= DUMP_ELF_INCOMPLETE;
>+		if (!write_buffer(fd, 0, &ehdr64, sizeof(Elf64_Ehdr), filename))
>+			goto out_close_file;
>+	} else {
>+		ehdr32.e_flags |= DUMP_ELF_INCOMPLETE;
>+		if (!write_buffer(fd, 0, &ehdr32, sizeof(Elf32_Ehdr), filename))
>+			goto out_close_file;
>+	}
>+
>+	ret = TRUE;
>+out_close_file:
>+	if (close(fd) < 0) {
>+		ERRMSG("Can't close the dump file(%s). %s\n",
>+		       filename, strerror(errno));
>+	}
>+
>+	return ret;
>+}
>+
>+int
> rearrange_dumpdata(void)
> {
> 	int read_size, tmp_read_size;
>@@ -5498,6 +5648,13 @@ write_elf_header(struct cache_data *cd_header)
> 	size_note          = note.p_filesz;
>
> 	/*
>+	 * Reserve a space to store the whole program headers.
>+	 */
>+	if (!reserve_diskspace(cd_header->fd, cd_header->offset,
>+				offset_note_dumpfile, cd_header->file_name))
>+		goto out;
>+
>+	/*
> 	 * Modify the note size in PT_NOTE header to accomodate eraseinfo data.
> 	 * Eraseinfo will be written later.
> 	 */
>@@ -8015,10 +8172,10 @@ writeout_dumpfile(void)
> 			goto out;
> 		if (info->flag_cyclic) {
> 			if (!write_elf_pages_cyclic(&cd_header, &cd_page))
>-				goto out;
>+				goto write_cache_enospc;
> 		} else {
> 			if (!write_elf_pages(&cd_header, &cd_page))
>-				goto out;
>+				goto write_cache_enospc;
> 		}
> 		if (!write_elf_eraseinfo(&cd_header))
> 			goto out;
>@@ -8045,6 +8202,11 @@ writeout_dumpfile(void)
> 	}
>
> 	ret = TRUE;
>+write_cache_enospc:
>+	if ((ret == FALSE) && info->flag_nospace && !info->flag_flatten) {
>+		if (!write_cache_bufsz(&cd_header))
>+			ERRMSG("This dumpfile may lost some important data.\n");
>+	}
> out:
> 	free_cache_data(&cd_header);
> 	free_cache_data(&cd_page);
>@@ -8237,8 +8399,17 @@ retry:
> 		 * to create a dumpfile with it again.
> 		 */
> 		num_retry++;
>-		if ((info->dump_level = get_next_dump_level(num_retry)) < 0)
>- 			return FALSE;
>+		if ((info->dump_level = get_next_dump_level(num_retry)) < 0) {
>+			if (!info->flag_flatten) {
>+				if (info->flag_elf_dumpfile) {
>+					if (check_and_modify_elf_headers(info->name_dumpfile))
>+						MSG("This is an incomplete dumpfile,"
>+						    " but might analyzable.\n");
>+				}
>+			}
>+
>+			return FALSE;
>+		}
> 		MSG("Retry to create a dumpfile by dump_level(%d).\n",
> 		    info->dump_level);
> 		if (!delete_dumpfile())
>--
>1.8.3.1
>
>
>_______________________________________________
>kexec mailing list
>kexec@lists.infradead.org
>http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

Re: [PATCH resend v2 2/3] makedumpfile: make the incomplete dumpfile generated by ENOSPC error analyzable

["Zhou, Wenjian/周文剑"]

On 10/08/2014 01:39 PM, Atsushi Kumagai wrote:
>> Since the incomplete dumpfile generated by ENOSPC error can't be anylyzed
>> by crash utility, but sometimes this file may contain important information
>> and the panic problem won't be reproduced, then we came up with an idea to
>> modify the exist data of the incomplete dumpfile to make it analyzable by
>> crash utility. And each of those dumpfiles has a flag to indicate that it
>> has been modified. As there are two formats of these dumpfiles, different
>> methods and flags are needed to deal with each of them,
>>
>> elf:
>> Modify the value of the PT_LOAD program header to reflect the actual size
>> of the incomplete dumpfile. This method can't be used to modify the dumpfile
>> written in flattened mode. This format use the "e_flags" of "elf header"
>> to indicate that it has been modified.
>
> Is it necessary to change the PT_LOAD headers in makedumpfile ?
> It sounds too much work for makedumpfile, I think it's better to
> avoid such an irreversible and more than enough change in capturing
> phase. Are there some reasons why you don't do the same thing on the
> fly in crash ?
>
> OTOH, setting the incomplete flag is certainly makedumpfile's task
> since crash can't detect ENOSPC.
>
>
> Thanks
> Atsushi Kumagai

hello

Actually, it just change the attribute:FileSiz of the PT_LOAD header 
having incomplete segment. The attribute shows the size of the segment 
in elf file. Because of that, crash can read the file correctly. And we 
haven't found it have anything to do with the original information so 
far,the information before makedumpfile. So, modifying header won't 
change too much data and won't lose any important information.

However, though it can be done in crash, the process will be much more 
complicated. I think it's not worth to do this just for that useless data.

If you think it is really needed, I will do that later.

Thanks
Zhou Wenjian


_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

RE: [PATCH resend v2 2/3] makedumpfile: make the incomplete dumpfile generated by ENOSPC error analyzable

[Atsushi Kumagai]

>On 10/08/2014 01:39 PM, Atsushi Kumagai wrote:
>>> Since the incomplete dumpfile generated by ENOSPC error can't be anylyzed
>>> by crash utility, but sometimes this file may contain important information
>>> and the panic problem won't be reproduced, then we came up with an idea to
>>> modify the exist data of the incomplete dumpfile to make it analyzable by
>>> crash utility. And each of those dumpfiles has a flag to indicate that it
>>> has been modified. As there are two formats of these dumpfiles, different
>>> methods and flags are needed to deal with each of them,
>>>
>>> elf:
>>> Modify the value of the PT_LOAD program header to reflect the actual size
>>> of the incomplete dumpfile. This method can't be used to modify the dumpfile
>>> written in flattened mode. This format use the "e_flags" of "elf header"
>>> to indicate that it has been modified.
>>
>> Is it necessary to change the PT_LOAD headers in makedumpfile ?
>> It sounds too much work for makedumpfile, I think it's better to
>> avoid such an irreversible and more than enough change in capturing
>> phase. Are there some reasons why you don't do the same thing on the
>> fly in crash ?
>>
>> OTOH, setting the incomplete flag is certainly makedumpfile's task
>> since crash can't detect ENOSPC.
>>
>>
>> Thanks
>> Atsushi Kumagai
>
>hello
>
>Actually, it just change the attribute:FileSiz of the PT_LOAD header
>having incomplete segment. The attribute shows the size of the segment
>in elf file. Because of that, crash can read the file correctly. And we
>haven't found it have anything to do with the original information so
>far,the information before makedumpfile. So, modifying header won't
>change too much data and won't lose any important information.
>
>However, though it can be done in crash, the process will be much more
>complicated. I think it's not worth to do this just for that useless data.

Can't you do just the same thing in crash when the incomplete flag is found ?

>If you think it is really needed, I will do that later.

Ideally, it's best to keep original data as much as possible, but
I'll accept this patches if the implementation for crash has to
be too complicated.


Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec