* [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format
@ 2013-05-15 2:29 Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap Qiao Nuohan
` (8 more replies)
0 siblings, 9 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Hi, all
The last version is here:
http://lists.gnu.org/archive/html/qemu-devel/2013-05/msg00803.html
Command 'dump-guest-memory' was introduced to dump guest's memory. But the
vmcore's format is only elf32 or elf64. The message is here:
http://lists.gnu.org/archive/html/qemu-devel/2012-04/msg03379.html
For migration, 'dump-guest-memory' is supposed to support compression feature.
Because of the regression, the missing of compression feature, we post these
patches to make 'dump-guest-memory' be able to dump guest's in kdump-compressed
format. Then vmcore can be much smaller, and easily be delivered.
The kdump-compressed format is *linux specific* *linux standard* crash dump
format used in kdump framework. The kdump-compressed format is readable only
with the crash utility, and it can be smaller than the ELF format because of
the compression support.
Note, similar to 'dump-guest-memory':
1. The guest should be x86 or x86_64. The other arch is not supported now.
2. If the OS is in the second kernel, gdb may not work well, and crash can
work by specifying '--machdep phys_addr=xxx' in the command line. The
reason is that the second kernel will update the page table, and we can
not get the page table for the first kernel.
3. The cpu's state is stored in QEMU note.
4. The vmcore are able to be compressed with zlib, lzo or snappy. zlib is
available by default, and option '--enable-lzo' or '--enable-snappy'
should be used with configure to make lzo or snappy available.
Changelog:
Changes from v1 to v2:
1. Address Eric & Daniel's comment: fix manner of string copy.
2. Address Eric's comment: replace reinventing new constants by using the
ready-made ones accoring.
3. Address Andreas's comment: remove useless include.
Qiao Nuohan (9):
Add API to manipulate dump_bitmap
Add API to manipulate cache_data
Move includes and struct definition to dump.h
Add API to create header of vmcore
Add API to create data of dump bitmap
Add API to create page
Add API to free buf used by creating header, bitmap and page
Add API to write header, bitmap and page into vmcore
Make monitor command 'dump-guest-memory' dump in kdump-compressed
format
Makefile.target | 2 +-
cache_data.c | 121 +++++++
configure | 50 +++
dump.c | 877 ++++++++++++++++++++++++++++++++++++++++++++++---
dump_bitmap.c | 171 ++++++++++
hmp-commands.hx | 12 +-
hmp.c | 23 ++-
include/cache_data.h | 56 ++++
include/dump_bitmap.h | 60 ++++
include/sysemu/dump.h | 176 ++++++++++
qapi-schema.json | 23 ++-
qmp-commands.hx | 6 +-
12 files changed, 1527 insertions(+), 50 deletions(-)
create mode 100644 cache_data.c
create mode 100644 dump_bitmap.c
create mode 100644 include/cache_data.h
create mode 100644 include/dump_bitmap.h
^ permalink raw reply [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-17 8:20 ` Andreas Färber
2013-05-15 2:29 ` [Qemu-devel] [PATCH 2/9 v2] Add API to manipulate cache_data Qiao Nuohan
` (7 subsequent siblings)
8 siblings, 1 reply; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Struct dump_bitmap is associated with a tmp file, and the tmp file can be used
to save data of bitmap in kdump-compressed format temporarily.
The following patch will use these functions to get the data of bitmap and cache
them into tmp files.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
Makefile.target | 2 +-
dump_bitmap.c | 171 +++++++++++++++++++++++++++++++++++++++++++++++++
include/dump_bitmap.h | 60 +++++++++++++++++
3 files changed, 232 insertions(+), 1 deletions(-)
create mode 100644 dump_bitmap.c
create mode 100644 include/dump_bitmap.h
diff --git a/Makefile.target b/Makefile.target
index ce4391f..00d4f13 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -112,7 +112,7 @@ obj-$(CONFIG_FDT) += device_tree.o
obj-$(CONFIG_KVM) += kvm-all.o
obj-y += memory.o savevm.o cputlb.o
obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
-obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
+obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o
obj-$(CONFIG_NO_GET_MEMORY_MAPPING) += memory_mapping-stub.o
obj-$(CONFIG_NO_CORE_DUMP) += dump-stub.o
LIBS+=$(libs_softmmu)
diff --git a/dump_bitmap.c b/dump_bitmap.c
new file mode 100644
index 0000000..d2f6c9f
--- /dev/null
+++ b/dump_bitmap.c
@@ -0,0 +1,171 @@
+/*
+ * QEMU dump bitmap
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "dump_bitmap.h"
+
+int init_dump_bitmap(struct dump_bitmap *db, const char *filename)
+{
+ int fd;
+ char *tmpname;
+
+ /* init the tmp file */
+ tmpname = getenv("TMPDIR");
+ if (!tmpname) {
+ tmpname = (char *)P_tmpdir;
+ }
+
+ db->file_name = (char *)g_strdup_printf("%s/%s", tmpname, filename);
+
+ fd = mkstemp(db->file_name);
+ if (fd < 0) {
+ return -1;
+ }
+
+ db->fd = fd;
+ unlink(db->file_name);
+
+ /* num_block should be set to -1, for nothing is store in buf now */
+ db->num_block = -1;
+ memset(db->buf, 0, BUFSIZE_BITMAP);
+ /* the tmp file starts to write at the very beginning */
+ db->offset = 0;
+
+ return 0;
+}
+
+int clear_dump_bitmap(struct dump_bitmap *db, size_t len_db)
+{
+ int ret;
+ char buf[BUFSIZE_BITMAP];
+ off_t offset_bit;
+
+ /* use buf to clear the tmp file block by block */
+ memset(buf, 0, sizeof(buf));
+
+ ret = lseek(db->fd, db->offset, SEEK_SET);
+ if (ret < 0) {
+ return -1;
+ }
+
+ offset_bit = 0;
+ while (offset_bit < len_db) {
+ if (write(db->fd, buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+
+ offset_bit += BUFSIZE_BITMAP;
+ }
+
+ return 0;
+}
+
+int set_dump_bitmap(struct dump_bitmap *db, unsigned long long pfn, int val)
+{
+ int byte, bit;
+ off_t old_offset, new_offset;
+ old_offset = db->offset + BUFSIZE_BITMAP * db->num_block;
+ new_offset = db->offset + BUFSIZE_BITMAP * (pfn / PFN_BUFBITMAP);
+
+ /*
+ * if the block needed to be set is not same as the one cached in buf,
+ * write the block back to the tmp file, then cache new block to the buf
+ */
+ if (0 <= db->num_block && old_offset != new_offset) {
+ if (lseek(db->fd, old_offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (write(db->fd, db->buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+ }
+
+ if (old_offset != new_offset) {
+ if (lseek(db->fd, new_offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (read(db->fd, db->buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+
+ db->num_block = pfn / PFN_BUFBITMAP;
+ }
+
+ /* get the exact place of the bit in the buf, set it */
+ byte = (pfn % PFN_BUFBITMAP) >> 3;
+ bit = (pfn % PFN_BUFBITMAP) & 7;
+ if (val) {
+ db->buf[byte] |= 1<<bit;
+ } else {
+ db->buf[byte] &= ~(1<<bit);
+ }
+
+ return 0;
+}
+
+int sync_dump_bitmap(struct dump_bitmap *db)
+{
+ off_t offset;
+ offset = db->offset + BUFSIZE_BITMAP * db->num_block;
+
+ /* db has not been used yet */
+ if (db->num_block < 0) {
+ return 0;
+ }
+
+ if (lseek(db->fd, offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (write(db->fd, db->buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * check if the bit is set to 1
+ */
+static inline int is_on(char *bitmap, int i)
+{
+ return bitmap[i>>3] & (1 << (i & 7));
+}
+
+int is_bit_set(struct dump_bitmap *db, unsigned long long pfn)
+{
+ off_t offset;
+
+ /* cache the block pfn belongs to, then check the block */
+ if (pfn == 0 || db->num_block != pfn/PFN_BUFBITMAP) {
+ offset = db->offset + BUFSIZE_BITMAP * (pfn/PFN_BUFBITMAP);
+ lseek(db->fd, offset, SEEK_SET);
+ read(db->fd, db->buf, BUFSIZE_BITMAP);
+ db->num_block = pfn/PFN_BUFBITMAP;
+ }
+
+ return is_on(db->buf, pfn%PFN_BUFBITMAP);
+}
+
+void free_dump_bitmap(struct dump_bitmap *db)
+{
+ if (db) {
+ if (db->file_name) {
+ g_free(db->file_name);
+ }
+
+ g_free(db);
+ }
+}
diff --git a/include/dump_bitmap.h b/include/dump_bitmap.h
new file mode 100644
index 0000000..ee91543
--- /dev/null
+++ b/include/dump_bitmap.h
@@ -0,0 +1,60 @@
+/*
+ * QEMU dump bitmap
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef DUMP_BITMAP_H
+#define DUMP_BITMAP_H
+
+#define BUFSIZE_BITMAP (4096)
+#define PFN_BUFBITMAP (CHAR_BIT * BUFSIZE_BITMAP)
+
+struct dump_bitmap {
+ int fd; /* fd of the tmp file */
+ int num_block; /* number of blocks cached in buf */
+ char *file_name; /* name of the tmp file */
+ char buf[BUFSIZE_BITMAP]; /* used to cache blocks */
+ off_t offset; /* offset of the tmp file */
+};
+
+/*
+ * create a tmp file used to store dump bitmap, then init buf of dump_bitmap
+ */
+int init_dump_bitmap(struct dump_bitmap *db, const char *filename);
+
+/*
+ * clear the content of the tmp file with all bits set to 0
+ */
+int clear_dump_bitmap(struct dump_bitmap *db, size_t len_db);
+
+/*
+ * 'pfn' is the number of bit needed to be set, use buf to cache the block which
+ * 'pfn' belongs to, then set 'val' to the bit
+ */
+int set_dump_bitmap(struct dump_bitmap *db, unsigned long long pfn, int val);
+
+/*
+ * when buf is caching a block, sync_dump_bitmap is needed to write the cached
+ * block to the tmp file
+ */
+int sync_dump_bitmap(struct dump_bitmap *db);
+
+/*
+ * check whether 'pfn' is set
+ */
+int is_bit_set(struct dump_bitmap *db, unsigned long long pfn);
+
+/*
+ * free the space used by dump_bitmap
+ */
+void free_dump_bitmap(struct dump_bitmap *db);
+
+#endif
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 2/9 v2] Add API to manipulate cache_data
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 3/9 v2] Move includes and struct definition to dump.h Qiao Nuohan
` (6 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Struct dump_bitmap is associated with a tmp file, and the tmp file can be used
to save data of page desc and page data in kdump-compressed format temporarily.
The following patch will use these function to get the data of page desc and
page data and cache them in tmp files.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
Makefile.target | 2 +-
cache_data.c | 121 ++++++++++++++++++++++++++++++++++++++++++++++++++
include/cache_data.h | 56 +++++++++++++++++++++++
3 files changed, 178 insertions(+), 1 deletions(-)
create mode 100644 cache_data.c
create mode 100644 include/cache_data.h
diff --git a/Makefile.target b/Makefile.target
index 00d4f13..b579aff 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -112,7 +112,7 @@ obj-$(CONFIG_FDT) += device_tree.o
obj-$(CONFIG_KVM) += kvm-all.o
obj-y += memory.o savevm.o cputlb.o
obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
-obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o
+obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o cache_data.o
obj-$(CONFIG_NO_GET_MEMORY_MAPPING) += memory_mapping-stub.o
obj-$(CONFIG_NO_CORE_DUMP) += dump-stub.o
LIBS+=$(libs_softmmu)
diff --git a/cache_data.c b/cache_data.c
new file mode 100644
index 0000000..469ed94
--- /dev/null
+++ b/cache_data.c
@@ -0,0 +1,121 @@
+/*
+ * QEMU cache data
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "cache_data.h"
+
+int init_cache_data(struct cache_data *cd, const char *filename)
+{
+ int fd;
+ char *tmpname;
+
+ /* init the tmp file */
+ tmpname = getenv("TMPDIR");
+ if (!tmpname) {
+ tmpname = (char *)P_tmpdir;
+ }
+
+ cd->file_name = (char *)g_strdup_printf("%s/%s", tmpname, filename);
+
+ fd = mkstemp(cd->file_name);
+ if (fd < 0) {
+ return -1;
+ }
+
+ cd->fd = fd;
+ unlink(cd->file_name);
+
+ /* init buf */
+ cd->buf_size = BUFSIZE_CACHE_DATA;
+ cd->cache_size = 0;
+ cd->buf = g_malloc0(BUFSIZE_CACHE_DATA);
+
+ cd->offset = 0;
+
+ return 0;
+}
+
+int write_cache(struct cache_data *cd, void *buf, size_t size)
+{
+ /*
+ * check if the space is enough to cache data, if not write cached
+ * data to the tmp file
+ */
+ if (cd->cache_size + size > cd->buf_size) {
+ if (lseek(cd->fd, cd->offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (write(cd->fd, cd->buf, cd->cache_size) != cd->cache_size) {
+ return -1;
+ }
+
+ cd->offset += cd->cache_size;
+ cd->cache_size = 0;
+ }
+
+ memcpy(cd->buf + cd->cache_size, buf, size);
+ cd->cache_size += size;
+
+ return 0;
+}
+
+int sync_cache(struct cache_data *cd)
+{
+ /* no data is cached in cache_data */
+ if (cd->cache_size == 0) {
+ return 0;
+ }
+
+ if (lseek(cd->fd, cd->offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (write(cd->fd, cd->buf, cd->cache_size) != cd->cache_size) {
+ return -1;
+ }
+
+ cd->offset += cd->cache_size;
+
+ return 0;
+}
+
+int read_cache(struct cache_data *cd)
+{
+ if (lseek(cd->fd, cd->offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (read(cd->fd, cd->buf, cd->cache_size) != cd->cache_size) {
+ return -1;
+ }
+
+ cd->offset += cd->cache_size;
+
+ return 0;
+}
+
+void free_cache_data(struct cache_data *cd)
+{
+ if (cd) {
+ if (cd->file_name) {
+ g_free(cd->file_name);
+ }
+
+ if (cd->buf) {
+ g_free(cd->buf);
+ }
+
+ g_free(cd);
+ }
+}
diff --git a/include/cache_data.h b/include/cache_data.h
new file mode 100644
index 0000000..cbce60e
--- /dev/null
+++ b/include/cache_data.h
@@ -0,0 +1,56 @@
+/*
+ * QEMU cache data
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef CACHE_DATA_H
+#define CACHE_DATA_H
+
+#define BUFSIZE_CACHE_DATA (4096 * 4)
+
+struct cache_data {
+ int fd; /* fd of the tmp file used to store cache data */
+ char *file_name; /* name of the tmp file */
+ char *buf; /* used to cache data */
+ size_t buf_size; /* size of the buf */
+ size_t cache_size; /* size of cached data in buf */
+ off_t offset; /* offset of the tmp file */
+};
+
+/*
+ * create a tmp file used to store cache data, then init the buf
+ */
+int init_cache_data(struct cache_data *cd, const char *filename);
+
+/*
+ * write data to the tmp file, the data may first be cached in the buf of
+ * cache_data
+ */
+int write_cache(struct cache_data *cd, void *buf, size_t size);
+
+/*
+ * when cache_data is caching data in the buf, sync_cache is needed to write the
+ * data back to tmp file
+ */
+int sync_cache(struct cache_data *cd);
+
+/* read data from the tmp file to the buf of 'cd', the start place is set by
+ * cd->offset, and the size is set by cd->cache_size. cd->offset is changed
+ * automaticlly according to the size of data read this time.
+ */
+int read_cache(struct cache_data *cd);
+
+/*
+ * free the space used by cache_data
+ */
+void free_cache_data(struct cache_data *cd);
+
+#endif
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 3/9 v2] Move includes and struct definition to dump.h
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 2/9 v2] Add API to manipulate cache_data Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 4/9 v2] Add API to create header of vmcore Qiao Nuohan
` (5 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Move includes and definition of struct DumpState into include/sysemu/dump.h.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
dump.c | 29 -----------------------------
include/sysemu/dump.h | 30 ++++++++++++++++++++++++++++++
2 files changed, 30 insertions(+), 29 deletions(-)
diff --git a/dump.c b/dump.c
index c0d3da5..705c978 100644
--- a/dump.c
+++ b/dump.c
@@ -12,17 +12,7 @@
*/
#include "qemu-common.h"
-#include "elf.h"
-#include "cpu.h"
-#include "exec/cpu-all.h"
-#include "exec/hwaddr.h"
-#include "monitor/monitor.h"
-#include "sysemu/kvm.h"
#include "sysemu/dump.h"
-#include "sysemu/sysemu.h"
-#include "sysemu/memory_mapping.h"
-#include "qapi/error.h"
-#include "qmp-commands.h"
static uint16_t cpu_convert_to_target16(uint16_t val, int endian)
{
@@ -57,25 +47,6 @@ static uint64_t cpu_convert_to_target64(uint64_t val, int endian)
return val;
}
-typedef struct DumpState {
- ArchDumpInfo dump_info;
- MemoryMappingList list;
- uint16_t phdr_num;
- uint32_t sh_info;
- bool have_section;
- bool resume;
- size_t note_size;
- hwaddr memory_offset;
- int fd;
-
- RAMBlock *block;
- ram_addr_t start;
- bool has_filter;
- int64_t begin;
- int64_t length;
- Error **errp;
-} DumpState;
-
static int dump_cleanup(DumpState *s)
{
int ret = 0;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b8c770f..b41469a 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -14,12 +14,42 @@
#ifndef DUMP_H
#define DUMP_H
+#include "elf.h"
+#include "cpu.h"
+#include "exec/cpu-all.h"
+#include "exec/hwaddr.h"
+#include "monitor/monitor.h"
+#include "sysemu/kvm.h"
+#include "sysemu/sysemu.h"
+#include "sysemu/memory_mapping.h"
+#include "qapi/error.h"
+#include "qmp-commands.h"
+
typedef struct ArchDumpInfo {
int d_machine; /* Architecture */
int d_endian; /* ELFDATA2LSB or ELFDATA2MSB */
int d_class; /* ELFCLASS32 or ELFCLASS64 */
} ArchDumpInfo;
+typedef struct DumpState {
+ ArchDumpInfo dump_info;
+ MemoryMappingList list;
+ uint16_t phdr_num;
+ uint32_t sh_info;
+ bool have_section;
+ bool resume;
+ size_t note_size;
+ hwaddr memory_offset;
+ int fd;
+
+ RAMBlock *block;
+ ram_addr_t start;
+ bool has_filter;
+ int64_t begin;
+ int64_t length;
+ Error **errp;
+} DumpState;
+
int cpu_get_dump_info(ArchDumpInfo *info);
ssize_t cpu_get_note_size(int class, int machine, int nr_cpus);
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 4/9 v2] Add API to create header of vmcore
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
` (2 preceding siblings ...)
2013-05-15 2:29 ` [Qemu-devel] [PATCH 3/9 v2] Move includes and struct definition to dump.h Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 5/9 v2] Add API to create data of dump bitmap Qiao Nuohan
` (4 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Add API to create header, sub header of vmcore in kdump-compressed format.
The data is store in struct DumpState.
The following patch will use this function to gather data of header, then
write them into vmcore.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
dump.c | 107 +++++++++++++++++++++++++++++++++++++++++++++++++
include/sysemu/dump.h | 95 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 202 insertions(+), 0 deletions(-)
diff --git a/dump.c b/dump.c
index 705c978..bdebb33 100644
--- a/dump.c
+++ b/dump.c
@@ -672,6 +672,113 @@ static ram_addr_t get_start_block(DumpState *s)
return -1;
}
+static int create_header32(DumpState *s)
+{
+ struct disk_dump_header32 *dh;
+ struct kdump_sub_header32 *kh;
+
+ /* create common header, the version of kdump-compressed format is 5th */
+ dh = g_malloc0(sizeof(struct disk_dump_header32));
+
+ strncpy(dh->signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+ dh->header_version = 5;
+ dh->block_size = s->page_size;
+ dh->sub_hdr_size = sizeof(struct kdump_sub_header32) + s->note_size;
+ dh->sub_hdr_size = divideup(dh->sub_hdr_size, dh->block_size);
+ dh->max_mapnr = s->max_mapnr;
+ dh->nr_cpus = s->nr_cpus;
+ dh->bitmap_blocks = divideup(s->len_dump_bitmap, s->page_size);
+
+ memcpy(&(dh->utsname.machine), "i686", 4);
+
+ s->dh = dh;
+
+ /* create sub header */
+ kh = g_malloc0(sizeof(struct kdump_sub_header32));
+
+ kh->phys_base = PHYS_BASE;
+ kh->dump_level = DUMP_LEVEL;
+
+ kh->offset_note = DISKDUMP_HEADER_BLOCKS * dh->block_size +
+ sizeof(struct kdump_sub_header32);
+ kh->note_size = s->note_size;
+
+ s->kh = kh;
+
+ /* get gap between header and sub header */
+ s->offset_sub_header = DISKDUMP_HEADER_BLOCKS * dh->block_size -
+ sizeof(struct disk_dump_header32);
+
+ /* get gap between header and dump_bitmap */
+ s->offset_dump_bitmap = dh->sub_hdr_size * dh->block_size -
+ (sizeof(struct kdump_sub_header32) + s->note_size);
+
+ /* get offset of page desc */
+ s->offset_page = (DISKDUMP_HEADER_BLOCKS + dh->sub_hdr_size +
+ dh->bitmap_blocks) * dh->block_size;
+
+ return 0;
+}
+
+static int create_header64(DumpState *s)
+{
+ struct disk_dump_header64 *dh;
+ struct kdump_sub_header64 *kh;
+
+ /* create common header, the version of kdump-compressed format is 5th */
+ dh = g_malloc0(sizeof(struct disk_dump_header64));
+
+ strncpy(dh->signature, KDUMP_SIGNATURE, strlen(KDUMP_SIGNATURE));
+ dh->header_version = 5;
+ dh->block_size = s->page_size;
+ dh->sub_hdr_size = sizeof(struct kdump_sub_header64) + s->note_size;
+ dh->sub_hdr_size = divideup(dh->sub_hdr_size, dh->block_size);
+ dh->max_mapnr = s->max_mapnr;
+ dh->nr_cpus = s->nr_cpus;
+ dh->bitmap_blocks = divideup(s->len_dump_bitmap, s->page_size);
+
+ memcpy(&(dh->utsname.machine), "x86_64", 6);
+
+ s->dh = dh;
+
+ /* create sub header */
+ kh = g_malloc0(sizeof(struct kdump_sub_header64));
+
+ kh->phys_base = PHYS_BASE;
+ kh->dump_level = DUMP_LEVEL;
+
+ kh->offset_note = DISKDUMP_HEADER_BLOCKS * dh->block_size +
+ sizeof(struct kdump_sub_header64);
+ kh->note_size = s->note_size;
+
+ s->kh = kh;
+
+ /* get gap between header and sub header */
+ s->offset_sub_header = DISKDUMP_HEADER_BLOCKS * dh->block_size -
+ sizeof(struct disk_dump_header64);
+
+ /* get gap between header and dump_bitmap */
+ s->offset_dump_bitmap = dh->sub_hdr_size * dh->block_size -
+ (sizeof(struct kdump_sub_header64) + s->note_size);
+
+ /* get offset of page desc */
+ s->offset_page = (DISKDUMP_HEADER_BLOCKS + dh->sub_hdr_size +
+ dh->bitmap_blocks) * dh->block_size;
+
+ return 0;
+}
+
+/*
+ * gather data of header and sub header
+ */
+static int create_header(DumpState *s)
+{
+ if (s->dump_info.d_machine == EM_386)
+ return create_header32(s);
+ else
+ return create_header64(s);
+}
+
static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
int64_t begin, int64_t length, Error **errp)
{
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b41469a..b7846c9 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -25,12 +25,95 @@
#include "qapi/error.h"
#include "qmp-commands.h"
+#include <sys/utsname.h>
+
+#define KDUMP_SIGNATURE "KDUMP "
+#define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
+#define DISKDUMP_HEADER_BLOCKS (1)
+#define PHYS_BASE (0)
+#define DUMP_LEVEL (1)
+
+#define divideup(x, y) (((x) + ((y) - 1)) / (y))
+
typedef struct ArchDumpInfo {
int d_machine; /* Architecture */
int d_endian; /* ELFDATA2LSB or ELFDATA2MSB */
int d_class; /* ELFCLASS32 or ELFCLASS64 */
} ArchDumpInfo;
+struct new_utsname {
+ char sysname[65];
+ char nodename[65];
+ char release[65];
+ char version[65];
+ char machine[65];
+ char domainname[65];
+};
+
+struct disk_dump_header32 {
+ char signature[SIG_LEN]; /* = "KDUMP " */
+ int header_version; /* Dump header version */
+ struct new_utsname utsname; /* copy of system_utsname */
+ char timestamp[8]; /* Time stamp */
+ unsigned int status; /* Above flags */
+ int block_size; /* Size of a block in byte */
+ int sub_hdr_size; /* Size of arch dependent header in blocks */
+ unsigned int bitmap_blocks; /* Size of Memory bitmap in block */
+ unsigned int max_mapnr; /* = max_mapnr */
+ unsigned int total_ram_blocks; /* Number of blocks should be written */
+ unsigned int device_blocks; /* Number of total blocks in dump device */
+ unsigned int written_blocks; /* Number of written blocks */
+ unsigned int current_cpu; /* CPU# which handles dump */
+ int nr_cpus; /* Number of CPUs */
+ struct task_struct *tasks[0];
+};
+
+struct disk_dump_header64 {
+ char signature[SIG_LEN]; /* = "KDUMP " */
+ int header_version; /* Dump header version */
+ struct new_utsname utsname; /* copy of system_utsname */
+ char timestamp[20]; /* Time stamp */
+ unsigned int status; /* Above flags */
+ int block_size; /* Size of a block in byte */
+ int sub_hdr_size; /* Size of arch dependent header in blocks */
+ unsigned int bitmap_blocks; /* Size of Memory bitmap in block */
+ unsigned int max_mapnr; /* = max_mapnr */
+ unsigned int total_ram_blocks; /* Number of blocks should be written */
+ unsigned int device_blocks; /* Number of total blocks in dump device */
+ unsigned int written_blocks; /* Number of written blocks */
+ unsigned int current_cpu; /* CPU# which handles dump */
+ int nr_cpus; /* Number of CPUs */
+ struct task_struct *tasks[0];
+};
+
+struct kdump_sub_header32 {
+ uint32_t phys_base;
+ uint32_t dump_level; /* header_version 1 and later */
+ uint32_t split; /* header_version 2 and later */
+ uint32_t start_pfn; /* header_version 2 and later */
+ uint32_t end_pfn; /* header_version 2 and later */
+ uint32_t offset_vmcoreinfo; /* header_version 3 and later */
+ uint32_t size_vmcoreinfo; /* header_version 3 and later */
+ uint32_t offset_note; /* header_version 4 and later */
+ uint32_t note_size; /* header_version 4 and later */
+ uint32_t offset_eraseinfo; /* header_version 5 and later */
+ uint32_t size_eraseinfo; /* header_version 5 and later */
+};
+
+struct kdump_sub_header64 {
+ uint64_t phys_base;
+ uint32_t dump_level; /* header_version 1 and later */
+ uint32_t split; /* header_version 2 and later */
+ uint64_t start_pfn; /* header_version 2 and later */
+ uint64_t end_pfn; /* header_version 2 and later */
+ uint64_t offset_vmcoreinfo; /* header_version 3 and later */
+ uint64_t size_vmcoreinfo; /* header_version 3 and later */
+ uint64_t offset_note; /* header_version 4 and later */
+ uint64_t note_size; /* header_version 4 and later */
+ uint64_t offset_eraseinfo; /* header_version 5 and later */
+ uint64_t size_eraseinfo; /* header_version 5 and later */
+};
+
typedef struct DumpState {
ArchDumpInfo dump_info;
MemoryMappingList list;
@@ -48,6 +131,18 @@ typedef struct DumpState {
int64_t begin;
int64_t length;
Error **errp;
+
+ int page_size;
+ unsigned long long max_mapnr;
+ int nr_cpus;
+ void *dh;
+ void *kh;
+ off_t offset_sub_header;
+
+ off_t offset_dump_bitmap;
+ unsigned long len_dump_bitmap;
+
+ off_t offset_page;
} DumpState;
int cpu_get_dump_info(ArchDumpInfo *info);
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 5/9 v2] Add API to create data of dump bitmap
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
` (3 preceding siblings ...)
2013-05-15 2:29 ` [Qemu-devel] [PATCH 4/9 v2] Add API to create header of vmcore Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 6/9 v2] Add API to create page Qiao Nuohan
` (3 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Add API to get data of the 1st and 2nd dump bitmap and save them into tmp files.
The following patch will use these functions to gather data of dump bitmap,
then write them into vmcore.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
dump.c | 98 +++++++++++++++++++++++++++++++++++++++++++++++++
include/sysemu/dump.h | 10 +++++
2 files changed, 108 insertions(+), 0 deletions(-)
diff --git a/dump.c b/dump.c
index bdebb33..998d71e 100644
--- a/dump.c
+++ b/dump.c
@@ -779,6 +779,104 @@ static int create_header(DumpState *s)
return create_header64(s);
}
+/*
+ * create two tmpfile and save 1st and 2nd bitmap separately
+ */
+static int prepare_dump_bitmap(DumpState *s)
+{
+ int ret;
+ struct dump_bitmap *db1;
+ struct dump_bitmap *db2;
+
+ db1 = g_malloc0(sizeof(struct dump_bitmap));
+
+ db2 = g_malloc0(sizeof(struct dump_bitmap));
+
+ ret = init_dump_bitmap(db1, FILENAME_BITMAP1);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to init db1.");
+ return -1;
+ }
+ s->dump_bitmap1 = db1;
+
+ ret = init_dump_bitmap(db2, FILENAME_BITMAP2);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to init db1.");
+ return -1;
+ }
+ s->dump_bitmap2 = db2;
+
+ return 0;
+}
+
+static int create_dump_bitmap(DumpState *s)
+{
+ int ret;
+ unsigned long long num_dumpable;
+ MemoryMapping *memory_mapping;
+ unsigned long long pfn_start, pfn_end, pfn;
+
+ ret = prepare_dump_bitmap(s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to prepare dump_bitmap.\n");
+ return -1;
+ }
+
+ ret = clear_dump_bitmap(s->dump_bitmap1, s->len_dump_bitmap / 2);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to clear dump_bitmap1.\n");
+ return -1;
+ }
+
+ ret = clear_dump_bitmap(s->dump_bitmap2, s->len_dump_bitmap / 2);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to clear dump_bitmap2.\n");
+ return -1;
+ }
+
+ /* write dump bitmap to tmp files */
+ num_dumpable = 0;
+
+ QTAILQ_FOREACH(memory_mapping, &s->list.head, next) {
+ pfn_start = paddr_to_pfn(memory_mapping->phys_addr, s->page_shift);
+ pfn_end = paddr_to_pfn(memory_mapping->phys_addr +
+ memory_mapping->length, s->page_shift);
+
+ for (pfn = pfn_start; pfn < pfn_end; pfn++) {
+ ret = set_dump_bitmap(s->dump_bitmap1, pfn, 1);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to set dump_bitmap1.\n");
+ return -1;
+ }
+ /* set dump_bitmap2, same as dump_bitmap1 */
+ ret = set_dump_bitmap(s->dump_bitmap2, pfn, 1);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to set dump_bitmap2.\n");
+ return -1;
+ }
+ num_dumpable++;
+ }
+ }
+
+ /* write cached data to tmp files */
+ ret = sync_dump_bitmap(s->dump_bitmap1);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to sync dump_bitmap1.\n");
+ return -1;
+ }
+
+ ret = sync_dump_bitmap(s->dump_bitmap2);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to sync dump_bitmap2.\n");
+ return -1;
+ }
+
+ /* get the number of dumpable page */
+ s->num_dumpable = num_dumpable;
+
+ return 0;
+}
+
static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
int64_t begin, int64_t length, Error **errp)
{
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index b7846c9..93c35bc 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -24,6 +24,7 @@
#include "sysemu/memory_mapping.h"
#include "qapi/error.h"
#include "qmp-commands.h"
+#include "dump_bitmap.h"
#include <sys/utsname.h>
@@ -32,8 +33,13 @@
#define DISKDUMP_HEADER_BLOCKS (1)
#define PHYS_BASE (0)
#define DUMP_LEVEL (1)
+#define ARCH_PFN_OFFSET (0)
+#define FILENAME_BITMAP1 "kdump_bitmap1_XXXXXX"
+#define FILENAME_BITMAP2 "kdump_bitmap2_XXXXXX"
#define divideup(x, y) (((x) + ((y) - 1)) / (y))
+#define paddr_to_pfn(X, page_shift) \
+ (((unsigned long long)(X) >> (page_shift)) - ARCH_PFN_OFFSET)
typedef struct ArchDumpInfo {
int d_machine; /* Architecture */
@@ -133,14 +139,18 @@ typedef struct DumpState {
Error **errp;
int page_size;
+ int page_shift;
unsigned long long max_mapnr;
int nr_cpus;
void *dh;
void *kh;
+ unsigned long long num_dumpable;
off_t offset_sub_header;
off_t offset_dump_bitmap;
unsigned long len_dump_bitmap;
+ struct dump_bitmap *dump_bitmap1;
+ struct dump_bitmap *dump_bitmap2;
off_t offset_page;
} DumpState;
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 6/9 v2] Add API to create page
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
` (4 preceding siblings ...)
2013-05-15 2:29 ` [Qemu-devel] [PATCH 5/9 v2] Add API to create data of dump bitmap Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 7/9 v2] Add API to free buf used by creating header, bitmap and page Qiao Nuohan
` (2 subsequent siblings)
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Add API to get data of page desc and page data and save them into tmp files.
The following patch will use these functions to gather data of page desc and
page data, then write them into vmcore
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
dump.c | 259 +++++++++++++++++++++++++++++++++++++++++++++++++
include/sysemu/dump.h | 32 ++++++
2 files changed, 291 insertions(+), 0 deletions(-)
diff --git a/dump.c b/dump.c
index 998d71e..ebfb190 100644
--- a/dump.c
+++ b/dump.c
@@ -877,6 +877,265 @@ static int create_dump_bitmap(DumpState *s)
return 0;
}
+/*
+ * create two tmpfile and save page_desc and page_data
+ */
+static int prepare_pages(DumpState *s)
+{
+ int ret;
+ struct cache_data *page_desc;
+ struct cache_data *page_data;
+
+ page_desc = g_malloc0(sizeof(struct cache_data));
+
+ page_data = g_malloc0(sizeof(struct cache_data));
+
+ ret = init_cache_data(page_desc, FILENAME_PAGE_DESC);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to init page_desc.\n");
+ return -1;
+ }
+ s->page_desc = page_desc;
+
+ ret = init_cache_data(page_data, FILENAME_PAGE_DATA);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to init page_desc.\n");
+ return -1;
+ }
+ s->page_data = page_data;
+
+ return 0;
+}
+
+/*
+ * memory should be read page by page, or it may exceed the boundary and
+ * fail to read
+ */
+static int readmem(void *bufptr, ram_addr_t addr, size_t size, DumpState *s)
+{
+ RAMBlock *block;
+
+ block = s->block;
+
+ while (block) {
+ if ((addr >= block->offset) &&
+ (addr + size <= block->offset + block->length)) {
+ memcpy(bufptr, block->host + (addr - block->offset), size);
+ return 0;
+ } else {
+ block = QTAILQ_NEXT(block, next);
+ }
+ }
+
+ return -1;
+}
+
+/*
+ * check if the page is all 0
+ */
+static inline int is_zero_page(unsigned char *buf, long page_size)
+{
+ size_t i;
+
+ for (i = 0; i < page_size; i++) {
+ if (buf[i]) {
+ return 0;
+ }
+ }
+
+ return 1;
+}
+
+static int create_pages(DumpState *s)
+{
+ int ret;
+ unsigned long long pfn;
+ unsigned char buf[s->page_size];
+ unsigned char *buf_out = NULL;
+ unsigned long len_buf_out;
+ unsigned long size_out;
+ int zero_page;
+ struct page_desc pd, pd_zero;
+ off_t offset_desc, offset_data;
+ unsigned long len_buf_out_zlib, len_buf_out_lzo, len_buf_out_snappy;
+
+ ret = 0;
+
+ ret = prepare_pages(s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to prepare pages.\n");
+ goto out;
+ }
+
+ /* init buf_out */
+ len_buf_out_zlib = len_buf_out_lzo = len_buf_out_snappy = 0;
+
+ /* buf size for zlib */
+ len_buf_out_zlib = compressBound(s->page_size);
+
+ /* buf size for lzo */
+#ifdef CONFIG_LZO
+ if (s->flag_compress & DUMP_DH_COMPRESSED_LZO) {
+ if (lzo_init() != LZO_E_OK) {
+ ret = -1;
+ dump_error(s, "dump: failed to use lzo compression");
+ goto out;
+ }
+ }
+
+ lzo_bytep wrkmem;
+
+ wrkmem = g_malloc(LZO1X_1_MEM_COMPRESS);
+
+ len_buf_out_lzo = s->page_size + s->page_size / 16 + 64 + 3;
+#endif
+
+ /* buf size for snappy */
+#ifdef CONFIG_SNAPPY
+ len_buf_out_snappy = snappy_max_compressed_length(s->page_size);
+#endif
+
+ /* get the biggest that can store all kinds of compressed page */
+ len_buf_out = MAX(len_buf_out_zlib,
+ MAX(len_buf_out_lzo, len_buf_out_snappy));
+
+ buf_out = g_malloc(len_buf_out);
+
+ /* get offset of page_desc and page_data in dump file */
+ offset_desc = s->offset_page;
+ offset_data = offset_desc + sizeof(page_desc_t) * s->num_dumpable;
+
+ /*
+ * init zero page's page_desc and page_data, and all zero pages
+ * will use the same page_data
+ */
+ pd_zero.size = s->page_size;
+ pd_zero.flags = 0;
+ pd_zero.offset = offset_data;
+ pd_zero.page_flags = 0;
+ memset(buf, 0, pd_zero.size);
+ write_cache(s->page_data, buf, pd_zero.size);
+ offset_data += pd_zero.size;
+
+ for (pfn = 0; pfn < s->max_mapnr; pfn++) {
+ /* check whether the page is dumpable */
+ if (!is_bit_set(s->dump_bitmap2, pfn)) {
+ continue;
+ }
+
+ memset(buf, 0, s->page_size);
+ ret = readmem(buf, pfn_to_paddr(pfn, s->page_shift), s->page_size, s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to read memory ");
+ goto out;
+ }
+
+ /* check zero page */
+ zero_page = is_zero_page(buf, s->page_size);
+ if (zero_page) {
+ write_cache(s->page_desc, &pd_zero, sizeof(page_desc_t));
+ } else {
+ /*
+ * not zero page, then:
+ * 1. compress the page
+ * 2. write the compressed page into the cache of page_data
+ * 3. get page desc of the compressed page and write it into the
+ * cache of page_desc
+ */
+ size_out = len_buf_out;
+ if ((s->flag_compress & DUMP_DH_COMPRESSED_ZLIB) &&
+ (compress2(buf_out, &size_out, buf, s->page_size,
+ Z_BEST_SPEED) == Z_OK) && (size_out < s->page_size)) {
+ pd.flags = DUMP_DH_COMPRESSED_ZLIB;
+ pd.size = size_out;
+
+ ret = write_cache(s->page_data, buf_out, pd.size);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write page data(zlib).\n");
+ goto out;
+ }
+#ifdef CONFIG_LZO
+ } else if ((s->flag_compress & DUMP_DH_COMPRESSED_LZO) &&
+ (lzo1x_1_compress(buf, s->page_size, buf_out, &size_out,
+ wrkmem) == LZO_E_OK) && (size_out < s->page_size)) {
+ pd.flags = DUMP_DH_COMPRESSED_LZO;
+ pd.size = size_out;
+
+ ret = write_cache(s->page_data, buf_out, pd.size);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write page data(lzo).\n");
+ goto out;
+ }
+#endif
+#ifdef CONFIG_SNAPPY
+ } else if ((s->flag_compress & DUMP_DH_COMPRESSED_SNAPPY) &&
+ (snappy_compress((char *)buf, s->page_size, (char *)buf_out,
+ (size_t *)&size_out) == SNAPPY_OK) &&
+ (size_out < s->page_size)) {
+ pd.flags = DUMP_DH_COMPRESSED_SNAPPY;
+ pd.size = size_out;
+
+ ret = write_cache(s->page_data, buf_out, pd.size);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write page data(snappy).\n");
+ goto out;
+ }
+#endif
+ } else {
+ pd.flags = 0;
+ pd.size = s->page_size;
+
+ ret = write_cache(s->page_data, buf, pd.size);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write page data.\n");
+ goto out;
+ }
+ }
+
+ /* get and write page desc here */
+ pd.page_flags = 0;
+ pd.offset = offset_data;
+ offset_data += pd.size;
+
+ ret = write_cache(s->page_desc, &pd, sizeof(page_desc_t));
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write page desc.\n");
+ goto out;
+ }
+ }
+ }
+
+ ret = sync_cache(s->page_desc);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to sync cache for page_desc.\n");
+ goto out;
+ }
+ ret = sync_cache(s->page_data);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to sync cache for page_data.\n");
+ goto out;
+ }
+
+ /* get size of page_desc and page_data, then reset their offset */
+ s->page_desc_size = s->page_desc->offset;
+ s->page_data_size = s->page_data->offset;
+ s->page_desc->offset = 0;
+ s->page_data->offset = 0;
+
+out:
+#ifdef CONFIG_LZO
+ if (wrkmem) {
+ g_free(wrkmem);
+ }
+#endif
+
+ if (buf_out) {
+ g_free(buf_out);
+ }
+
+ return ret;
+}
+
static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
int64_t begin, int64_t length, Error **errp)
{
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 93c35bc..7cb74d6 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -25,8 +25,23 @@
#include "qapi/error.h"
#include "qmp-commands.h"
#include "dump_bitmap.h"
+#include "cache_data.h"
#include <sys/utsname.h>
+#include <zlib.h>
+#ifdef CONFIG_LZO
+#include <lzo/lzo1x.h>
+#endif
+#ifdef CONFIG_SNAPPY
+#include <snappy-c.h>
+#endif
+
+/*
+ * flag used in page desc of kdump-compressed format
+ */
+#define DUMP_DH_COMPRESSED_ZLIB (0x1)
+#define DUMP_DH_COMPRESSED_LZO (0x2)
+#define DUMP_DH_COMPRESSED_SNAPPY (0x4)
#define KDUMP_SIGNATURE "KDUMP "
#define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
@@ -36,10 +51,14 @@
#define ARCH_PFN_OFFSET (0)
#define FILENAME_BITMAP1 "kdump_bitmap1_XXXXXX"
#define FILENAME_BITMAP2 "kdump_bitmap2_XXXXXX"
+#define FILENAME_PAGE_DESC "kdump_page_desc_XXXXXX"
+#define FILENAME_PAGE_DATA "kdump_page_data_XXXXXX"
#define divideup(x, y) (((x) + ((y) - 1)) / (y))
#define paddr_to_pfn(X, page_shift) \
(((unsigned long long)(X) >> (page_shift)) - ARCH_PFN_OFFSET)
+#define pfn_to_paddr(X, page_shift) \
+ (((unsigned long long)(X) + ARCH_PFN_OFFSET) << (page_shift))
typedef struct ArchDumpInfo {
int d_machine; /* Architecture */
@@ -120,6 +139,14 @@ struct kdump_sub_header64 {
uint64_t size_eraseinfo; /* header_version 5 and later */
};
+/* descriptor of each page for vmcore */
+typedef struct page_desc {
+ off_t offset; /* the offset of the page data*/
+ unsigned int size; /* the size of this dump page */
+ unsigned int flags; /* flags */
+ unsigned long long page_flags; /* page flags */
+} page_desc_t;
+
typedef struct DumpState {
ArchDumpInfo dump_info;
MemoryMappingList list;
@@ -146,6 +173,7 @@ typedef struct DumpState {
void *kh;
unsigned long long num_dumpable;
off_t offset_sub_header;
+ int flag_compress;
off_t offset_dump_bitmap;
unsigned long len_dump_bitmap;
@@ -153,6 +181,10 @@ typedef struct DumpState {
struct dump_bitmap *dump_bitmap2;
off_t offset_page;
+ unsigned long long page_desc_size;
+ unsigned long long page_data_size;
+ struct cache_data *page_desc;
+ struct cache_data *page_data;
} DumpState;
int cpu_get_dump_info(ArchDumpInfo *info);
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 7/9 v2] Add API to free buf used by creating header, bitmap and page
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
` (5 preceding siblings ...)
2013-05-15 2:29 ` [Qemu-devel] [PATCH 6/9 v2] Add API to create page Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 8/9 v2] Add API to write header, bitmap and page into vmcore Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
When calling create_header, create_dump_bitmap and create_pages, some memory spaces
are allocated. The following patch will use this function to free these spaces.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
dump.c | 19 +++++++++++++++++++
1 files changed, 19 insertions(+), 0 deletions(-)
diff --git a/dump.c b/dump.c
index ebfb190..75ef032 100644
--- a/dump.c
+++ b/dump.c
@@ -1252,6 +1252,25 @@ cleanup:
return -1;
}
+static void clean_state(DumpState *s)
+{
+ if (s->dh) {
+ g_free(s->dh);
+ }
+
+ if (s->kh) {
+ g_free(s->kh);
+ }
+
+ free_dump_bitmap(s->dump_bitmap1);
+
+ free_dump_bitmap(s->dump_bitmap2);
+
+ free_cache_data(s->page_desc);
+
+ free_cache_data(s->page_data);
+}
+
void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
int64_t begin, bool has_length, int64_t length,
Error **errp)
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 8/9 v2] Add API to write header, bitmap and page into vmcore
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
` (6 preceding siblings ...)
2013-05-15 2:29 ` [Qemu-devel] [PATCH 7/9 v2] Add API to free buf used by creating header, bitmap and page Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
8 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
The following patch will use these functions to write cached data into vmcore.
Header is cached in DumpState, and bitmap and page are cached in tmp files.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
dump.c | 259 +++++++++++++++++++++++++++++++++++++++++++++++++
include/sysemu/dump.h | 1 +
2 files changed, 260 insertions(+), 0 deletions(-)
diff --git a/dump.c b/dump.c
index 75ef032..c04e2ae 100644
--- a/dump.c
+++ b/dump.c
@@ -67,6 +67,36 @@ static void dump_error(DumpState *s, const char *reason)
dump_cleanup(s);
}
+/*
+ * write 'size' of 'c' to dump file
+ */
+static int fill_space(size_t size, int c, void *opaque)
+{
+ DumpState *s = opaque;
+ char tmpbuf[TMP_BUF_SIZE];
+ size_t fill_size;
+ size_t written_size;
+
+ memset(&tmpbuf, c, TMP_BUF_SIZE);
+
+ while (size) {
+ if (size > TMP_BUF_SIZE) {
+ fill_size = TMP_BUF_SIZE;
+ } else {
+ fill_size = size;
+ }
+
+ size -= fill_size;
+
+ written_size = qemu_write_full(s->fd, tmpbuf, fill_size);
+ if (written_size != fill_size) {
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
static int fd_write_vmcore(void *buf, size_t size, void *opaque)
{
DumpState *s = opaque;
@@ -672,6 +702,235 @@ static ram_addr_t get_start_block(DumpState *s)
return -1;
}
+static int write_dump_header(DumpState *s)
+{
+ int ret;
+ void *dh;
+ void *kh;
+
+ /* write common header */
+ dh = s->dh;
+
+ if (s->dump_info.d_machine == EM_386) {
+ ret = fd_write_vmcore(dh, sizeof(struct disk_dump_header32), s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write disk dump header.\n");
+ return -1;
+ }
+ } else {
+ ret = fd_write_vmcore(dh, sizeof(struct disk_dump_header64), s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write disk dump header.\n");
+ return -1;
+ }
+ }
+
+ /* fill gap between command header and sub header */
+ ret = fill_space(s->offset_sub_header, 0, s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to fill the space between header and\
+ sub header.\n");
+ return -1;
+ }
+
+ /* write sub header */
+ kh = s->kh;
+
+ if (s->dump_info.d_machine == EM_386) {
+ ret = fd_write_vmcore(kh, sizeof(struct kdump_sub_header32), s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write kdump sub header.\n");
+ return -1;
+ }
+ } else {
+ ret = fd_write_vmcore(kh, sizeof(struct kdump_sub_header64), s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to write kdump sub header.\n");
+ return -1;
+ }
+ }
+
+ /* write note */
+ if (s->dump_info.d_class == ELFCLASS64) {
+ if (write_elf64_notes(s) < 0) {
+ return -1;
+ }
+ } else {
+ if (write_elf32_notes(s) < 0) {
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
+static int write_dump_bitmap(DumpState *s)
+{
+ struct cache_data bm;
+ long buf_size;
+ int ret;
+ int no_bitmap;
+
+ /* fill gap between header and dump_bitmap */
+ ret = fill_space(s->offset_dump_bitmap, 0, s);
+ if (ret < 0) {
+ dump_error(s, "dump: failed to fill the space between header and\
+ dump_bitmap.\n");
+ goto out;
+ }
+
+ bm.buf = g_malloc0(TMP_BUF_SIZE);
+
+ /* write dump_bitmap1 */
+ bm.fd = s->dump_bitmap1->fd;
+ no_bitmap = 1;
+
+again:
+ buf_size = s->len_dump_bitmap / 2;
+ bm.offset = 0;
+
+ while (buf_size > 0) {
+ if (buf_size >= TMP_BUF_SIZE) {
+ bm.cache_size = TMP_BUF_SIZE;
+ } else {
+ bm.cache_size = buf_size;
+ }
+
+ ret = read_cache(&bm);
+ if (ret < 0) {
+ goto out;
+ }
+
+ ret = fd_write_vmcore(bm.buf, bm.cache_size, s);
+ if (ret < 0) {
+ goto out;
+ }
+
+ buf_size -= bm.cache_size;
+ }
+
+ /* switch to dump_bitmap2 */
+ if (no_bitmap == 1) {
+ no_bitmap = 2;
+ bm.fd = s->dump_bitmap2->fd;
+ goto again;
+ }
+
+ return 0;
+
+out:
+ if (bm.buf) {
+ g_free(bm.buf);
+ }
+
+ return -1;
+}
+
+static int write_dump_pages(DumpState *s)
+{
+ struct cache_data page;
+ unsigned long long total_size;
+ int is_page_desc;
+ int ret;
+
+ page.buf = g_malloc0(TMP_BUF_SIZE);
+
+ /* write page_desc */
+ is_page_desc = 1;
+ total_size = s->page_desc_size;
+ page.fd = s->page_desc->fd;
+ page.offset = s->page_desc->offset;
+
+again:
+ while (total_size > 0) {
+ if (total_size > TMP_BUF_SIZE) {
+ page.cache_size = TMP_BUF_SIZE;
+ } else {
+ page.cache_size = total_size;
+ }
+
+ ret = read_cache(&page);
+ if (ret < 0) {
+ goto out;
+ }
+
+ ret = fd_write_vmcore(page.buf, page.cache_size, s);
+ if (ret < 0) {
+ goto out;
+ }
+
+ total_size -= page.cache_size;
+ }
+
+ /* switch to page_data */
+ if (is_page_desc) {
+ is_page_desc = 0;
+ total_size = s->page_data_size;
+ page.fd = s->page_data->fd;
+ page.offset = s->page_data->offset;
+ goto again;
+ }
+
+ return 0;
+
+out:
+ if (page.buf) {
+ g_free(page.buf);
+ }
+
+ return -1;
+}
+
+static int create_kdump_vmcore(DumpState *s)
+{
+ int ret;
+
+ /*
+ * the kdump-compressed format is:
+ * File offset
+ * +------------------------------------------+ 0x0
+ * | main header (struct disk_dump_header) |
+ * |------------------------------------------+ block 1
+ * | sub header (struct kdump_sub_header) |
+ * |------------------------------------------+ block 2
+ * | 1st-dump_bitmap |
+ * |------------------------------------------+ block 2 + X blocks
+ * | 2nd-dump_bitmap | (aligned by block)
+ * |------------------------------------------+ block 2 + 2 * X blocks
+ * | page desc for pfn 0 (struct page_desc) | (aligned by block)
+ * | page desc for pfn 1 (struct page_desc) |
+ * | : |
+ * | page desc for pfn Z (struct page_desc) |
+ * |------------------------------------------| (not aligned by block)
+ * | page data (pfn 0) |
+ * | page data (pfn 1) |
+ * | : |
+ * | page data (pfn Z) |
+ * +------------------------------------------+ offset_eraseinfo
+ * | : |
+ * +------------------------------------------+
+ */
+
+ ret = write_dump_header(s);
+ if (ret < 0) {
+ return -1;
+ }
+
+ ret = write_dump_bitmap(s);
+ if (ret < 0) {
+ return -1;
+ }
+
+ ret = write_dump_pages(s);
+ if (ret < 0) {
+ return -1;
+ }
+
+ dump_completed(s);
+
+ return 0;
+}
+
static int create_header32(DumpState *s)
{
struct disk_dump_header32 *dh;
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 7cb74d6..000455f 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -48,6 +48,7 @@
#define DISKDUMP_HEADER_BLOCKS (1)
#define PHYS_BASE (0)
#define DUMP_LEVEL (1)
+#define TMP_BUF_SIZE (1024)
#define ARCH_PFN_OFFSET (0)
#define FILENAME_BITMAP1 "kdump_bitmap1_XXXXXX"
#define FILENAME_BITMAP2 "kdump_bitmap2_XXXXXX"
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
` (7 preceding siblings ...)
2013-05-15 2:29 ` [Qemu-devel] [PATCH 8/9 v2] Add API to write header, bitmap and page into vmcore Qiao Nuohan
@ 2013-05-15 2:29 ` Qiao Nuohan
2013-05-15 2:55 ` Eric Blake
8 siblings, 1 reply; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 2:29 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, kumagai-atsushi, anderson, afaerber
Make monitor command 'dump-guest-memory' dump in kdump-compressed format.
The command's usage:
dump [-p] protocol [begin] [length] [format]
'format' is used to specified the format of vmcore and can be:
1. 'elf': ELF format, without compression
2. 'zlib': kdump-compressed format, with zlib-compressed
3. 'lzo': kdump-compressed format, with lzo-compressed
4. 'snappy': kdump-compressed format, with snappy-compressed
And without 'format' being set, vmcore will be in ELF format.
Note:
1. The kdump-compressed format is readable only with the crash utility, and it
can be smaller than the ELF format because of the compression support.
2. The kdump-compressed format is the 5th edition.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
configure | 50 +++++++++++++++++++++++
dump.c | 106 +++++++++++++++++++++++++++++++++++++++++++-----
hmp-commands.hx | 12 ++++--
hmp.c | 23 ++++++++++-
include/sysemu/dump.h | 8 ++++
qapi-schema.json | 23 ++++++++++-
qmp-commands.hx | 6 ++-
7 files changed, 208 insertions(+), 20 deletions(-)
diff --git a/configure b/configure
index 9439f1c..b89beea 100755
--- a/configure
+++ b/configure
@@ -229,6 +229,8 @@ libusb=""
usb_redir=""
glx=""
zlib="yes"
+lzo="no"
+snappy="no"
guest_agent="yes"
want_tools="yes"
libiscsi=""
@@ -898,6 +900,10 @@ for opt do
;;
--disable-zlib-test) zlib="no"
;;
+ --enable-lzo) lzo="yes"
+ ;;
+ --enable-snappy) snappy="yes"
+ ;;
--enable-guest-agent) guest_agent="yes"
;;
--disable-guest-agent) guest_agent="no"
@@ -1494,6 +1500,42 @@ fi
libs_softmmu="$libs_softmmu -lz"
##########################################
+# lzo check
+
+if test "$lzo" != "no" ; then
+ cat > $TMPC << EOF
+#include <lzo/lzo1x.h>
+int main(void) { lzo_version(); return 0; }
+EOF
+ if compile_prog "" "-llzo2" ; then
+ :
+ else
+ error_exit "lzo check failed" \
+ "Make sure to have the lzo libs and headers installed."
+ fi
+
+ libs_softmmu="$libs_softmmu -llzo2"
+fi
+
+##########################################
+# snappy check
+
+if test "$snappy" != "no" ; then
+ cat > $TMPC << EOF
+#include <snappy-c.h>
+int main(void) { snappy_max_compressed_length(4096); return 0; }
+EOF
+ if compile_prog "" "-lsnappy" ; then
+ :
+ else
+ error_exit "snappy check failed" \
+ "Make sure to have the snappy libs and headers installed."
+ fi
+
+ libs_softmmu="$libs_softmmu -lsnappy"
+fi
+
+##########################################
# libseccomp check
if test "$seccomp" != "no" ; then
@@ -3891,6 +3933,14 @@ if test "$glx" = "yes" ; then
echo "GLX_LIBS=$glx_libs" >> $config_host_mak
fi
+if test "$lzo" = "yes" ; then
+ echo "CONFIG_LZO=y" >> $config_host_mak
+fi
+
+if test "$snappy" = "yes" ; then
+ echo "CONFIG_SNAPPY=y" >> $config_host_mak
+fi
+
if test "$libiscsi" = "yes" ; then
echo "CONFIG_LIBISCSI=y" >> $config_host_mak
fi
diff --git a/dump.c b/dump.c
index c04e2ae..74aa727 100644
--- a/dump.c
+++ b/dump.c
@@ -1027,6 +1027,16 @@ static int create_header64(DumpState *s)
return 0;
}
+static void get_max_mapnr(DumpState *s)
+{
+ MemoryMapping *memory_mapping;
+
+ QTAILQ_FOREACH(memory_mapping, &s->list.head, next) {
+ s->max_mapnr = paddr_to_pfn(memory_mapping->phys_addr +
+ memory_mapping->length, s->page_shift);
+ }
+}
+
/*
* gather data of header and sub header
*/
@@ -1395,12 +1405,14 @@ out:
return ret;
}
-static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
- int64_t begin, int64_t length, Error **errp)
+static int dump_init(DumpState *s, int fd, bool compress_format,
+ DumpGuestMemoryFormat format, bool paging,
+ bool has_filter, int64_t begin, int64_t length, Error **errp)
{
CPUArchState *env;
int nr_cpus;
int ret;
+ unsigned long tmp;
if (runstate_is_running()) {
vm_stop(RUN_STATE_SAVE_VM);
@@ -1455,6 +1467,56 @@ static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
qemu_get_guest_simple_memory_mapping(&s->list);
}
+ /* init for kdump-compressed format */
+ if (compress_format) {
+ switch (format) {
+ case DUMP_GUEST_MEMORY_FORMAT_ZLIB:
+ s->flag_compress = DUMP_DH_COMPRESSED_ZLIB;
+ break;
+ case DUMP_GUEST_MEMORY_FORMAT_LZO:
+ s->flag_compress = DUMP_DH_COMPRESSED_LZO;
+ break;
+ case DUMP_GUEST_MEMORY_FORMAT_SNAPPY:
+ s->flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
+ break;
+ default:
+ s->flag_compress = 0;
+ }
+
+ s->nr_cpus = nr_cpus;
+ s->page_size = PAGE_SIZE;
+ s->page_shift = ffs(s->page_size) - 1;
+
+ get_max_mapnr(s);
+
+ tmp = divideup(divideup(s->max_mapnr, CHAR_BIT), s->page_size);
+ s->len_dump_bitmap = tmp * s->page_size * 2;
+
+ /*
+ * gather data of header, dump_bitmap, page_desc and page_data, then
+ * cache them in tmp files
+ */
+ ret = create_header(s);
+ if (ret < 0) {
+ error_set(errp, QERR_UNSUPPORTED);
+ goto cleanup;
+ }
+
+ ret = create_dump_bitmap(s);
+ if (ret < 0) {
+ error_set(errp, QERR_UNSUPPORTED);
+ goto cleanup;
+ }
+
+ ret = create_pages(s);
+ if (ret < 0) {
+ error_set(errp, QERR_UNSUPPORTED);
+ goto cleanup;
+ }
+
+ return 0;
+ }
+
if (s->has_filter) {
memory_mapping_filter(&s->list, s->begin, s->length);
}
@@ -1530,14 +1592,26 @@ static void clean_state(DumpState *s)
free_cache_data(s->page_data);
}
-void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
- int64_t begin, bool has_length, int64_t length,
- Error **errp)
+void qmp_dump_guest_memory(bool paging, const char *protocol, bool has_begin,
+ int64_t begin, bool has_length,
+ int64_t length, bool has_format,
+ DumpGuestMemoryFormat format, Error **errp)
{
const char *p;
int fd = -1;
DumpState *s;
int ret;
+ int compress_format = 0;
+
+ if (has_format && format != DUMP_GUEST_MEMORY_FORMAT_ELF) {
+ compress_format = 1;
+ }
+
+ /* kdump-compressed format is invalid with paging or filter */
+ if (compress_format && (paging || has_begin || has_length)) {
+ error_set(errp, QERR_INVALID_PARAMETER_COMBINATION);
+ return;
+ }
if (has_begin && !has_length) {
error_set(errp, QERR_MISSING_PARAMETER, "length");
@@ -1549,7 +1623,7 @@ void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
}
#if !defined(WIN32)
- if (strstart(file, "fd:", &p)) {
+ if (strstart(protocol, "fd:", &p)) {
fd = monitor_get_fd(cur_mon, p, errp);
if (fd == -1) {
return;
@@ -1557,7 +1631,7 @@ void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
}
#endif
- if (strstart(file, "file:", &p)) {
+ if (strstart(protocol, "file:", &p)) {
fd = qemu_open(p, O_WRONLY | O_CREAT | O_TRUNC | O_BINARY, S_IRUSR);
if (fd < 0) {
error_set(errp, QERR_OPEN_FILE_FAILED, p);
@@ -1570,17 +1644,27 @@ void qmp_dump_guest_memory(bool paging, const char *file, bool has_begin,
return;
}
- s = g_malloc(sizeof(DumpState));
+ /* initialize DumpState to zero */
+ s = g_malloc0(sizeof(DumpState));
- ret = dump_init(s, fd, paging, has_begin, begin, length, errp);
+ ret = dump_init(s, fd, compress_format, format, paging, has_begin,
+ begin, length, errp);
if (ret < 0) {
g_free(s);
return;
}
- if (create_vmcore(s) < 0 && !error_is_set(s->errp)) {
- error_set(errp, QERR_IO_ERROR);
+ if (compress_format) {
+ if (create_kdump_vmcore(s) < 0 && !error_is_set(s->errp)) {
+ error_set(errp, QERR_IO_ERROR);
+ }
+ } else {
+ if (create_vmcore(s) < 0 && !error_is_set(s->errp)) {
+ error_set(errp, QERR_IO_ERROR);
+ }
}
+ clean_state(s);
+
g_free(s);
}
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 9cea415..b12e50b 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -992,17 +992,19 @@ ETEXI
#if defined(CONFIG_HAVE_CORE_DUMP)
{
.name = "dump-guest-memory",
- .args_type = "paging:-p,filename:F,begin:i?,length:i?",
- .params = "[-p] filename [begin] [length]",
+ .args_type = "paging:-p,filename:F,begin:i?,length:i?,format:s?",
+ .params = "[-p] filename [begin] [length] [format]",
.help = "dump guest memory to file"
"\n\t\t\t begin(optional): the starting physical address"
- "\n\t\t\t length(optional): the memory size, in bytes",
+ "\n\t\t\t length(optional): the memory size, in bytes"
+ "\n\t\t\t format(optional): the format of guest memory dump,"
+ "\n\t\t\t it can be elf|zlib|lzo|snappy",
.mhandler.cmd = hmp_dump_guest_memory,
},
STEXI
-@item dump-guest-memory [-p] @var{protocol} @var{begin} @var{length}
+@item dump-guest-memory [-p] @var{protocol} @var{begin} @var{length} @var{format}
@findex dump-guest-memory
Dump guest memory to @var{protocol}. The file can be processed with crash or
gdb.
@@ -1012,6 +1014,8 @@ gdb.
specified with length together.
length: the memory size, in bytes. It's optional, and should be specified
with begin together.
+ format: the format of guest memory dump. It's optional, and can be
+ elf|zlib|lzo|snappy.
ETEXI
#endif
diff --git a/hmp.c b/hmp.c
index 4fb76ec..e863267 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1185,9 +1185,12 @@ void hmp_dump_guest_memory(Monitor *mon, const QDict *qdict)
const char *file = qdict_get_str(qdict, "filename");
bool has_begin = qdict_haskey(qdict, "begin");
bool has_length = qdict_haskey(qdict, "length");
+ bool has_format = qdict_haskey(qdict, "format");
int64_t begin = 0;
int64_t length = 0;
+ const char *format = NULL;
char *prot;
+ enum DumpGuestMemoryFormat dump_format;
if (has_begin) {
begin = qdict_get_int(qdict, "begin");
@@ -1195,11 +1198,29 @@ void hmp_dump_guest_memory(Monitor *mon, const QDict *qdict)
if (has_length) {
length = qdict_get_int(qdict, "length");
}
+ if (has_format) {
+ format = qdict_get_str(qdict, "format");
+ }
+
+ if (strcmp(format, "elf") == 0) {
+ dump_format = DUMP_GUEST_MEMORY_FORMAT_ELF;
+ } else if (strcmp(format, "zlib") == 0) {
+ dump_format = DUMP_GUEST_MEMORY_FORMAT_ZLIB;
+ } else if (strcmp(format, "lzo") == 0) {
+ dump_format = DUMP_GUEST_MEMORY_FORMAT_LZO;
+ } else if (strcmp(format, "snappy") == 0) {
+ dump_format = DUMP_GUEST_MEMORY_FORMAT_SNAPPY;
+ } else {
+ error_set(&errp, QERR_INVALID_PARAMETER_VALUE,
+ "format", "elf|zlib|lzo|snappy");
+ hmp_handle_error(mon, &errp);
+ return;
+ }
prot = g_strconcat("file:", file, NULL);
qmp_dump_guest_memory(paging, prot, has_begin, begin, has_length, length,
- &errp);
+ has_format, dump_format, &errp);
hmp_handle_error(mon, &errp);
g_free(prot);
}
diff --git a/include/sysemu/dump.h b/include/sysemu/dump.h
index 000455f..4da8a33 100644
--- a/include/sysemu/dump.h
+++ b/include/sysemu/dump.h
@@ -37,6 +37,13 @@
#endif
/*
+ * dump format
+ */
+#define FLAG_DUMP_COMPRESS_ZLIB (0x1) /* compressed with zlib */
+#define FLAG_DUMP_COMPRESS_LZO (0x2) /* compressed with lzo */
+#define FLAG_DUMP_COMPRESS_SNAPPY (0x4) /* compressed with snappy */
+
+/*
* flag used in page desc of kdump-compressed format
*/
#define DUMP_DH_COMPRESSED_ZLIB (0x1)
@@ -46,6 +53,7 @@
#define KDUMP_SIGNATURE "KDUMP "
#define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
#define DISKDUMP_HEADER_BLOCKS (1)
+#define PAGE_SIZE (4096)
#define PHYS_BASE (0)
#define DUMP_LEVEL (1)
#define TMP_BUF_SIZE (1024)
diff --git a/qapi-schema.json b/qapi-schema.json
index 199744a..535dbec 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -2381,6 +2381,18 @@
{ 'command': 'device_del', 'data': {'id': 'str'} }
##
+# @DumpGuestMemoryFormat
+#
+# Format of Guest memory dump
+#
+# Each represents a format of guest memory dump.
+#
+# Since: 1.6
+##
+{ 'enum': 'DumpGuestMemoryFormat',
+ 'data': [ 'elf', 'zlib', 'lzo', 'snappy' ] }
+
+##
# @dump-guest-memory
#
# Dump guest's memory to vmcore. It is a synchronous operation that can take
@@ -2416,13 +2428,20 @@
# want to dump all guest's memory, please specify the start @begin
# and @length
#
+# @format: #optional if specified, the format of guest memory dump.
+#
+# 1. elf: dump file will be in elf format
+# 2. zlib: dump file will be in kdump format with zlib-compressed
+# 3. lzo: dump file will be in kdump format with lzo-compressed
+# 4. snappy: dump file will be in kdump format with snappy-compressed
+#
# Returns: nothing on success
#
-# Since: 1.2
+# Since: 1.6
##
{ 'command': 'dump-guest-memory',
'data': { 'paging': 'bool', 'protocol': 'str', '*begin': 'int',
- '*length': 'int' } }
+ '*length': 'int', '*format': 'DumpGuestMemoryFormat' } }
##
# @netdev_add:
diff --git a/qmp-commands.hx b/qmp-commands.hx
index ffd130e..f106fc1 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -788,8 +788,8 @@ EQMP
{
.name = "dump-guest-memory",
- .args_type = "paging:b,protocol:s,begin:i?,end:i?",
- .params = "-p protocol [begin] [length]",
+ .args_type = "paging:b,protocol:s,begin:i?,length:i?,format:s?",
+ .params = "-p protocol [begin] [length] [format]",
.help = "dump guest memory to file",
.user_print = monitor_user_noop,
.mhandler.cmd_new = qmp_marshal_input_dump_guest_memory,
@@ -810,6 +810,8 @@ Arguments:
with length together (json-int)
- "length": the memory size, in bytes. It's optional, and should be specified
with begin together (json-int)
+- "format": the format of guest memory dump. It's optional, and can be
+ elf|zlib|lzo|snappy(json-string)
Example:
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format
2013-05-15 2:29 ` [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
@ 2013-05-15 2:55 ` Eric Blake
2013-05-15 5:27 ` Qiao Nuohan
0 siblings, 1 reply; 16+ messages in thread
From: Eric Blake @ 2013-05-15 2:55 UTC (permalink / raw)
To: Qiao Nuohan
Cc: qemu-devel, d.hatayama, kumagai-atsushi, zhangxh, anderson, afaerber
[-- Attachment #1: Type: text/plain, Size: 4796 bytes --]
On 05/14/2013 08:29 PM, Qiao Nuohan wrote:
> Make monitor command 'dump-guest-memory' dump in kdump-compressed format.
> The command's usage:
> dump [-p] protocol [begin] [length] [format]
> 'format' is used to specified the format of vmcore and can be:
> 1. 'elf': ELF format, without compression
> 2. 'zlib': kdump-compressed format, with zlib-compressed
> 3. 'lzo': kdump-compressed format, with lzo-compressed
> 4. 'snappy': kdump-compressed format, with snappy-compressed
> And without 'format' being set, vmcore will be in ELF format.
>
> Note:
> 1. The kdump-compressed format is readable only with the crash utility, and it
> can be smaller than the ELF format because of the compression support.
> 2. The kdump-compressed format is the 5th edition.
>
> Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
> Signed-off-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
> ---
> -static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
> - int64_t begin, int64_t length, Error **errp)
> +static int dump_init(DumpState *s, int fd, bool compress_format,
> + DumpGuestMemoryFormat format, bool paging,
> + bool has_filter, int64_t begin, int64_t length, Error **errp)
> {
Why do you need compress_format as a separate parameter, when that
information is redundant with the contents of format?
> +void qmp_dump_guest_memory(bool paging, const char *protocol, bool has_begin,
> + int64_t begin, bool has_length,
> + int64_t length, bool has_format,
> + DumpGuestMemoryFormat format, Error **errp)
> {
> const char *p;
> int fd = -1;
> DumpState *s;
> int ret;
> + int compress_format = 0;
You are using this as a bool, so type it as a bool rather than int.
> +++ b/include/sysemu/dump.h
> @@ -37,6 +37,13 @@
> #endif
>
> /*
> + * dump format
> + */
> +#define FLAG_DUMP_COMPRESS_ZLIB (0x1) /* compressed with zlib */
> +#define FLAG_DUMP_COMPRESS_LZO (0x2) /* compressed with lzo */
> +#define FLAG_DUMP_COMPRESS_SNAPPY (0x4) /* compressed with snappy */
Why are you skipping 3? Besides, these aren't flags that can be
bitwise-or'd together, so the name FLAG_ is misleading. Why not just
make it an enum, with value 0, 1, 2 (or with 1, 2, 3)? In fact, why
even declare this at all, instead of reusing the DumpGuestMemoryFormat
enum created for you by the QMP code generation from the schema?
> +
> +/*
> * flag used in page desc of kdump-compressed format
> */
> #define DUMP_DH_COMPRESSED_ZLIB (0x1)
> @@ -46,6 +53,7 @@
> #define KDUMP_SIGNATURE "KDUMP "
> #define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
> #define DISKDUMP_HEADER_BLOCKS (1)
> +#define PAGE_SIZE (4096)
Is that true for all platforms?
> +++ b/qapi-schema.json
> @@ -2381,6 +2381,18 @@
> { 'command': 'device_del', 'data': {'id': 'str'} }
>
> ##
> +# @DumpGuestMemoryFormat
> +#
> +# Format of Guest memory dump
> +#
> +# Each represents a format of guest memory dump.
> +#
> +# Since: 1.6
> +##
> +{ 'enum': 'DumpGuestMemoryFormat',
> + 'data': [ 'elf', 'zlib', 'lzo', 'snappy' ] }
For several other enums, we document what each enum value means here.
That way, if the enum gets used more than once, you won't have to repeat...
> +
> +##
> # @dump-guest-memory
> #
> # Dump guest's memory to vmcore. It is a synchronous operation that can take
> @@ -2416,13 +2428,20 @@
> # want to dump all guest's memory, please specify the start @begin
> # and @length
> #
> +# @format: #optional if specified, the format of guest memory dump.
> +#
> +# 1. elf: dump file will be in elf format
> +# 2. zlib: dump file will be in kdump format with zlib-compressed
> +# 3. lzo: dump file will be in kdump format with lzo-compressed
> +# 4. snappy: dump file will be in kdump format with snappy-compressed
...this text in each other use of the enum. Also, if we extend the enum
in the future, you don't have to revisit quite as much documentation.
> +#
> # Returns: nothing on success
> #
> -# Since: 1.2
> +# Since: 1.6
Not quite right. The dump-guest-memory command is still Since: 1.2.
Rather, it is the @format: line that should add "(since 1.6)" to show
that one additional option was added later. Look at drive-mirror's
@buf-size for an example.
Interface is looking better, though; thanks for folding in the review
suggestions.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format
2013-05-15 2:55 ` Eric Blake
@ 2013-05-15 5:27 ` Qiao Nuohan
2013-05-15 12:10 ` Eric Blake
0 siblings, 1 reply; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-15 5:27 UTC (permalink / raw)
To: Eric Blake
Cc: qemu-devel, d.hatayama, kumagai-atsushi, zhangxh, anderson, afaerber
On 05/15/2013 10:55 AM, Eric Blake wrote:
> On 05/14/2013 08:29 PM, Qiao Nuohan wrote:
>> Make monitor command 'dump-guest-memory' dump in kdump-compressed format.
>> The command's usage:
>> dump [-p] protocol [begin] [length] [format]
>> 'format' is used to specified the format of vmcore and can be:
>> 1. 'elf': ELF format, without compression
>> 2. 'zlib': kdump-compressed format, with zlib-compressed
>> 3. 'lzo': kdump-compressed format, with lzo-compressed
>> 4. 'snappy': kdump-compressed format, with snappy-compressed
>> And without 'format' being set, vmcore will be in ELF format.
>>
>> Note:
>> 1. The kdump-compressed format is readable only with the crash utility, and it
>> can be smaller than the ELF format because of the compression support.
>> 2. The kdump-compressed format is the 5th edition.
>>
>> Signed-off-by: Qiao Nuohan<qiaonuohan@cn.fujitsu.com>
>> Signed-off-by: Zhang Xiaohe<zhangxh@cn.fujitsu.com>
>> ---
>
>> -static int dump_init(DumpState *s, int fd, bool paging, bool has_filter,
>> - int64_t begin, int64_t length, Error **errp)
>> +static int dump_init(DumpState *s, int fd, bool compress_format,
>> + DumpGuestMemoryFormat format, bool paging,
>> + bool has_filter, int64_t begin, int64_t length, Error **errp)
>> {
>
> Why do you need compress_format as a separate parameter, when that
> information is redundant with the contents of format?
I need this variable to decide whether in kdump-compressed format or
not. Without using it, I need to repeat "has_format && format !=
DUMP_GUEST_MEMORY_FORMAT_ELF" three times.
>
>> +void qmp_dump_guest_memory(bool paging, const char *protocol, bool has_begin,
>> + int64_t begin, bool has_length,
>> + int64_t length, bool has_format,
>> + DumpGuestMemoryFormat format, Error **errp)
>> {
>> const char *p;
>> int fd = -1;
>> DumpState *s;
>> int ret;
>> + int compress_format = 0;
>
> You are using this as a bool, so type it as a bool rather than int.
Got it.
>
>> +++ b/include/sysemu/dump.h
>> @@ -37,6 +37,13 @@
>> #endif
>>
>> /*
>> + * dump format
>> + */
>> +#define FLAG_DUMP_COMPRESS_ZLIB (0x1) /* compressed with zlib */
>> +#define FLAG_DUMP_COMPRESS_LZO (0x2) /* compressed with lzo */
>> +#define FLAG_DUMP_COMPRESS_SNAPPY (0x4) /* compressed with snappy */
>
> Why are you skipping 3? Besides, these aren't flags that can be
> bitwise-or'd together, so the name FLAG_ is misleading. Why not just
> make it an enum, with value 0, 1, 2 (or with 1, 2, 3)? In fact, why
> even declare this at all, instead of reusing the DumpGuestMemoryFormat
> enum created for you by the QMP code generation from the schema?
Sorry about this, I didn't use these but forgot to remove them.
>
>> +
>> +/*
>> * flag used in page desc of kdump-compressed format
>> */
>> #define DUMP_DH_COMPRESSED_ZLIB (0x1)
>> @@ -46,6 +53,7 @@
>> #define KDUMP_SIGNATURE "KDUMP "
>> #define SIG_LEN (sizeof(KDUMP_SIGNATURE) - 1)
>> #define DISKDUMP_HEADER_BLOCKS (1)
>> +#define PAGE_SIZE (4096)
>
> Is that true for all platforms?
No. In "note" of patch 0/9,
"The guest should be x86 or x86_64. The other arch is not supported
now."
PAGE_SIZE here is only supposed to support x86 or x86_64.
--
Regards
Qiao Nuohan
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format
2013-05-15 5:27 ` Qiao Nuohan
@ 2013-05-15 12:10 ` Eric Blake
0 siblings, 0 replies; 16+ messages in thread
From: Eric Blake @ 2013-05-15 12:10 UTC (permalink / raw)
To: Qiao Nuohan
Cc: qemu-devel, d.hatayama, kumagai-atsushi, zhangxh, anderson, afaerber
[-- Attachment #1: Type: text/plain, Size: 907 bytes --]
On 05/14/2013 11:27 PM, Qiao Nuohan wrote:
>>> +static int dump_init(DumpState *s, int fd, bool compress_format,
>>> + DumpGuestMemoryFormat format, bool paging,
>>> + bool has_filter, int64_t begin, int64_t length,
>>> Error **errp)
>>> {
>>
>> Why do you need compress_format as a separate parameter, when that
>> information is redundant with the contents of format?
>
> I need this variable to decide whether in kdump-compressed format or
> not. Without using it, I need to repeat "has_format && format !=
> DUMP_GUEST_MEMORY_FORMAT_ELF" three times.
Then set it up as a local stack-allocated variable within the function
body. No need to pass it as part of the function signature, to make all
callers pass redundant information.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 621 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap
2013-05-15 2:29 ` [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap Qiao Nuohan
@ 2013-05-17 8:20 ` Andreas Färber
2013-05-17 8:33 ` Qiao Nuohan
0 siblings, 1 reply; 16+ messages in thread
From: Andreas Färber @ 2013-05-17 8:20 UTC (permalink / raw)
To: Qiao Nuohan
Cc: qemu-devel, d.hatayama, kumagai-atsushi, zhangxh, anderson,
Jens Freimann
Am 15.05.2013 04:29, schrieb Qiao Nuohan:
> Struct dump_bitmap is associated with a tmp file, and the tmp file can be used
> to save data of bitmap in kdump-compressed format temporarily.
> The following patch will use these functions to get the data of bitmap and cache
> them into tmp files.
>
> Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
> Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
> ---
> Makefile.target | 2 +-
> dump_bitmap.c | 171 +++++++++++++++++++++++++++++++++++++++++++++++++
> include/dump_bitmap.h | 60 +++++++++++++++++
> 3 files changed, 232 insertions(+), 1 deletions(-)
> create mode 100644 dump_bitmap.c
> create mode 100644 include/dump_bitmap.h
>
> diff --git a/Makefile.target b/Makefile.target
> index ce4391f..00d4f13 100644
> --- a/Makefile.target
> +++ b/Makefile.target
> @@ -112,7 +112,7 @@ obj-$(CONFIG_FDT) += device_tree.o
> obj-$(CONFIG_KVM) += kvm-all.o
> obj-y += memory.o savevm.o cputlb.o
> obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
> -obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
> +obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o
> obj-$(CONFIG_NO_GET_MEMORY_MAPPING) += memory_mapping-stub.o
> obj-$(CONFIG_NO_CORE_DUMP) += dump-stub.o
> LIBS+=$(libs_softmmu)
[snip]
Jens and me have been working towards removing the dependency on
CONFIG_HAVE_CORE_DUMP. Would there be any issue building this code
unconditionally using obj-y += dump_bitmap.o ?
Regards,
Andreas
--
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap
2013-05-17 8:20 ` Andreas Färber
@ 2013-05-17 8:33 ` Qiao Nuohan
0 siblings, 0 replies; 16+ messages in thread
From: Qiao Nuohan @ 2013-05-17 8:33 UTC (permalink / raw)
To: Andreas Färber
Cc: qemu-devel, d.hatayama, kumagai-atsushi, zhangxh, anderson,
Jens Freimann
On 05/17/2013 04:20 PM, Andreas Färber wrote:
> Jens and me have been working towards removing the dependency on
> CONFIG_HAVE_CORE_DUMP. Would there be any issue building this code
> unconditionally using obj-y += dump_bitmap.o ?
No problems will be involved.
--
Regards
Qiao Nuohan
^ permalink raw reply [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap
2013-05-08 2:12 [Qemu-devel] [PATCH 0/9 " qiaonuohan
@ 2013-05-08 2:12 ` qiaonuohan
0 siblings, 0 replies; 16+ messages in thread
From: qiaonuohan @ 2013-05-08 2:12 UTC (permalink / raw)
To: qemu-devel
Cc: Qiao Nuohan, d.hatayama, zhangxh, anderson, kumagai-atsushi, afaerber
From: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Struct dump_bitmap is associated with a tmp file, and the tmp file can be used
to save data of bitmap in kdump-compressed format temporarily.
The following patch will use these functions to get the data of bitmap and cache
them into tmp files.
Signed-off-by: Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
Reviewed-by: Zhang Xiaohe <zhangxh@cn.fujitsu.com>
---
Makefile.target | 2 +-
dump_bitmap.c | 171 +++++++++++++++++++++++++++++++++++++++++++++++++
include/dump_bitmap.h | 60 +++++++++++++++++
3 files changed, 232 insertions(+), 1 deletions(-)
create mode 100644 dump_bitmap.c
create mode 100644 include/dump_bitmap.h
diff --git a/Makefile.target b/Makefile.target
index ce4391f..00d4f13 100644
--- a/Makefile.target
+++ b/Makefile.target
@@ -112,7 +112,7 @@ obj-$(CONFIG_FDT) += device_tree.o
obj-$(CONFIG_KVM) += kvm-all.o
obj-y += memory.o savevm.o cputlb.o
obj-$(CONFIG_HAVE_GET_MEMORY_MAPPING) += memory_mapping.o
-obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o
+obj-$(CONFIG_HAVE_CORE_DUMP) += dump.o dump_bitmap.o
obj-$(CONFIG_NO_GET_MEMORY_MAPPING) += memory_mapping-stub.o
obj-$(CONFIG_NO_CORE_DUMP) += dump-stub.o
LIBS+=$(libs_softmmu)
diff --git a/dump_bitmap.c b/dump_bitmap.c
new file mode 100644
index 0000000..d2f6c9f
--- /dev/null
+++ b/dump_bitmap.c
@@ -0,0 +1,171 @@
+/*
+ * QEMU dump bitmap
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu-common.h"
+#include "dump_bitmap.h"
+
+int init_dump_bitmap(struct dump_bitmap *db, const char *filename)
+{
+ int fd;
+ char *tmpname;
+
+ /* init the tmp file */
+ tmpname = getenv("TMPDIR");
+ if (!tmpname) {
+ tmpname = (char *)P_tmpdir;
+ }
+
+ db->file_name = (char *)g_strdup_printf("%s/%s", tmpname, filename);
+
+ fd = mkstemp(db->file_name);
+ if (fd < 0) {
+ return -1;
+ }
+
+ db->fd = fd;
+ unlink(db->file_name);
+
+ /* num_block should be set to -1, for nothing is store in buf now */
+ db->num_block = -1;
+ memset(db->buf, 0, BUFSIZE_BITMAP);
+ /* the tmp file starts to write at the very beginning */
+ db->offset = 0;
+
+ return 0;
+}
+
+int clear_dump_bitmap(struct dump_bitmap *db, size_t len_db)
+{
+ int ret;
+ char buf[BUFSIZE_BITMAP];
+ off_t offset_bit;
+
+ /* use buf to clear the tmp file block by block */
+ memset(buf, 0, sizeof(buf));
+
+ ret = lseek(db->fd, db->offset, SEEK_SET);
+ if (ret < 0) {
+ return -1;
+ }
+
+ offset_bit = 0;
+ while (offset_bit < len_db) {
+ if (write(db->fd, buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+
+ offset_bit += BUFSIZE_BITMAP;
+ }
+
+ return 0;
+}
+
+int set_dump_bitmap(struct dump_bitmap *db, unsigned long long pfn, int val)
+{
+ int byte, bit;
+ off_t old_offset, new_offset;
+ old_offset = db->offset + BUFSIZE_BITMAP * db->num_block;
+ new_offset = db->offset + BUFSIZE_BITMAP * (pfn / PFN_BUFBITMAP);
+
+ /*
+ * if the block needed to be set is not same as the one cached in buf,
+ * write the block back to the tmp file, then cache new block to the buf
+ */
+ if (0 <= db->num_block && old_offset != new_offset) {
+ if (lseek(db->fd, old_offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (write(db->fd, db->buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+ }
+
+ if (old_offset != new_offset) {
+ if (lseek(db->fd, new_offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (read(db->fd, db->buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+
+ db->num_block = pfn / PFN_BUFBITMAP;
+ }
+
+ /* get the exact place of the bit in the buf, set it */
+ byte = (pfn % PFN_BUFBITMAP) >> 3;
+ bit = (pfn % PFN_BUFBITMAP) & 7;
+ if (val) {
+ db->buf[byte] |= 1<<bit;
+ } else {
+ db->buf[byte] &= ~(1<<bit);
+ }
+
+ return 0;
+}
+
+int sync_dump_bitmap(struct dump_bitmap *db)
+{
+ off_t offset;
+ offset = db->offset + BUFSIZE_BITMAP * db->num_block;
+
+ /* db has not been used yet */
+ if (db->num_block < 0) {
+ return 0;
+ }
+
+ if (lseek(db->fd, offset, SEEK_SET) < 0) {
+ return -1;
+ }
+
+ if (write(db->fd, db->buf, BUFSIZE_BITMAP) != BUFSIZE_BITMAP) {
+ return -1;
+ }
+
+ return 0;
+}
+
+/*
+ * check if the bit is set to 1
+ */
+static inline int is_on(char *bitmap, int i)
+{
+ return bitmap[i>>3] & (1 << (i & 7));
+}
+
+int is_bit_set(struct dump_bitmap *db, unsigned long long pfn)
+{
+ off_t offset;
+
+ /* cache the block pfn belongs to, then check the block */
+ if (pfn == 0 || db->num_block != pfn/PFN_BUFBITMAP) {
+ offset = db->offset + BUFSIZE_BITMAP * (pfn/PFN_BUFBITMAP);
+ lseek(db->fd, offset, SEEK_SET);
+ read(db->fd, db->buf, BUFSIZE_BITMAP);
+ db->num_block = pfn/PFN_BUFBITMAP;
+ }
+
+ return is_on(db->buf, pfn%PFN_BUFBITMAP);
+}
+
+void free_dump_bitmap(struct dump_bitmap *db)
+{
+ if (db) {
+ if (db->file_name) {
+ g_free(db->file_name);
+ }
+
+ g_free(db);
+ }
+}
diff --git a/include/dump_bitmap.h b/include/dump_bitmap.h
new file mode 100644
index 0000000..ee91543
--- /dev/null
+++ b/include/dump_bitmap.h
@@ -0,0 +1,60 @@
+/*
+ * QEMU dump bitmap
+ *
+ * Copyright Fujitsu, Corp. 2013
+ *
+ * Authors:
+ * Qiao Nuohan <qiaonuohan@cn.fujitsu.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#ifndef DUMP_BITMAP_H
+#define DUMP_BITMAP_H
+
+#define BUFSIZE_BITMAP (4096)
+#define PFN_BUFBITMAP (CHAR_BIT * BUFSIZE_BITMAP)
+
+struct dump_bitmap {
+ int fd; /* fd of the tmp file */
+ int num_block; /* number of blocks cached in buf */
+ char *file_name; /* name of the tmp file */
+ char buf[BUFSIZE_BITMAP]; /* used to cache blocks */
+ off_t offset; /* offset of the tmp file */
+};
+
+/*
+ * create a tmp file used to store dump bitmap, then init buf of dump_bitmap
+ */
+int init_dump_bitmap(struct dump_bitmap *db, const char *filename);
+
+/*
+ * clear the content of the tmp file with all bits set to 0
+ */
+int clear_dump_bitmap(struct dump_bitmap *db, size_t len_db);
+
+/*
+ * 'pfn' is the number of bit needed to be set, use buf to cache the block which
+ * 'pfn' belongs to, then set 'val' to the bit
+ */
+int set_dump_bitmap(struct dump_bitmap *db, unsigned long long pfn, int val);
+
+/*
+ * when buf is caching a block, sync_dump_bitmap is needed to write the cached
+ * block to the tmp file
+ */
+int sync_dump_bitmap(struct dump_bitmap *db);
+
+/*
+ * check whether 'pfn' is set
+ */
+int is_bit_set(struct dump_bitmap *db, unsigned long long pfn);
+
+/*
+ * free the space used by dump_bitmap
+ */
+void free_dump_bitmap(struct dump_bitmap *db);
+
+#endif
--
1.7.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
end of thread, other threads:[~2013-05-17 9:00 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-05-15 2:29 [Qemu-devel] [PATCH 0/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap Qiao Nuohan
2013-05-17 8:20 ` Andreas Färber
2013-05-17 8:33 ` Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 2/9 v2] Add API to manipulate cache_data Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 3/9 v2] Move includes and struct definition to dump.h Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 4/9 v2] Add API to create header of vmcore Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 5/9 v2] Add API to create data of dump bitmap Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 6/9 v2] Add API to create page Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 7/9 v2] Add API to free buf used by creating header, bitmap and page Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 8/9 v2] Add API to write header, bitmap and page into vmcore Qiao Nuohan
2013-05-15 2:29 ` [Qemu-devel] [PATCH 9/9 v2] Make monitor command 'dump-guest-memory' dump in kdump-compressed format Qiao Nuohan
2013-05-15 2:55 ` Eric Blake
2013-05-15 5:27 ` Qiao Nuohan
2013-05-15 12:10 ` Eric Blake
-- strict thread matches above, loose matches on Subject: below --
2013-05-08 2:12 [Qemu-devel] [PATCH 0/9 " qiaonuohan
2013-05-08 2:12 ` [Qemu-devel] [PATCH 1/9 v2] Add API to manipulate dump_bitmap qiaonuohan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.