From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32827) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YEIKV-0000hY-0U for qemu-devel@nongnu.org; Thu, 22 Jan 2015 09:01:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YEIKN-00017G-JI for qemu-devel@nongnu.org; Thu, 22 Jan 2015 09:01:50 -0500 Received: from cantor2.suse.de ([195.135.220.15]:39672 helo=mx2.suse.de) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YEIKN-000170-Cj for qemu-devel@nongnu.org; Thu, 22 Jan 2015 09:01:43 -0500 From: Alexander Graf Date: Thu, 22 Jan 2015 15:01:36 +0100 Message-Id: <1421935300-8579-1-git-send-email-agraf@suse.de> Subject: [Qemu-devel] [PATCH v4 0/4] Migration Deciphering aid List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: quintela@redhat.com, amit.shah@redhat.com, pbonzini@redhat.com, alex.bennee@linaro.org, afaerber@suse.de Migration is a black hole to most people. One of the biggest reasons for this is that its protocol is a secret, undocumented sauce of code rolling around random parts of the QEMU code base. But what if we simply exposed the description of how the format looks like alongside the actual migration stream? This is what this patch set does. It adds a new section that comes after the end of stream marker (so that it doesn't slow down migration) that contains a JSON description of the device state description. Along with this patch set also comes a python script that can read said JSON from a migration dump and decipher the device state and ram contents of the migration dump using it. With this, you can now fully examine all glorious details that go over the wire when virtual machine state gets dumped, such as during live migration. We discussed the approach taken here during KVM Forum 2013. Originally, my idea was to include a special device that contains the JSON data which can be enabled on demand. Anthony suggested however to just always include the description data after the end marker which I think is a great idea. Example decoded migration: http://csgraf.de/mig/mig.txt Example migration description: http://csgraf.de/mig/mig.desc.txt Presentation: https://www.youtube.com/watch?v=iq1x40Qsrew Slides: https://www.dropbox.com/s/otp2pk2n3g087zp/Live%20Migration.pdf v1 -> v2: - a lot, v1 was from 2013 v2 -> v3: - QOMify the QJSON object, makes for easier destruction - improve ftell_fast, now works with bdrv too - Dont compress objects with subsections - Destroy QJSON object - Add tell function to MigrationFile - Report where subsections were not found - Print ram sections with size - Remove foreign desc file support - Add memory dump support (-m option) - Add -x option to extract all sections into files v3 -> v4: - Squash properly - Remove unused field from struct - Update patch description - Change copyright date to 2015 Alexander Graf (4): QJSON: Add JSON writer qemu-file: Add fast ftell code path migration: Append JSON description of migration stream Add migration stream analyzation script Makefile.objs | 1 + hw/pci/pci.c | 2 +- hw/scsi/spapr_vscsi.c | 2 +- hw/virtio/virtio.c | 2 +- include/migration/migration.h | 1 + include/migration/qemu-file.h | 1 + include/migration/vmstate.h | 3 +- include/qjson.h | 29 +++ migration/qemu-file.c | 16 ++ migration/vmstate.c | 186 ++++++++++++- qjson.c | 129 +++++++++ savevm.c | 54 +++- scripts/analyze-migration.py | 592 ++++++++++++++++++++++++++++++++++++++++++ tests/test-vmstate.c | 6 +- 14 files changed, 1005 insertions(+), 19 deletions(-) create mode 100644 include/qjson.h create mode 100644 qjson.c create mode 100755 scripts/analyze-migration.py -- 1.7.12.4