From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:37747) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QgXwt-0002qA-VE for qemu-devel@nongnu.org; Tue, 12 Jul 2011 04:04:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QgXwp-000463-Sy for qemu-devel@nongnu.org; Tue, 12 Jul 2011 04:04:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:44431) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QgXwp-00045a-DC for qemu-devel@nongnu.org; Tue, 12 Jul 2011 04:04:03 -0400 Message-ID: <4E1C009C.1010408@redhat.com> Date: Tue, 12 Jul 2011 10:06:52 +0200 From: Kevin Wolf MIME-Version: 1.0 References: <4E131D0D.307@redhat.com> <20110711125432.GA19686@stefanha-thinkpad.localdomain> <20110711163226.GA10924@amt.cnet> In-Reply-To: <20110711163226.GA10924@amt.cnet> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] live block copy/stream/snapshot discussion List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Marcelo Tosatti Cc: Anthony Liguori , Stefan Hajnoczi , Stefan Hajnoczi , Dor Laor , qemu-devel , Avi Kivity , Adam Litke Am 11.07.2011 18:32, schrieb Marcelo Tosatti: > On Mon, Jul 11, 2011 at 03:47:15PM +0100, Stefan Hajnoczi wrote: >> Kevin, Marcelo, >> I'd like to reach agreement on the QMP/HMP APIs for live block copy >> and image streaming. Libvirt has acked the image streaming APIs that >> Adam proposed and I think they are a good fit for the feature. I have >> described that API below for your review (it's exactly what the QED >> Image Streaming patches provide). >> >> Marcelo: Are you happy with this API for live block copy? Also please >> take a look at the switch command that I am proposing. >> >> Image streaming API >> =================== >> >> For leaf images with copy-on-read semantics, the stream commands allow the user >> to populate local blocks by manually streaming them from the backing image. >> Once all blocks have been streamed, the dependency on the original backing >> image can be removed. Therefore, stream commands can be used to implement >> post-copy live block migration and rapid deployment. >> >> The block_stream command can be used to stream a single cluster, to >> start streaming the entire device, and to cancel an active stream. It >> is easiest to allow the block_stream command to manage streaming for the >> entire device but a managent tool could use single cluster mode to >> throttle the I/O rate. As discussed earlier, having the management send requests for each single cluster doesn't make any sense at all. It wouldn't only throttle the I/O rate but bring it down to a level that makes it unusable. What you really want is to allow the management to give us a range (offset + length) that qemu should stream. >> The command synopses are as follows: >> >> block_stream >> ------------ >> >> Copy data from a backing file into a block device. >> >> If the optional 'all' argument is true, this operation is performed in the >> background until the entire backing file has been copied. The status of >> ongoing block_stream operations can be checked with query-block-stream. Not sure if it's a good idea to use a bool argument to turn a command into its opposite. I think having a separate command for stopping would be cleaner. Something for the QMP folks to decide, though. >> Arguments: >> >> - all: copy entire device (json-bool, optional) >> - stop: stop copying to device (json-bool, optional) >> - device: device name (json-string) > > It must be possible to specify backing file that will be > active after streaming finishes (data from that file will not > be streamed into active file, of course). Yes, I think the common base image belongs here. With all = false, where does the streaming begin? Do you have something like the "current streaming offset" in the state of each BlockDriverState? As I said above, I would prefer adding offset and length to the arguments. >> Return: >> >> - device: device name (json-string) >> - len: size of the device, in bytes (json-int) >> - offset: ending offset of the completed I/O, in bytes (json-int) So you only get the reply when the request has completed? With the current monitor, this means that QMP is blocked while we stream, doesn't it? How are you supposed to send the stop command then? Two of three examples below have an empty return value instead, so they are not compliant to this specification. >> Examples: >> >> -> { "execute": "block_stream", "arguments": { "device": "virtio0" } } >> <- { "return": { "device": "virtio0", "len": 10737418240, "offset": 512 } } >> >> -> { "execute": "block_stream", "arguments": { "all": true, "device": >> "virtio0" } } >> <- { "return": {} } >> >> -> { "execute": "block_stream", "arguments": { "stop": true, "device": >> "virtio0" } } >> <- { "return": {} } >> >> query-block-stream >> ------------------ >> >> Show progress of ongoing block_stream operations. >> >> Return a json-array of all operations. If no operation is active then an empty >> array will be returned. Each operation is a json-object with the following >> data: >> >> - device: device name (json-string) >> - len: size of the device, in bytes (json-int) >> - offset: ending offset of the completed I/O, in bytes (json-int) >> >> Example: >> >> -> { "execute": "query-block-stream" } >> <- { "return":[ >> { "device": "virtio0", "len": 10737418240, "offset": 709632} >> ] >> } When block_stream is changed, this will have to make the same changes. >> Block device switching API >> ========================== >> >> Extend the 'change' command to support changing the image file without >> media change notification. >> >> Perhaps we should take the opportunity to add a "format" argument for >> image files? >> >> change >> ------ >> >> Change a removable medium or VNC configuration. >> >> Arguments: >> >> - "device": device name (json-string) >> - "target": filename or item (json-string) >> - "arg": additional argument (json-string, optional) >> - "notify": whether to notify guest, defaults to true (json-bool, optional) >> >> Examples: >> >> 1. Change a removable medium >> >> -> { "execute": "change", >> "arguments": { "device": "ide1-cd0", >> "target": "/srv/images/Fedora-12-x86_64-DVD.iso" } } >> <- { "return": {} } >> >> 2. Change a disk without media change notification >> >> -> { "execute": "change", >> "arguments": { "device": "virtio-blk0", >> "target": "/srv/images/vm_1.img", >> "notify": false } } >> >> 3. Change VNC password >> >> -> { "execute": "change", >> "arguments": { "device": "vnc", "target": "password", >> "arg": "foobar1" } } >> <- { "return": {} } I find it rather disturbing that a command like 'change' has made it into QMP... Anyway, I don't think this is really what we need. We have two switches to do. The first one happens before starting the copy: Creating the copy, with the source as its backing file, and switching to that. The monitor command to achieve this is snapshot_blkdev. The second switch is after the copy has completed. At this point you can remove the source as the backing file and use the common base image instead. This is a call to bdrv_change_backing_file(), for which a monitor command doesn't exist yet (and unless we want to overload 'change' even more, it's not the right command to do this). Kevin