From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50966) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZljHE-00063I-74 for qemu-devel@nongnu.org; Mon, 12 Oct 2015 16:00:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZljHB-0008Uu-QE for qemu-devel@nongnu.org; Mon, 12 Oct 2015 16:00:56 -0400 From: Max Reitz Date: Mon, 12 Oct 2015 22:00:03 +0200 Message-Id: <1444680042-13207-1-git-send-email-mreitz@redhat.com> Subject: [Qemu-devel] [PATCH v6 00/39] blockdev: BlockBackend and media List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-block@nongnu.org Cc: Kevin Wolf , Alberto Garcia , Markus Armbruster , qemu-devel@nongnu.org, Max Reitz , John Snow , Stefan Hajnoczi *** This series is based on v3 of Kevin's *** *** "block: Get rid of bdrv_swap()" series *** This series reworks a lot regarding BlockBackend and media. Basically, it allows empty BlockBackends, that is BBs without a BDS tree. Before this series, empty drives are represented by a BlockBackend with an empty BDS attached to it (a BDS with a NULL driver). However, now we have BlockBackends, thus an empty drive should be represented by a BlockBackend without any BDS tree attached to it. This is what this series does. Quick and early summary for the v6 changes: - Rebase on master and Kevin's bdrv_swap() series - Addressed Kevin's comments for v5 Justification for each of the patches and their order: -- Preparation before _is_inserted() patches -- 1: Patch 9 will not take care not to break host floppy support, so that support needs to be removed first. 2: Needed for patch 3, so that blockdev-added BDSs without a BB still get the BDRV_O_INCOMING flag set. 3: Needed for patch 4. Patch 26 is a follow-up after BDS-less BBs are allowed. 4: bdrv_close_all() is broken ("block: Rework bdrv_close_all()"). Patch 7 will break iotest 071 (actually, just make the problem apparent). So this patch is required to work around the issue. (with "the issue" being that bdrv_close_all() does not unref() the BDSs it is closing, but just force-closes everything, even if the BDS may still be in use somewhere) -- _is_inserted() patches -- 5: General clean-up work, nice to have before patch 7 (and goes in tune with patch 6). 6: Using the same BB as a guest device means that the data read from there should be exactly the same. Opening the guest tray should therefore result in no data being readable. This is what we then need this function for. 7: General clean-up work (in the _is_inserted() area). 8: General clean-up work (in the _is_inserted() area). 9: General clean-up work (also regarding _is_inserted()). 10: Required so inserting a floppy will not result in the tray being reported as closed (you need to "push in" the floppy first, using blockdev-close-tray). It's here in the "_is_inserted() patches area" because I feel like that's a very related topic. -- Support for BDS-less BBs -- 11: Preparation for BDS-less BBs 12: Preparation for BDS-less BBs 13: Preparation for BDS-less BBs (BB properties should be in the BB, and not in the root BDS) 14: Patch 15 removes BlockAcctStats from the BDS, but wr_highest_sector is BDS-dependent, so it needs to stay here 15: Preparation for BDS-less BBs (BB properties should be in the BB, and not in the root BDS) 16: Preparation for BDS-less BBs (BB properties should be in the BB, and not in the root BDS) 17: Needed for patch 18 18: Preparation for BDS-less BBs (Removing a BDS tree should retain some properties for legacy reasons, which must therefore be stored in the BB(RS)) 19: Preparation for BDS-less BBs 20: Preparation for BDS-less BBs 21: Preparation for BDS-less BBs 22: Ability to add BDS trees to empty BBs ("inserting a medium") 23: Preparation for BDS-less BBs (needs patch 22) 24: One goal of this series, and fixes the "opening tray" event for empty drives when shutting down qemu 25: Needed for patch 26 26: Completion of what patch 3 begun 27: Ability to detach BDS trees from BBs -- "Atomic" QMP tray operations -- 28: blockdev-open-tray 29: blockdev-close-tray 30: blockdev-remove-medium 31: blockdev-insert-medium -- Reimplementation of change/eject -- 32: eject 33: change 34: Clean-up patch -- New QMP blockdev-change-medium command -- 35: New QMP command 36: Use for HMP change command 37: Add flag to that command for changing the read-only access mode (which was my original intention for this series) 38: Same flag for HMP -- Tests -- 39: iotests are always nice, so here is one v6: - Patch 8: Trivial rebase conflict due to Kevin's bdrv_swap() series - Patch 17: Added, so that the throttle group can be strongly referenced by the BBRS - Patch 18: - Keep a strong reference to the throttle group in the BBRS [Kevin]; this is done by keeping the name of the group (which the external interface of the throttle implementation generally uses for identification) and a pointer to the ThrottleState. Just storing the name is not enough, we actually do need the ThrottleState for patch 24. The ThrottleConfig is dropped since the ThrottleGroup's configuration should not be overwritten when a new BDS is added. - Patch 19: blk_{set_,}enable_write_cache() can fall back to the BBRS, too [Kevin] - Patch 21: Added blk_drain(), removed blk_{set_,}enable_write_cache() [Kevin] - Patch 22: blk_insert_bs() is no longer idempotent [Kevin, Eric] (i.e., inserting a BDS into its attached BB will now fail an assertion) - Patch 24: - Adopted to how the BBRS stores the ThrottleGroup reference now. An important point is that if a BDS tree is created, bdrv_set_io_limits() is called after bdrv_io_limits_enable(). Therefore, when creating an empty BB but assigning it to a throttle group, we have to change the group's configuration, too. This is why we need the ThrottleState pointer in the BBRS, so we can change a throttle group's configuration without knowing a BDS which is part of the group. - Patch 27: - blk_remove_bs() is no longer idempotent (i.e. removing a BDS from a BB without an attached BDS will now fail an assertion) - blk_insert_bs() using a BDS that is already attached to a BB will no longer automatically detach that BDS from its BB, but will now fail an assertion - Patch 28: s/qmp_marshal_input_/qmp_marshal_/ - Patch 29: s/qmp_marshal_input_/qmp_marshal_/ - Patch 30: s/qmp_marshal_input_/qmp_marshal_/ - Patch 31: - s/qmp_marshal_input_/qmp_marshal_/ - Following the changes to patch 27, trying to insert a BDS tree that already has a BB into another BB is now an error (silently detaching the old BB was a bad idea) - Patch 33: The new BDS is now just added to the throttle group thus setting its throttling configuration. bdrv_set_io_limits() is no longer called (this would reset the group's configuration). [Kevin] - Patch 35: s/qmp_marshal_input_/qmp_marshal_/ git-backport-diff against v5: Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/39:[----] [--] 'block: Remove host floppy support' 002/39:[----] [--] 'block: Set BDRV_O_INCOMING in bdrv_fill_options()' 003/39:[----] [--] 'blockdev: Allow creation of BDS trees without BB' 004/39:[----] [--] 'iotests: Only create BB if necessary' 005/39:[----] [-C] 'block: Make bdrv_is_inserted() return a bool' 006/39:[----] [--] 'block: Add blk_is_available()' 007/39:[----] [--] 'block: Make bdrv_is_inserted() recursive' 008/39:[0002] [FC] 'block/raw_bsd: Drop raw_is_inserted()' 009/39:[----] [-C] 'block: Invoke change media CB before NULLing drv' 010/39:[----] [--] 'hw/block/fdc: Implement tray status' 011/39:[----] [--] 'hw/usb-storage: Check whether BB is inserted' 012/39:[----] [--] 'block: Fix BB AIOCB AioContext without BDS' 013/39:[----] [-C] 'block: Move guest_block_size into BlockBackend' 014/39:[----] [-C] 'block: Remove wr_highest_sector from BlockAcctStats' 015/39:[----] [--] 'block: Move BlockAcctStats into BlockBackend' 016/39:[----] [-C] 'block: Move I/O status and error actions into BB' 017/39:[down] 'block/throttle-groups: Make incref/decref public' 018/39:[0029] [FC] 'block: Add BlockBackendRootState' 019/39:[0016] [FC] 'block: Make some BB functions fall back to BBRS' 020/39:[----] [--] 'block: Fail requests to empty BlockBackend' 021/39:[0010] [FC] 'block: Prepare remaining BB functions for NULL BDS' 022/39:[0007] [FC] 'block: Add blk_insert_bs()' 023/39:[----] [--] 'block: Prepare for NULL BDS' 024/39:[0006] [FC] 'blockdev: Do not create BDS for empty drive' 025/39:[----] [--] 'blockdev: Pull out blockdev option extraction' 026/39:[----] [--] 'blockdev: Allow more options for BB-less BDS tree' 027/39:[0010] [FC] 'block: Add blk_remove_bs()' 028/39:[0002] [FC] 'blockdev: Add blockdev-open-tray' 029/39:[0002] [FC] 'blockdev: Add blockdev-close-tray' 030/39:[0002] [FC] 'blockdev: Add blockdev-remove-medium' 031/39:[0008] [FC] 'blockdev: Add blockdev-insert-medium' 032/39:[----] [--] 'blockdev: Implement eject with basic operations' 033/39:[0005] [FC] 'blockdev: Implement change with basic operations' 034/39:[----] [--] 'block: Inquire tray state before tray-moved events' 035/39:[0002] [FC] 'qmp: Introduce blockdev-change-medium' 036/39:[----] [--] 'hmp: Use blockdev-change-medium for change command' 037/39:[----] [-C] 'blockdev: read-only-mode for blockdev-change-medium' 038/39:[----] [--] 'hmp: Add read-only-mode option to change command' 039/39:[----] [--] 'iotests: Add test for change-related QMP commands' Max Reitz (39): block: Remove host floppy support block: Set BDRV_O_INCOMING in bdrv_fill_options() blockdev: Allow creation of BDS trees without BB iotests: Only create BB if necessary block: Make bdrv_is_inserted() return a bool block: Add blk_is_available() block: Make bdrv_is_inserted() recursive block/raw_bsd: Drop raw_is_inserted() block: Invoke change media CB before NULLing drv hw/block/fdc: Implement tray status hw/usb-storage: Check whether BB is inserted block: Fix BB AIOCB AioContext without BDS block: Move guest_block_size into BlockBackend block: Remove wr_highest_sector from BlockAcctStats block: Move BlockAcctStats into BlockBackend block: Move I/O status and error actions into BB block/throttle-groups: Make incref/decref public block: Add BlockBackendRootState block: Make some BB functions fall back to BBRS block: Fail requests to empty BlockBackend block: Prepare remaining BB functions for NULL BDS block: Add blk_insert_bs() block: Prepare for NULL BDS blockdev: Do not create BDS for empty drive blockdev: Pull out blockdev option extraction blockdev: Allow more options for BB-less BDS tree block: Add blk_remove_bs() blockdev: Add blockdev-open-tray blockdev: Add blockdev-close-tray blockdev: Add blockdev-remove-medium blockdev: Add blockdev-insert-medium blockdev: Implement eject with basic operations blockdev: Implement change with basic operations block: Inquire tray state before tray-moved events qmp: Introduce blockdev-change-medium hmp: Use blockdev-change-medium for change command blockdev: read-only-mode for blockdev-change-medium hmp: Add read-only-mode option to change command iotests: Add test for change-related QMP commands block.c | 180 ++------ block/accounting.c | 8 - block/backup.c | 17 +- block/block-backend.c | 389 +++++++++++++++-- block/commit.c | 3 +- block/io.c | 10 +- block/mirror.c | 17 +- block/qapi.c | 36 +- block/raw-posix.c | 230 +--------- block/raw_bsd.c | 6 - block/stream.c | 3 +- block/throttle-groups.c | 19 +- blockdev.c | 921 +++++++++++++++++++++++++++++----------- blockjob.c | 5 +- hmp-commands.hx | 20 +- hmp.c | 47 +- hw/block/fdc.c | 20 +- hw/block/xen_disk.c | 4 +- hw/usb/dev-storage.c | 30 +- include/block/accounting.h | 3 - include/block/block.h | 16 +- include/block/block_int.h | 25 +- include/block/throttle-groups.h | 3 + include/qemu/typedefs.h | 1 + include/sysemu/block-backend.h | 14 +- include/sysemu/blockdev.h | 2 - migration/block.c | 5 + monitor.c | 4 + qapi-schema.json | 6 +- qapi/block-core.json | 138 +++++- qmp-commands.hx | 213 +++++++++- qmp.c | 9 +- tests/fdc-test.c | 4 +- tests/qemu-iotests/071 | 54 ++- tests/qemu-iotests/071.out | 12 +- tests/qemu-iotests/081 | 18 +- tests/qemu-iotests/081.out | 5 +- tests/qemu-iotests/087 | 2 +- tests/qemu-iotests/087.out | 4 +- tests/qemu-iotests/118 | 638 ++++++++++++++++++++++++++++ tests/qemu-iotests/118.out | 5 + tests/qemu-iotests/group | 1 + 42 files changed, 2313 insertions(+), 834 deletions(-) create mode 100755 tests/qemu-iotests/118 create mode 100644 tests/qemu-iotests/118.out -- 2.6.1