From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:41353) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c01Id-00011K-7d for qemu-devel@nongnu.org; Fri, 28 Oct 2016 03:10:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c01IY-0000AR-VZ for qemu-devel@nongnu.org; Fri, 28 Oct 2016 03:09:59 -0400 From: Alberto Garcia Date: Fri, 28 Oct 2016 10:08:01 +0300 Message-Id: Subject: [Qemu-devel] [PATCH v13 00/19] Support streaming to an intermediate layer List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: qemu-block@nongnu.org, Kevin Wolf , Eric Blake , Markus Armbruster , Max Reitz , Alberto Garcia Hi, thank you Kevin and Eric for your comments. I addressed them all in this series. Berto v13: - Patch 10: explain that the 'base-node' parameter can be used to detect whether we support streaming to an intermediate node. - Patch 12: Revert the blkdebug removal, it is indeed necessary to keep it. - Patch 15: Fix the block-commit call in test_stream_commit(). - Patch 19: Add a new test to check that 'base' and 'base-node' cannot be specified at the same time. v12: https://lists.gnu.org/archive/html/qemu-block/2016-10/msg01085.html - Rebase on top of the current master. - Patch 1: Add a note explaining that no new block jobs or BDSs can be added between bdrv_drain_all_begin/end(). v11: https://lists.gnu.org/archive/html/qemu-block/2016-10/msg00518.html - Patches 1-2: Add bdrv_drain_all_{begin,end}() and use these functions in bdrv_reopen_multiple() instead of pausing all block jobs. - Patch 3: Don't unblock BLOCK_OP_TYPE_DATAPLANE in block_job_add_bdrv() - Patch 7: In commit_start(), don't call block_job_add_bdrv() twice on the active node. - Patch 10: In qmp_block_stream(), don't check twice for op blockers in 'bs'. - Patch 11: Clarify that live-block-ops.txt is incomplete and it only covers the block-stream operation. - Patches 18-19: Add a 'base-node' parameter to 'block-stream', and iotests for it. v10: https://lists.gnu.org/archive/html/qemu-devel/2016-10/msg01047.html - Rebase the code. - Pause block jobs during bdrv_reopen(). - Allow jobs in parallel again, and modify jobs to block all involved nodes, not just the root one. - Add more tests. v9: https://lists.gnu.org/archive/html/qemu-devel/2016-04/msg00411.html - Rebase the code - Block the active layer in order to forbid other block jobs in the same chain. - Split the patch that adds block_job_next() into 4 (new patches 1-4). - Replace the test that performs several block-stream operations in parallel with one that check that they're forbidden. - Remove patches that have already been merged. v8: https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg05754.html - Rebased on top of Stefan's block branch (0a35bce416) - The loop that pauses the block jobs in bdrv_drain_all() is now split in two: one that iterates the list of block jobs to stop them, and one that iterates the root bds in order to get the aio contexts. v7: https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg02580.html - Rebased against the current master - Updated bdrv_drain_all() to use the new block_job_next() API. v6: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg03046.html - fix the no-op test following Max's suggestions v5: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg03006.html - Fix a few typos - Minor documentation updates - Update test_stream_partial() to test no-ops - New test case: test_stream_parallel() - New test case: test_stream_overlapping() v4: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg01878.html - Refactor find_block_job to use the error from bdrv_lookup_bs() - Don't use QERR_DEVICE_IN_USE in block_job_create() since we can be dealing with nodes now. - Fix @device comment in the BlockJobInfo documentation - stream_start(): simplify the bdrv_reopen() call and use bdrv_get_device_or_node_name() for error messages. - Use a different variable name for BlockDriverState *i - Documentation fixes in docs/live-block-ops.txt - Update iotest 30 since now test_device_not_found() returns GenericError - Fix test case test_stream_partial() - Add new test case test_stream_intermediate() - Fix typos v3: https://lists.gnu.org/archive/html/qemu-devel/2015-04/msg00806.html - Keep a list of block jobs and make qmp_query_block_jobs() iterate over it. v2: https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg04798.html - The 'block-stream' command does not have a 'node-name' parameter anymore and reuses 'device' for that purpose. - Block jobs can now be owned by any intermediate node, and not just by the ones at the root. query-block-jobs is updated to reflect that change. - The 'device' parameter of all 'block-job-*' commands can now take a node name. - The BlockJobInfo type and all BLOCK_JOB_* events report the node name in the 'device' field if the node does not have a device name. - All intermediate nodes are blocked (and checked for blockers) during the streaming operation. v1: https://lists.gnu.org/archive/html/qemu-devel/2015-02/msg04116.html backport-diff against v12: Key: [----] : patches are identical [####] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/19:[----] [--] 'block: Add bdrv_drain_all_{begin,end}()' 002/19:[----] [--] 'block: Pause all jobs during bdrv_reopen_multiple()' 003/19:[----] [--] 'block: Add block_job_add_bdrv()' 004/19:[----] [--] 'block: Use block_job_add_bdrv() in mirror_start_job()' 005/19:[----] [--] 'block: Use block_job_add_bdrv() in backup_start()' 006/19:[----] [--] 'block: Check blockers in all nodes involved in a block-commit job' 007/19:[----] [--] 'block: Block all nodes involved in the block-commit operation' 008/19:[----] [--] 'block: Block all intermediate nodes in commit_active_start()' 009/19:[----] [--] 'block: Support streaming to an intermediate layer' 010/19:[0005] [FC] 'block: Add QMP support for streaming to an intermediate layer' 011/19:[----] [--] 'docs: Document how to stream to an intermediate layer' 012/19:[0002] [FC] 'qemu-iotests: Test streaming to an intermediate layer' 013/19:[----] [--] 'qemu-iotests: Test block-stream operations in parallel' 014/19:[----] [--] 'qemu-iotests: Test overlapping stream and commit operations' 015/19:[0002] [FC] 'qemu-iotests: Test block-stream and block-commit in parallel' 016/19:[----] [--] 'qemu-iotests: Add iotests.supports_quorum()' 017/19:[----] [--] 'qemu-iotests: Test streaming to a Quorum child' 018/19:[----] [--] 'block: Add 'base-node' parameter to the 'block-stream' command' 019/19:[0004] [FC] 'qemu-iotests: Test the 'base-node' parameter of 'block-stream'' Alberto Garcia (19): block: Add bdrv_drain_all_{begin,end}() block: Pause all jobs during bdrv_reopen_multiple() block: Add block_job_add_bdrv() block: Use block_job_add_bdrv() in mirror_start_job() block: Use block_job_add_bdrv() in backup_start() block: Check blockers in all nodes involved in a block-commit job block: Block all nodes involved in the block-commit operation block: Block all intermediate nodes in commit_active_start() block: Support streaming to an intermediate layer block: Add QMP support for streaming to an intermediate layer docs: Document how to stream to an intermediate layer qemu-iotests: Test streaming to an intermediate layer qemu-iotests: Test block-stream operations in parallel qemu-iotests: Test overlapping stream and commit operations qemu-iotests: Test block-stream and block-commit in parallel qemu-iotests: Add iotests.supports_quorum() qemu-iotests: Test streaming to a Quorum child block: Add 'base-node' parameter to the 'block-stream' command qemu-iotests: Test the 'base-node' parameter of 'block-stream' block.c | 9 +- block/backup.c | 5 +- block/commit.c | 14 ++ block/io.c | 27 +++- block/mirror.c | 11 +- block/stream.c | 24 ++++ blockdev.c | 41 +++++- blockjob.c | 17 ++- docs/live-block-ops.txt | 36 +++-- docs/qmp-commands.txt | 7 +- hmp.c | 2 +- include/block/block.h | 2 + include/block/blockjob.h | 14 ++ qapi/block-core.json | 21 ++- tests/qemu-iotests/030 | 313 +++++++++++++++++++++++++++++++++++++++++- tests/qemu-iotests/030.out | 4 +- tests/qemu-iotests/041 | 27 ++-- tests/qemu-iotests/139 | 3 +- tests/qemu-iotests/iotests.py | 5 +- 19 files changed, 525 insertions(+), 57 deletions(-) -- 2.9.3