From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E751C47083 for ; Wed, 2 Jun 2021 13:39:29 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC9B96135D for ; Wed, 2 Jun 2021 13:39:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC9B96135D Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=virtuozzo.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48764 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1loR5j-0003v5-NF for qemu-devel@archiver.kernel.org; Wed, 02 Jun 2021 09:39:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34376) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1loQgC-000323-64; Wed, 02 Jun 2021 09:13:04 -0400 Received: from mail-eopbgr130108.outbound.protection.outlook.com ([40.107.13.108]:50294 helo=EUR01-HE1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1loQg8-0003nQ-8Q; Wed, 02 Jun 2021 09:13:03 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m8h9dX+DEuFW8p0kKHZt/z++7l4LBbRp+LHvaivKpYfsT9hxf/11w5YofwXvCVqEUTuvjvwC6zjmh4wpMkWYFtccgd3iPpjVNpVmN8Sk2T9CXAF5Ft8l3YzuBb4VO7PqSEIH+9gSEvUp4860ADNFixNwYLdW+qjQIi704vj/cNQHpmU1NkWlDFEVE/t4K9V1H6UwlD0Orqn4di3cUXcIQ9N10kJ3Z9xqdAf5wh4MzQihqgbbk6n/abAnldjsmtNeVz6xXRrsQxVKeD2ZzB4WLdtood4eZ2+KKV70JBfDfqtnF0tbFU4DfGqH/8YuJVPMvEZDD/9TNeQUCUKGK51GcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A5nciexShtPATulwjI+NVK3OnuPoB2K9ZiF+peWiDmg=; b=nE6quAr7VvRpgCiBP2yDbnHJpQCDvW1og7MuzWTnsD5Rbxmnv0TW+3hzyUsKN8y79s8WzrTRW+D1gB2gDy7EK243cDWj8udt2UKIbDDaRIvjNBFFRZ7KkxBIDikR9crM+5mJnNqP5dOGIL7QPh7C+ONgSaPq27WgY9VLnNl9xlKusOPpPhzfy2YnzsXCGtXJXc6vE3KCYWl+HSivQKqukdNkw8mMtwQrzcWUdo9EAyEeCkqpQWo/1WT39w3uhW7QgMvyEY+a/qlev+Q1l9Upax+1Dda7M4hu6XL2tq2GH4zQHd8gwPxt1idhvcRvs9kHqedT9hQOYZP8MGEyBuo8FA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A5nciexShtPATulwjI+NVK3OnuPoB2K9ZiF+peWiDmg=; b=G/1R/xhzz1tOQPJw8chvalo8ef1rGWUNTpq5ND3L34kK1HHGjlZmM2ju8eSn7zfiXk3ZD+Z02HvzOB0vbwMJAfYCXSRRJTnmQe66ThrNfujKZIXuAS2a/9ljRCvtm3XgyNrEOWJGDeBKzYDq4TFrlpK4Mgr5QCAEBKUPHKwOUYA= Authentication-Results: nongnu.org; dkim=none (message not signed) header.d=none;nongnu.org; dmarc=none action=none header.from=virtuozzo.com; Received: from AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) by AM6PR08MB4536.eurprd08.prod.outlook.com (2603:10a6:20b:bf::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.20; Wed, 2 Jun 2021 13:12:40 +0000 Received: from AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f928:f4f2:77c0:74b4]) by AM7PR08MB5494.eurprd08.prod.outlook.com ([fe80::f928:f4f2:77c0:74b4%7]) with mapi id 15.20.4173.030; Wed, 2 Jun 2021 13:12:40 +0000 From: Vladimir Sementsov-Ogievskiy To: qemu-block@nongnu.org Cc: qemu-devel@nongnu.org, eblake@redhat.com, armbru@redhat.com, crosa@redhat.com, ehabkost@redhat.com, berrange@redhat.com, pbonzini@redhat.com, vsementsov@virtuozzo.com, jsnow@redhat.com, mreitz@redhat.com, kwolf@redhat.com Subject: [PATCH v4 35/35] iotests/image-fleecing: add test-case for copy-before-write filter Date: Wed, 2 Jun 2021 16:11:08 +0300 Message-Id: <20210602131108.74979-36-vsementsov@virtuozzo.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210602131108.74979-1-vsementsov@virtuozzo.com> References: <20210602131108.74979-1-vsementsov@virtuozzo.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Originating-IP: [185.215.60.211] X-ClientProxiedBy: HE1PR0501CA0011.eurprd05.prod.outlook.com (2603:10a6:3:1a::21) To AM7PR08MB5494.eurprd08.prod.outlook.com (2603:10a6:20b:dc::15) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (185.215.60.211) by HE1PR0501CA0011.eurprd05.prod.outlook.com (2603:10a6:3:1a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4195.15 via Frontend Transport; Wed, 2 Jun 2021 13:12:13 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 894be773-83bf-4ba1-5852-08d925c80a79 X-MS-TrafficTypeDiagnostic: AM6PR08MB4536: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:635; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ksBdXzueE6VoI4pzb2OOoAD3x45QX8BVRfpr7FL5iP2ii3v+vB0jFUVmAaEWzUNww81iPq4kQNvLiLF+vo2FzukNAeIztfg5FIRBsoAhSDRsRKVaeN7i5/aotamXCrGFfMt8xgN+3VKKgfuKCrBHqfTxX2WPQFw22DGbyXzXitju0WiqyaLo4JezBiAkpb0QvrCvCzmB7yYQbsKdbuo1Oze0FfX7RK2kzBraJQEEVX4l8olfYoppEfuRFZex5tL0NE7+aQAoZH/WNVaTLUwgJQZq9BbBkbiO9aQu347ljBX8HUREHhB7dRoUBbtfOtMky4BfUCZcEQ7wW80oSoLSbuqQuuvIKmoxha/frPpaA0b4vkXN/Grv0bgjZcdYcRxZvlVRA9bNdH3rtncNqhRWRMpFSrmVw1yjoD8In1M7Q1gXQr31rsfibuIdYvcIwNuK4WxKvBY/z9/RDEXCyxNMIpjNe+kjLCUuMJYhRkuHjY8JwUfAWwvtCkV+/mmvkZ8DJ/iH+cY01rlBsLvfsg7npzisRHIjO/JsSJbtRoEGZ/ntoMOsLVyaLGOOupdCGvDeX0/so26NbQOrcFTJjYvcNs3PdmWicnA6P/m2kpPVTQaRWB25uyCCF7OKycn8Dj/UxqkdnHn2WGbIyeeDvmLc6e8ZXUs5w10EtnXD0SWrmKBL+k2vF3MHlGsaXm3Oix03 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR08MB5494.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(366004)(136003)(39840400004)(2906002)(36756003)(316002)(83380400001)(5660300002)(66556008)(1076003)(66476007)(6512007)(66946007)(52116002)(7416002)(956004)(8676002)(186003)(6916009)(6486002)(16526019)(6506007)(8936002)(38100700002)(38350700002)(6666004)(2616005)(86362001)(478600001)(4326008)(26005)(69590400013); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: =?us-ascii?Q?dWRZtm93OCq2Jy4q7JQ8HA1lnEAfZFZE9wmtqAVp+SY18+Dv/SpILvsxp/PQ?= =?us-ascii?Q?xCsOAs3+vWy1HdJ/EELba2T0Pp83B+yUkMKwn4/R5GledshO6eecm29uQzLs?= =?us-ascii?Q?Kk6g6iTW7Gqz3MBin6QzY0CHec5goktWoNRk9/qB7YgR5Y2iXf+s5GBtn1dY?= =?us-ascii?Q?RVqRy5ArjY25o8SkA/BKl9NzpcdtXU9dPIFdjYLd0f3K+nlHIaRpbhHbirZP?= =?us-ascii?Q?niLx7Uo3ybVwPq29CI2cnZ9DlE/PjMAmSLsLSSV1Qp4phj04kjUkrKl6NQxl?= =?us-ascii?Q?/1KDZfn3MGnT5hEppYCrIoCLVwBdvLf8E5Gq4cNHyRmIFMzDOGDTkfS87+JU?= =?us-ascii?Q?1kJ7Ue7XYwSGF3c3IzItTkAxCpBIoWWCjqbd33ByegqVnA7aG9a3U6TAmbgH?= =?us-ascii?Q?wS0tOCu8xtz0Yw0L0eesdi7AsmM/8RBHPQmYMm6rFvWEbsiW7UN/BInbn+5/?= =?us-ascii?Q?rQCt4joTESm7djaCgrXY+TSfS8ENOWpyBXksPMC7PUcQ6QYfEcRzFG7DPG/8?= =?us-ascii?Q?tqI9NXOLQzpSQ0+h/wni2t3nRQUCqohfx/U7B8PFokdp2wWOC/OaQyscZspM?= =?us-ascii?Q?8xAaS4fl0VwzLljSLMdB3azvhfQNWe39UIT4ZSeQmZPhOE+OhaCGlMHU8PBj?= =?us-ascii?Q?nJmqt7wEZk1qzIXIY0jqIvBYzQbqQhF6II1GL7mSrKy2RMxiIpF+Ij9D94Qc?= =?us-ascii?Q?e1avY8M5c1S4L6Vf8zU7hi6SoPHmtF7QTpUoBOLroXFtRHGL8mL/o13r+P7r?= =?us-ascii?Q?N+g+WrwcAaXTPHpqPqtYQWfh5cD81KeXpFU92hIc31JFlz1kBLtUwKyyxbda?= =?us-ascii?Q?StWAGMYPlTsnLj/ArntOLXmC3PrUJrTx7/GIvuEq1duXjG5i3ERzYNDjQdVE?= =?us-ascii?Q?Ahj+BiDNqs8iTnWdyehy2xNQOQt3FF9wdmvcGHrIaY9re4pRcpjCJtlD4dMa?= =?us-ascii?Q?iwlNS9Wpf39vSk7XMw3e9EfgoJEi31q9xDiAPD/v1GZgk4l1QTEbuMt/kQue?= =?us-ascii?Q?XFMcGdSQc6AdwGWsscOydlT38k52toxZPXg3CmZd916bi49LgXc7Kblchysm?= =?us-ascii?Q?OmGjxT7dk8VLGFK6sbs3xIVEYrydxdjiXYQwvEMIG75mSptv/ryJrdDBs53V?= =?us-ascii?Q?mRP62bxvHyXRB+pI1TSBHaMAToutTRIQaVMsIHWSOdpe9IZqgi8lVGrFQSbr?= =?us-ascii?Q?LWBlWl6xGs3IohHNy/Tfv2WsttTgIaD7aJxi7f9qkLIJFa99Zxl+LLvyfQ+D?= =?us-ascii?Q?rwF4ePLPlUaBrCCPpynIPBN/Pm8VFvH2ZAALUj/OzhwbDcbjT8yEt/qLzHt4?= =?us-ascii?Q?+z06ho/yC86CzNyhznLXlLln?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 894be773-83bf-4ba1-5852-08d925c80a79 X-MS-Exchange-CrossTenant-AuthSource: AM7PR08MB5494.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jun 2021 13:12:15.0371 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: iFxhe2bQ94lkOPkD0/tdCdq4BuMR5dEvoxcdTmdp6BWr2B2oeY9E/KX8Wgr+QbkNfRwJOU83nErkUkqtFCkL3Tgg1G4ZIPHpQAZNF6LZxjY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR08MB4536 Received-SPF: pass client-ip=40.107.13.108; envelope-from=vsementsov@virtuozzo.com; helo=EUR01-HE1-obe.outbound.protection.outlook.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" New fleecing method becomes available: copy-before-write filter. Actually we don't need backup job to setup image fleecing. Add test for new recommended way of image fleecing. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/tests/image-fleecing | 50 +++++++++----- tests/qemu-iotests/tests/image-fleecing.out | 72 +++++++++++++++++++++ 2 files changed, 107 insertions(+), 15 deletions(-) diff --git a/tests/qemu-iotests/tests/image-fleecing b/tests/qemu-iotests/tests/image-fleecing index e210c00d28..f6318492c6 100755 --- a/tests/qemu-iotests/tests/image-fleecing +++ b/tests/qemu-iotests/tests/image-fleecing @@ -48,7 +48,7 @@ remainder = [('0xd5', '0x108000', '32k'), # Right-end of partial-left [1] ('0xdc', '32M', '32k'), # Left-end of partial-right [2] ('0xcd', '0x3ff0000', '64k')] # patterns[3] -def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): +def do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm): log('--- Setting up images ---') log('') @@ -67,6 +67,7 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): src_node = 'source' tmp_node = 'temp' + qom_path = '/machine/peripheral/sda' vm.add_blockdev(f'driver={iotests.imgfmt},file.driver=file,' f'file.filename={base_img_path},node-name={src_node}') vm.add_device('virtio-scsi') @@ -90,12 +91,22 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): 'backing': src_node, })) - # Establish COW from source to fleecing node - log(vm.qmp('blockdev-backup', - job_id='fleecing', - device=src_node, - target=tmp_node, - sync='none')) + # Establish CBW from source to fleecing node + if use_cbw: + log(vm.qmp('blockdev-add', { + 'driver': 'copy-before-write', + 'node-name': 'fl-cbw', + 'file': src_node, + 'target': tmp_node + })) + + log(vm.qmp('qom-set', path=qom_path, property='drive', value='fl-cbw')) + else: + log(vm.qmp('blockdev-backup', + job_id='fleecing', + device=src_node, + target=tmp_node, + sync='none')) log('') log('--- Setting up NBD Export ---') @@ -124,7 +135,7 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): for p in overwrite: cmd = 'write -P%s %s %s' % p log(cmd) - log(vm.hmp_qemu_io('/machine/peripheral/sda', cmd, qdev=True)) + log(vm.hmp_qemu_io(qom_path, cmd, qdev=True)) log('') log('--- Verifying Data ---') @@ -139,10 +150,15 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): log('--- Cleanup ---') log('') - log(vm.qmp('block-job-cancel', device='fleecing')) - e = vm.event_wait('BLOCK_JOB_CANCELLED') - assert e is not None - log(e, filters=[iotests.filter_qmp_event]) + if use_cbw: + log(vm.qmp('qom-set', path=qom_path, property='drive', value=src_node)) + log(vm.qmp('blockdev-del', node_name='fl-cbw')) + else: + log(vm.qmp('block-job-cancel', device='fleecing')) + e = vm.event_wait('BLOCK_JOB_CANCELLED') + assert e is not None + log(e, filters=[iotests.filter_qmp_event]) + log(vm.qmp('nbd-server-stop')) log(vm.qmp('blockdev-del', node_name=tmp_node)) vm.shutdown() @@ -160,13 +176,17 @@ def do_test(base_img_path, fleece_img_path, nbd_sock_path, vm): log('Done') -def test(): +def test(use_cbw): with iotests.FilePath('base.img') as base_img_path, \ iotests.FilePath('fleece.img') as fleece_img_path, \ iotests.FilePath('nbd.sock', base_dir=iotests.sock_dir) as nbd_sock_path, \ iotests.VM() as vm: - do_test(base_img_path, fleece_img_path, nbd_sock_path, vm) + do_test(use_cbw, base_img_path, fleece_img_path, nbd_sock_path, vm) + +log('=== Test backup(sync=none) based fleecing ===\n') +test(False) -test() +log('=== Test filter based fleecing ===\n') +test(True) diff --git a/tests/qemu-iotests/tests/image-fleecing.out b/tests/qemu-iotests/tests/image-fleecing.out index 314a1b54e9..e96d122a8b 100644 --- a/tests/qemu-iotests/tests/image-fleecing.out +++ b/tests/qemu-iotests/tests/image-fleecing.out @@ -1,3 +1,5 @@ +=== Test backup(sync=none) based fleecing === + --- Setting up images --- Done @@ -65,3 +67,73 @@ read -P0xdc 32M 32k read -P0xcd 0x3ff0000 64k Done +=== Test filter based fleecing === + +--- Setting up images --- + +Done + +--- Launching VM --- + +Done + +--- Setting up Fleecing Graph --- + +{"return": {}} +{"return": {}} +{"return": {}} + +--- Setting up NBD Export --- + +{"return": {}} +{"return": {}} + +--- Sanity Check --- + +read -P0x5d 0 64k +read -P0xd5 1M 64k +read -P0xdc 32M 64k +read -P0xcd 0x3ff0000 64k +read -P0 0x00f8000 32k +read -P0 0x2010000 32k +read -P0 0x3fe0000 64k + +--- Testing COW --- + +write -P0xab 0 64k +{"return": ""} +write -P0xad 0x00f8000 64k +{"return": ""} +write -P0x1d 0x2008000 64k +{"return": ""} +write -P0xea 0x3fe0000 64k +{"return": ""} + +--- Verifying Data --- + +read -P0x5d 0 64k +read -P0xd5 1M 64k +read -P0xdc 32M 64k +read -P0xcd 0x3ff0000 64k +read -P0 0x00f8000 32k +read -P0 0x2010000 32k +read -P0 0x3fe0000 64k + +--- Cleanup --- + +{"return": {}} +{"return": {}} +{"return": {}} +{"return": {}} + +--- Confirming writes --- + +read -P0xab 0 64k +read -P0xad 0x00f8000 64k +read -P0x1d 0x2008000 64k +read -P0xea 0x3fe0000 64k +read -P0xd5 0x108000 32k +read -P0xdc 32M 32k +read -P0xcd 0x3ff0000 64k + +Done -- 2.29.2