From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B252DC433ED for ; Mon, 5 Apr 2021 04:27:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14FA261397 for ; Mon, 5 Apr 2021 04:27:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14FA261397 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=bugs.launchpad.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:33200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lTGpd-0003hQ-8r for qemu-devel@archiver.kernel.org; Mon, 05 Apr 2021 00:27:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36842) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lTGoH-0002ow-Po for qemu-devel@nongnu.org; Mon, 05 Apr 2021 00:25:57 -0400 Received: from indium.canonical.com ([91.189.90.7]:48808) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lTGoE-0008J1-FE for qemu-devel@nongnu.org; Mon, 05 Apr 2021 00:25:57 -0400 Received: from loganberry.canonical.com ([91.189.90.37]) by indium.canonical.com with esmtp (Exim 4.86_2 #2 (Debian)) id 1lTGoC-00026t-AY for ; Mon, 05 Apr 2021 04:25:52 +0000 Received: from loganberry.canonical.com (localhost [127.0.0.1]) by loganberry.canonical.com (Postfix) with ESMTP id 1F4BF2E8171 for ; Mon, 5 Apr 2021 04:25:52 +0000 (UTC) MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Date: Mon, 05 Apr 2021 04:17:20 -0000 From: Launchpad Bug Tracker <1810603@bugs.launchpad.net> To: qemu-devel@nongnu.org X-Launchpad-Notification-Type: bug X-Launchpad-Bug: product=qemu; status=Expired; importance=Undecided; assignee=None; X-Launchpad-Bug-Information-Type: Public X-Launchpad-Bug-Private: no X-Launchpad-Bug-Security-Vulnerability: no X-Launchpad-Bug-Commenters: eblake janitor kwolf-redhat lennyhers th-huth X-Launchpad-Bug-Reporter: Lenny Helpline (lennyhers) X-Launchpad-Bug-Modifier: Launchpad Janitor (janitor) References: <154670237006.5111.1505843603378488710.malonedeb@wampee.canonical.com> Message-Id: <161759624076.31194.10108426097403882012.malone@loganberry.canonical.com> Subject: [Bug 1810603] Re: QEMU QCow Images grow dramatically X-Launchpad-Message-Rationale: Subscriber (QEMU) @qemu-devel-ml X-Launchpad-Message-For: qemu-devel-ml Precedence: bulk X-Generated-By: Launchpad (canonical.com); Revision="57f1f603f707b9cfa764cae8dd0f3999026b4763"; Instance="production" X-Launchpad-Hash: 7e01515b894b31f033c60dfa70a2c406cb2672ee Received-SPF: none client-ip=91.189.90.7; envelope-from=bounces@canonical.com; helo=indium.canonical.com X-Spam_score_int: -66 X-Spam_score: -6.7 X-Spam_bar: ------ X-Spam_report: (-6.7 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Bug 1810603 <1810603@bugs.launchpad.net> Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" [Expired for QEMU because there has been no activity for 60 days.] ** Changed in: qemu Status: Incomplete =3D> Expired -- = You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1810603 Title: QEMU QCow Images grow dramatically Status in QEMU: Expired Bug description: I've recently migrated our VM infrastructure (~200 guest on 15 hosts) from vbox to Qemu (using KVM / libvirt). We have a master image (QEMU QCow v3) from which we spawn multiple instances (linked clones). All guests are being revert once per hour for security reasons. About 2 weeks after we successfully migrated to Qemu, we noticed that almost all disks went full across all 15 hosts. Our investigation showed that the initial qcow disk images blow up from a few gigabytes to 100GB and more. This should not happen, as we revert all VMs back to the initial snapshot once per hour and hence all changes that have been made to disks must be reverted too. We did an addition test with 24 hour time frame with which we could reproduce this bug as documented below. Initial disk image size (created on Jan 04): -rw-r--r-- 1 root root 7.1G Jan 4 15:59 W10-TS01-0.img -rw-r--r-- 1 root root 7.3G Jan 4 15:59 W10-TS02-0.img -rw-r--r-- 1 root root 7.4G Jan 4 15:59 W10-TS03-0.img -rw-r--r-- 1 root root 8.3G Jan 4 16:02 W10-CLIENT01-0.img -rw-r--r-- 1 root root 8.6G Jan 4 16:05 W10-CLIENT02-0.img -rw-r--r-- 1 root root 8.0G Jan 4 16:05 W10-CLIENT03-0.img -rw-r--r-- 1 root root 8.3G Jan 4 16:08 W10-CLIENT04-0.img -rw-r--r-- 1 root root 8.1G Jan 4 16:12 W10-CLIENT05-0.img -rw-r--r-- 1 root root 8.0G Jan 4 16:12 W10-CLIENT06-0.img -rw-r--r-- 1 root root 8.1G Jan 4 16:16 W10-CLIENT07-0.img -rw-r--r-- 1 root root 7.6G Jan 4 16:16 W10-CLIENT08-0.img -rw-r--r-- 1 root root 7.6G Jan 4 16:19 W10-CLIENT09-0.img -rw-r--r-- 1 root root 7.5G Jan 4 16:21 W10-ROUTER-0.img -rw-r--r-- 1 root root 18G Jan 4 16:25 W10-MASTER-IMG.qcow2 Disk image size after 24 hours (printed on Jan 05): -rw-r--r-- 1 root root 13G Jan 5 15:07 W10-TS01-0.img -rw-r--r-- 1 root root 8.9G Jan 5 14:20 W10-TS02-0.img -rw-r--r-- 1 root root 9.0G Jan 5 15:07 W10-TS03-0.img -rw-r--r-- 1 root root 10G Jan 5 15:08 W10-CLIENT01-0.img -rw-r--r-- 1 root root 11G Jan 5 15:08 W10-CLIENT02-0.img -rw-r--r-- 1 root root 11G Jan 5 15:08 W10-CLIENT03-0.img -rw-r--r-- 1 root root 11G Jan 5 15:08 W10-CLIENT04-0.img -rw-r--r-- 1 root root 19G Jan 5 15:07 W10-CLIENT05-0.img -rw-r--r-- 1 root root 14G Jan 5 15:08 W10-CLIENT06-0.img -rw-r--r-- 1 root root 9.7G Jan 5 15:07 W10-CLIENT07-0.img -rw-r--r-- 1 root root 35G Jan 5 15:08 W10-CLIENT08-0.img -rw-r--r-- 1 root root 9.2G Jan 5 15:07 W10-CLIENT09-0.img -rw-r--r-- 1 root root 41G Jan 5 15:08 W10-ROUTER-0.img -rw-r--r-- 1 root root 18G Jan 4 16:25 W10-MASTER-IMG.qcow2 You can reproduce this bug as follow: 1) create an initial disk image 2) create a linked clone 3) create a snapshot of the linked clone 4) revert the snapshot every X minutes / hours Due the described behavior / bug, our VM farm is completely down at the moment (as we run out of disk space on all host systems). A quick fix for this bug would be much appreciated. Host OS: Ubuntu 18.04.01 LTS Kernel: 4.15.0-43-generic Qemu: 3.1.0 libvirt: 4.10.0 Guest OS: Windows 10 64bit To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1810603/+subscriptions