From: "Richard W.M. Jones" <rjones@redhat.com>
To: Eric Blake <eblake@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>,
"qemu-block@nongnu.org" <qemu-block@nongnu.org>,
"qemu-devel@nongnu.org" <qemu-devel@nongnu.org>,
"mreitz@redhat.com" <mreitz@redhat.com>,
"Wang, Wei W" <wei.w.wang@intel.com>,
"lersek@redhat.com" <lersek@redhat.com>
Subject: Re: Question on Compression for Raw Image
Date: Tue, 20 Oct 2020 15:42:17 +0100 [thread overview]
Message-ID: <20201020144217.GT30079@redhat.com> (raw)
In-Reply-To: <ec72d814-109c-4ed9-cb26-32b9d903f8c2@redhat.com>
On Tue, Oct 20, 2020 at 09:32:23AM -0500, Eric Blake wrote:
> On 10/20/20 9:22 AM, Wang, Wei W wrote:
> > On Tuesday, October 20, 2020 4:01 PM, Kevin Wolf wrote:
> >> Am 20.10.2020 um 03:31 hat Wang, Wei W geschrieben:
> >>> Hi,
> >>>
> >>> Does anyone know the reason why raw-format.c doesn't have
> >> compression
> >>> support (but qcow has the supported added)? For example, raw image
> >>> backup with compression, "qemu-img convert -c -O raw origin.img
> >>> dist.img", doesn't work.
> >>
> >> A raw image is by definition a file that contains the exact same sequence of
> >> bytes as the guest sees, without any additional information or encoding. If
> >> you compress a raw file, the guest will see compressed data on its hard disk
> >> instead of the real data.
> >
> > Ok, thanks. I'm thinking QEMU could do decompression of the compressed data in raw.img when guest reads data.
> >
> >>
> >> Anything you could do to add transparent compression to it would mean that
> >> it's not a raw image any more, but a new image format.
> >>
> > Yes, decompression makes it transparent to the guest. Would you think it's good to reuse the raw image implementation, just add the compress/decompress option?
>
> My recommendation would be implementing a new BDS filter that does
> uncompression. Then, you could do things like:
>
> raw -> decompress -> file.xz
>
> or even
>
> qcow2 -> decompress -> file.qcow2.xz
>
> By the way, the notion of filters is already possible in other ways.
> For example, you can point qemu to read from an NBD server, and then use
> nbdkit to do the decompression with its filters:
>
> nbdkit --filter=xz file file.xz
> raw -> nbd://localhost:10809
nbdkit can already act as a front-end to qemu-nbd, allowing you to use
the rich nbdkit filter ecosystem with qemu block devices. There are
some examples in the link below, but --filter=xz could also be used
here:
https://libguestfs.org/nbdkit-nbd-plugin.1.html#Use-qemu-nbd-to-open-a-qcow2-file
> Also note that serving a decompressed view of a compressed image tends
> to be a read-only proposition (you really CAN'T write to the image
> without recompressing, but even if recompression has been blocked for
> parallelism, you would end up writing far more of the file after
> recompression than the amount of data written by a guest).
Yup!
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v
next prev parent reply other threads:[~2020-10-20 14:45 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-20 1:31 Question on Compression for Raw Image Wang, Wei W
2020-10-20 8:00 ` Kevin Wolf
2020-10-20 14:22 ` Wang, Wei W
2020-10-20 14:32 ` Eric Blake
2020-10-20 14:42 ` Richard W.M. Jones [this message]
2020-10-20 14:49 ` Alberto Garcia
2020-10-20 14:36 ` Alberto Garcia
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201020144217.GT30079@redhat.com \
--to=rjones@redhat.com \
--cc=eblake@redhat.com \
--cc=kwolf@redhat.com \
--cc=lersek@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=wei.w.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).