Linux-Block Archive on lore.kernel.org
 help / color / Atom feed
From: Manuel Bentele <manuel-bentele@web.de>
To: Ming Lei <tom.leiming@gmail.com>
Cc: Hannes Reinecke <hare@suse.de>,
	linux-block <linux-block@vger.kernel.org>,
	Mike Snitzer <snitzer@redhat.com>
Subject: Re: Adding QCOW2 reading/writing support
Date: Tue, 14 May 2019 10:56:31 +0200
Message-ID: <30d73468-a117-86a5-d38a-48f79546e5ee@web.de> (raw)
In-Reply-To: <1352c7de-b57d-2042-8bba-5a7a544390b8@web.de>

Hi

On 4/18/19 12:02 PM, Manuel Bentele wrote:
> On 18.04.19 03:05, Ming Lei wrote:
>> On Thu, Apr 18, 2019 at 5:04 AM Manuel Bentele <manuel-bentele@web.de> wrote:
>>> On 17.04.19 14:16, Hannes Reinecke wrote:
>>>> On 4/17/19 1:32 PM, Manuel Bentele wrote:
>>>>> Hi,
>>>>>
>>>>> On 17.04.19 03:35, Ming Lei wrote:
>>>>>> Hi,
>>>>>>
>>>>>> On Wed, Apr 17, 2019 at 5:33 AM Manuel Bentele
>>>>>> <manuel-bentele@web.de> wrote:
>>>>>>> Hi everyone
>>>>>>>
>>>>>>> I'm going to implement an in-kernel reading of QCOW2 images.
>>>>>>> In the project, I only need the reading of QCOW2 images, but it's
>>>>>>> essential to make thoughts for the implementation of the writing, too.
>>>>>>> One of the difficulties seems to be the support of making an image
>>>>>>> sparse (resizing the disk image).
>>>>>> Could you describe this requirement in a bit more detail? Especially
>>>>>> why
>>>>>> do you want to read/write QCOW2 in kernel?
>>>>> Yes, of course. The implementation of reading a QCOW2 disk image
>>>>> in-kernel is required for an already existing system in the university
>>>>> environment.
>>>>> At the moment, the Linux kernel, initramfs, etc. for each client in the
>>>>> system is loaded via PXE boot and then the block device with the default
>>>>> file system is included with the help of a modified nbd version, called
>>>>> dnbd (distributed nbd).
>>>>> Due to the fact that the data on the default file system is only for non
>>>>> persistent one-time provision of a client, read access is sufficient.
>>>>> The user related data is stored on a network storage, as mostly done in
>>>>> large scale infrastructures.
>>>>>
>>>>> Now, the goal is to minimize the network usage and avoid nbd.
>>>>> Furthermore, fixed configured and packed boot images should be avoided.
>>>>> Therefore, the advantage of the sparse and compression functionality of
>>>>> QCOW2 should be used.
>>>>> A workaround for that problem could be the local usage of nbd to include
>>>>> the QCOW2 disk image as block device, but it involves a lot of
>>>>> interaction between user and kernel space and thus an decreasing
>>>>> performance. That leads to the motivation to implement the reading of
>>>>> QCOW2 disk images directly in the kernel and aim for an merge into the
>>>>> mainline kernel source to avoid out-of-kernel-tree maintenance.
>>>>>
>>>>> If you have any questions related to the described use case or if you
>>>>> require more information, please let me know.
>>>>> Thanks for your help.
>>>>>
>>>> cramfs?
>>>> Or btrfs with compression enabled?
>>>>
>>>> Cheers,
>>>>
>>>> Hannes
>>> Thanks for your simple idea to choose cramfs or btrfs with compression
>>> enabled.
>>> I will suggest that as alternative at the next project meeting.
>> Or vdo which provides compression in block device level:
>>
>> https://github.com/dm-vdo/vdo
> Thanks for your hint. I will take a look at it.

I suggested all your ideas at the project meeting. The meeting revealed
that most of the people want to hold on the QCOW2 container format and
want to avoid the 'stacking' of other solutions.
In addition to that, FUSE should not be used due to performance reasons.

All in all, that leads to the plan to extend the existing loop device
driver to support not only raw binary files, but also the QCOW2
container format. If there exists a usable and clean solution at the
end, I will inform you and propose it for a review.

Thanks for your help.

Regards,
Manuel
<mailto:manuel-bentele@web.de>

  reply index

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-16 21:30 Manuel Bentele
2019-04-17  1:35 ` Ming Lei
2019-04-17 11:32   ` Manuel Bentele
2019-04-17 12:16     ` Hannes Reinecke
2019-04-17 21:04       ` Manuel Bentele
2019-04-18  1:05         ` Ming Lei
2019-04-18 10:02           ` Manuel Bentele
2019-05-14  8:56             ` Manuel Bentele [this message]
2019-04-17 11:58 ` Hannes Reinecke
2019-04-17 21:53   ` Manuel Bentele
2019-05-14 14:28     ` Roman Penyaev
2019-05-20 13:05       ` Manuel Bentele

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=30d73468-a117-86a5-d38a-48f79546e5ee@web.de \
    --to=manuel-bentele@web.de \
    --cc=hare@suse.de \
    --cc=linux-block@vger.kernel.org \
    --cc=snitzer@redhat.com \
    --cc=tom.leiming@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Block Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-block/0 linux-block/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-block linux-block/ https://lore.kernel.org/linux-block \
		linux-block@vger.kernel.org linux-block@archiver.kernel.org
	public-inbox-index linux-block


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-block


AGPL code for this site: git clone https://public-inbox.org/ public-inbox