Re: Adding QCOW2 reading/writing support

[Ming Lei <tom.leiming@gmail.com>]

On Thu, Apr 18, 2019 at 5:04 AM Manuel Bentele <manuel-bentele@web.de> wrote:
>
> On 17.04.19 14:16, Hannes Reinecke wrote:
> > On 4/17/19 1:32 PM, Manuel Bentele wrote:
> >> Hi,
> >>
> >> On 17.04.19 03:35, Ming Lei wrote:
> >>> Hi,
> >>>
> >>> On Wed, Apr 17, 2019 at 5:33 AM Manuel Bentele
> >>> <manuel-bentele@web.de> wrote:
> >>>> Hi everyone
> >>>>
> >>>> I'm going to implement an in-kernel reading of QCOW2 images.
> >>>> In the project, I only need the reading of QCOW2 images, but it's
> >>>> essential to make thoughts for the implementation of the writing, too.
> >>>> One of the difficulties seems to be the support of making an image
> >>>> sparse (resizing the disk image).
> >>> Could you describe this requirement in a bit more detail? Especially
> >>> why
> >>> do you want to read/write QCOW2 in kernel?
> >>
> >> Yes, of course. The implementation of reading a QCOW2 disk image
> >> in-kernel is required for an already existing system in the university
> >> environment.
> >> At the moment, the Linux kernel, initramfs, etc. for each client in the
> >> system is loaded via PXE boot and then the block device with the default
> >> file system is included with the help of a modified nbd version, called
> >> dnbd (distributed nbd).
> >> Due to the fact that the data on the default file system is only for non
> >> persistent one-time provision of a client, read access is sufficient.
> >> The user related data is stored on a network storage, as mostly done in
> >> large scale infrastructures.
> >>
> >> Now, the goal is to minimize the network usage and avoid nbd.
> >> Furthermore, fixed configured and packed boot images should be avoided.
> >> Therefore, the advantage of the sparse and compression functionality of
> >> QCOW2 should be used.
> >> A workaround for that problem could be the local usage of nbd to include
> >> the QCOW2 disk image as block device, but it involves a lot of
> >> interaction between user and kernel space and thus an decreasing
> >> performance. That leads to the motivation to implement the reading of
> >> QCOW2 disk images directly in the kernel and aim for an merge into the
> >> mainline kernel source to avoid out-of-kernel-tree maintenance.
> >>
> >> If you have any questions related to the described use case or if you
> >> require more information, please let me know.
> >> Thanks for your help.
> >>
> > cramfs?
> > Or btrfs with compression enabled?
> >
> > Cheers,
> >
> > Hannes
>
> Thanks for your simple idea to choose cramfs or btrfs with compression
> enabled.
> I will suggest that as alternative at the next project meeting.

Or vdo which provides compression in block device level:

https://github.com/dm-vdo/vdo


Thanks,
Ming Lei