[Block dev] : Qemu block ide_dma_read call routine

[Block dev] : Qemu block ide_dma_read call routine

[Shailesh Kumar]

Hi,

    I am implementing read equivalent routine in qemu. Can some one
help me understand  control flow of the qemu read/write
implementation.

I am using xen-4.2.0 and qemu-1.6.1

My requirement is simple:

I have a 1024*1024 buffer already filled with some useful data.

Now when windows (my guest OS) does IDE_DMA_READ command to the disk,
I want to intercept it and fill data from my private buffer.

my intention is to leverage existing dma_read infrastructure and
overwrite the read buffer-data at the lowest level of qemu . That way
the buffers /vectors "qiov" which are prepared due to cmd IDE_DMA_READ
will copy and return data from my data-buffer to guest-OS.

I could trace the control from.

ide_sector_start_dma
-> s->bus->dma->ops->start_dma
    -> ide_dma_cb
        ->dma_bdrv_read
           -> bdrv_aio_readv
.              ->bdrv_co_aio_rw_vector
                  -> bdrv_co_do_rw   "coroutine"
                      -> bdrv_co_do_readv
                          -> drv->bdrv_co_readv (( in my case it is
from raw.c raw_co_readv ))
                             -> bdrv_co_readv
                      -> bdrv_co_do_readv

                   ->in bdrv_co_do_rw  the bottom half is scheduled
bdrv_co_em_bh -->> this will invoke     -> ide_dma_cb () which is
again the starting point. Looks like there a double-linked list
maintained for the coroutine entries and are off loaded to qemu-wait
queue during this process.

Now I need help to understand where to look for to find the last
read/write system call which will get the data out from the disk for
guest-OS (windows) .

I am seeking suggestions and help for the same.

thanks
S. Kumar

Re: [Qemu-devel] [Block dev] : Qemu block ide_dma_read call routine

[Kevin Wolf]

Am 11.02.2015 um 04:51 hat Shailesh Kumar geschrieben:
> Hi,
> 
>     I am implementing read equivalent routine in qemu. Can some one
> help me understand  control flow of the qemu read/write
> implementation.
> 
> I am using xen-4.2.0 and qemu-1.6.1
> 
> My requirement is simple:
> 
> I have a 1024*1024 buffer already filled with some useful data.
> 
> Now when windows (my guest OS) does IDE_DMA_READ command to the disk,
> I want to intercept it and fill data from my private buffer.
> 
> my intention is to leverage existing dma_read infrastructure and
> overwrite the read buffer-data at the lowest level of qemu . That way
> the buffers /vectors "qiov" which are prepared due to cmd IDE_DMA_READ
> will copy and return data from my data-buffer to guest-OS.
> 
> I could trace the control from.
> 
> ide_sector_start_dma
> -> s->bus->dma->ops->start_dma
>     -> ide_dma_cb
>         ->dma_bdrv_read
>            -> bdrv_aio_readv
> .              ->bdrv_co_aio_rw_vector
>                   -> bdrv_co_do_rw   "coroutine"
>                       -> bdrv_co_do_readv
>                           -> drv->bdrv_co_readv (( in my case it is
> from raw.c raw_co_readv ))
>                              -> bdrv_co_readv

You need to be aware that BlockDriverStates are actually stacked. In
most common cases you have one protocol driver that accesses the image
file (this might be file, nbd, gluster, http, ssh, etc.) and on top of
that a driver that interprets the image format (like raw, qcow2, vmdk,
etc.)

You stopped your call chain at the point where the request has passed
through the 'raw' format driver. This final bdrv_co_readv() calls into
drv->bdrv_co_readv of the 'file' driver in raw-posix.c, which is doing
the actual syscalls.

>                       -> bdrv_co_do_readv
> 
>                    ->in bdrv_co_do_rw  the bottom half is scheduled
> bdrv_co_em_bh -->> this will invoke     -> ide_dma_cb () which is
> again the starting point. Looks like there a double-linked list
> maintained for the coroutine entries and are off loaded to qemu-wait
> queue during this process.
> 
> Now I need help to understand where to look for to find the last
> read/write system call which will get the data out from the disk for
> guest-OS (windows) .
> 
> I am seeking suggestions and help for the same.

As it might already have occurred to you reading the above, the stacking
provides you a clean way to implement your interception. You just need to
insert a filtering BlockDriver in the chain, so that you end up with:

raw -> intercept -> file

You could have a look at the quorum or blkdebug drivers, which can be
inserted in the same way. In contrast to those two drivers, I'd
recommend you to implement bdrv_co_readv/writev instead of
bdrv_aio_readv/writev, because they are probably simpler to use for your
case.

Kevin

Re: [Qemu-devel] [Block dev] : Qemu block ide_dma_read call routine

[Shailesh Kumar]

On Mon, Feb 23, 2015 at 3:25 AM, Kevin Wolf <kwolf@redhat.com> wrote:
> Am 11.02.2015 um 04:51 hat Shailesh Kumar geschrieben:
>> Hi,
>>
>>     I am implementing read equivalent routine in qemu. Can some one
>> help me understand  control flow of the qemu read/write
>> implementation.
>>
>> I am using xen-4.2.0 and qemu-1.6.1
>>
>> My requirement is simple:
>>
>> I have a 1024*1024 buffer already filled with some useful data.
>>
>> Now when windows (my guest OS) does IDE_DMA_READ command to the disk,
>> I want to intercept it and fill data from my private buffer.
>>
>> my intention is to leverage existing dma_read infrastructure and
>> overwrite the read buffer-data at the lowest level of qemu . That way
>> the buffers /vectors "qiov" which are prepared due to cmd IDE_DMA_READ
>> will copy and return data from my data-buffer to guest-OS.
>>
>> I could trace the control from.
>>
>> ide_sector_start_dma
>> -> s->bus->dma->ops->start_dma
>>     -> ide_dma_cb
>>         ->dma_bdrv_read
>>            -> bdrv_aio_readv
>> .              ->bdrv_co_aio_rw_vector
>>                   -> bdrv_co_do_rw   "coroutine"
>>                       -> bdrv_co_do_readv
>>                           -> drv->bdrv_co_readv (( in my case it is
>> from raw.c raw_co_readv ))
>>                              -> bdrv_co_readv
>
> You need to be aware that BlockDriverStates are actually stacked. In
> most common cases you have one protocol driver that accesses the image
> file (this might be file, nbd, gluster, http, ssh, etc.) and on top of
> that a driver that interprets the image format (like raw, qcow2, vmdk,
> etc.)
>
> You stopped your call chain at the point where the request has passed
> through the 'raw' format driver. This final bdrv_co_readv() calls into
> drv->bdrv_co_readv of the 'file' driver in raw-posix.c, which is doing
> the actual syscalls.
>
>>                       -> bdrv_co_do_readv
>>
>>                    ->in bdrv_co_do_rw  the bottom half is scheduled
>> bdrv_co_em_bh -->> this will invoke     -> ide_dma_cb () which is
>> again the starting point. Looks like there a double-linked list
>> maintained for the coroutine entries and are off loaded to qemu-wait
>> queue during this process.
>>
>> Now I need help to understand where to look for to find the last
>> read/write system call which will get the data out from the disk for
>> guest-OS (windows) .
>>
>> I am seeking suggestions and help for the same.
>
> As it might already have occurred to you reading the above, the stacking
> provides you a clean way to implement your interception. You just need to
> insert a filtering BlockDriver in the chain, so that you end up with:
>
> raw -> intercept -> file
>
> You could have a look at the quorum or blkdebug drivers, which can be
> inserted in the same way. In contrast to those two drivers, I'd
> recommend you to implement bdrv_co_readv/writev instead of
> bdrv_aio_readv/writev, because they are probably simpler to use for your
> case.
>
> Kevin

Kevin,

Thanks for the detailed explanation.
Your email gave me little more insight about the control flow.

Gmail has put this email at different place and I was not able to find
it till now. hence delayed reply.

I am confused with format_names.
What does host_device mean and what does raw means ?
drv->format_name does it mean what form of the driver to be used ?
what attributes decide which driver to be used.

for e.g: if I print the drv->format_name in bdrv_co_readv I see raw
and host_device being printed alternately.

-Sailesh

Re: [Qemu-devel] [Block dev] : Qemu block ide_dma_read call routine

[Kevin Wolf]

Am 23.07.2015 um 21:20 hat Shailesh Kumar geschrieben:
> On Mon, Feb 23, 2015 at 3:25 AM, Kevin Wolf <kwolf@redhat.com> wrote:
> > Am 11.02.2015 um 04:51 hat Shailesh Kumar geschrieben:
> >> Hi,
> >>
> >>     I am implementing read equivalent routine in qemu. Can some one
> >> help me understand  control flow of the qemu read/write
> >> implementation.
> >>
> >> I am using xen-4.2.0 and qemu-1.6.1
> >>
> >> My requirement is simple:
> >>
> >> I have a 1024*1024 buffer already filled with some useful data.
> >>
> >> Now when windows (my guest OS) does IDE_DMA_READ command to the disk,
> >> I want to intercept it and fill data from my private buffer.
> >>
> >> my intention is to leverage existing dma_read infrastructure and
> >> overwrite the read buffer-data at the lowest level of qemu . That way
> >> the buffers /vectors "qiov" which are prepared due to cmd IDE_DMA_READ
> >> will copy and return data from my data-buffer to guest-OS.
> >>
> >> I could trace the control from.
> >>
> >> ide_sector_start_dma
> >> -> s->bus->dma->ops->start_dma
> >>     -> ide_dma_cb
> >>         ->dma_bdrv_read
> >>            -> bdrv_aio_readv
> >> .              ->bdrv_co_aio_rw_vector
> >>                   -> bdrv_co_do_rw   "coroutine"
> >>                       -> bdrv_co_do_readv
> >>                           -> drv->bdrv_co_readv (( in my case it is
> >> from raw.c raw_co_readv ))
> >>                              -> bdrv_co_readv
> >
> > You need to be aware that BlockDriverStates are actually stacked. In
> > most common cases you have one protocol driver that accesses the image
> > file (this might be file, nbd, gluster, http, ssh, etc.) and on top of
> > that a driver that interprets the image format (like raw, qcow2, vmdk,
> > etc.)
> >
> > You stopped your call chain at the point where the request has passed
> > through the 'raw' format driver. This final bdrv_co_readv() calls into
> > drv->bdrv_co_readv of the 'file' driver in raw-posix.c, which is doing
> > the actual syscalls.
> >
> >>                       -> bdrv_co_do_readv
> >>
> >>                    ->in bdrv_co_do_rw  the bottom half is scheduled
> >> bdrv_co_em_bh -->> this will invoke     -> ide_dma_cb () which is
> >> again the starting point. Looks like there a double-linked list
> >> maintained for the coroutine entries and are off loaded to qemu-wait
> >> queue during this process.
> >>
> >> Now I need help to understand where to look for to find the last
> >> read/write system call which will get the data out from the disk for
> >> guest-OS (windows) .
> >>
> >> I am seeking suggestions and help for the same.
> >
> > As it might already have occurred to you reading the above, the stacking
> > provides you a clean way to implement your interception. You just need to
> > insert a filtering BlockDriver in the chain, so that you end up with:
> >
> > raw -> intercept -> file
> >
> > You could have a look at the quorum or blkdebug drivers, which can be
> > inserted in the same way. In contrast to those two drivers, I'd
> > recommend you to implement bdrv_co_readv/writev instead of
> > bdrv_aio_readv/writev, because they are probably simpler to use for your
> > case.
> >
> > Kevin
> 
> Kevin,
> 
> Thanks for the detailed explanation.
> Your email gave me little more insight about the control flow.
> 
> Gmail has put this email at different place and I was not able to find
> it till now. hence delayed reply.
> 
> I am confused with format_names.
> What does host_device mean and what does raw means ?
> drv->format_name does it mean what form of the driver to be used ?
> what attributes decide which driver to be used.
> 
> for e.g: if I print the drv->format_name in bdrv_co_readv I see raw
> and host_device being printed alternately.

You can read 'host_device' everywhere where I wrote 'file' above. There
are a few differences between regular files and block devices, so they
have two different drivers, but they share most of the code. They are
both implemented in block/raw-posix.c for Linux.

The process of opening an image is relatively complex. The rough outline
is that first the protocol driver is opened. If you didn't specify it
explicitly, the image path is assumed to be a local filename and the
right driver is selected for it (file for regular files, and for block
devices one of host_cdrom, host_floppy or host_device).

On top of that, a format driver is opened. If you didn't specify it, the
first 2k of the image file are read in (using the protocol driver opened
before) and the image format is probed, i.e. we look at magic numbers
in the image header etc.

Kevin