[Qemu-devel] iolimits for virtio-9p

[Qemu-devel] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 354 bytes --]

Hi All,

We are planning to implement the io-limits for the virtio-9p driver i.e for
fsdev devices.
So, I am looking into the code base and how it has done for the block io
devices.

I would like to know how difficult is this and is there some one out there
who has any plan to do this?

Any suggestions/opinions are very much helpful.

Regards,
Pradeep

[-- Attachment #2: Type: text/html, Size: 496 bytes --]

Re: [Qemu-devel] iolimits for virtio-9p

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 513 bytes --]

On Tue, Apr 19, 2016 at 02:09:24PM +0200, Pradeep Kiruvale wrote:
> We are planning to implement the io-limits for the virtio-9p driver i.e for
> fsdev devices.
> So, I am looking into the code base and how it has done for the block io
> devices.
> 
> I would like to know how difficult is this and is there some one out there
> who has any plan to do this?

CCing Alberto Garcia, QEMU I/O throttling maintainer.

Take a look at include/qemu/throttle.h.  That is the common throttling
API.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 723 bytes --]

Hi Stefan,

Thanks for the reply and adding Alberto.

I am going through those APIs already.

Thanks & Regards,
Pradeep

On 26 April 2016 at 11:21, Stefan Hajnoczi <stefanha@gmail.com> wrote:

> On Tue, Apr 19, 2016 at 02:09:24PM +0200, Pradeep Kiruvale wrote:
> > We are planning to implement the io-limits for the virtio-9p driver i.e
> for
> > fsdev devices.
> > So, I am looking into the code base and how it has done for the block io
> > devices.
> >
> > I would like to know how difficult is this and is there some one out
> there
> > who has any plan to do this?
>
> CCing Alberto Garcia, QEMU I/O throttling maintainer.
>
> Take a look at include/qemu/throttle.h.  That is the common throttling
> API.
>
> Stefan
>

[-- Attachment #2: Type: text/html, Size: 1245 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Alberto Garcia]

On Tue 19 Apr 2016 02:09:24 PM CEST, Pradeep Kiruvale wrote:

> We are planning to implement the io-limits for the virtio-9p driver
> i.e for fsdev devices.
> So, I am looking into the code base and how it has done for the block
> io devices.
>
> I would like to know how difficult is this and is there some one out
> there who has any plan to do this?

Hi,

as Stefan said already, the common API is in throttle.h.

It should be generic enough to be used in other parts of QEMU, but tell
me if you feel that it needs changes.

Once you configure the throttling settings you essentially only need to
call throttle_schedule_timer() to see if a request needs to be
throttled, and afterwards throttle_account() to register the I/O that
has been peformed.

You'll see that there's also throttle-group.c, but that's specific to
the block layer and not meant to be generic.

Berto

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 1392 bytes --]

On 26 April 2016 at 14:08, Alberto Garcia <berto@igalia.com> wrote:

> On Tue 19 Apr 2016 02:09:24 PM CEST, Pradeep Kiruvale wrote:
>
> > We are planning to implement the io-limits for the virtio-9p driver
> > i.e for fsdev devices.
> > So, I am looking into the code base and how it has done for the block
> > io devices.
> >
> > I would like to know how difficult is this and is there some one out
> > there who has any plan to do this?
>
> Hi,
>
> as Stefan said already, the common API is in throttle.h.
>
> It should be generic enough to be used in other parts of QEMU, but tell
> me if you feel that it needs changes.
>
> Once you configure the throttling settings you essentially only need to
> call throttle_schedule_timer() to see if a request needs to be
> throttled, and afterwards throttle_account() to register the I/O that
> has been peformed.
>
> You'll see that there's also throttle-group.c, but that's specific to
> the block layer and not meant to be generic.
>
>
> Hi Alberto,

Thanks for the reply. I am still in the early phase, I will let you know
if any changes are needed for the APIs.

We might also have to implement throttle-group.c for 9p devices, if we want
to
apply throttle for group of devices.

I just want some one from virtio-9p side to jump in to the discussion
as well. That will help me a lot to understand from virtio-9p perspective.

Regards,
Pradeep

[-- Attachment #2: Type: text/html, Size: 1976 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Alberto Garcia]

On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:

> Thanks for the reply. I am still in the early phase, I will let you
> know if any changes are needed for the APIs.
> 
> We might also have to implement throttle-group.c for 9p devices, if
> we want to apply throttle for group of devices.

Fair enough, but again please note that:

- throttle-group.c is not meant to be generic, but it's tied to
  BlockDriverState / BlockBackend.
- it is currently being rewritten:
  https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html

If you can explain your use case with a bit more detail we can try to
see what can be done about it.

Regards,

Berto

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 1267 bytes --]

On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com> wrote:

> On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:
>
> > Thanks for the reply. I am still in the early phase, I will let you
> > know if any changes are needed for the APIs.
> >
> > We might also have to implement throttle-group.c for 9p devices, if
> > we want to apply throttle for group of devices.
>
> Fair enough, but again please note that:
>
> - throttle-group.c is not meant to be generic, but it's tied to
>   BlockDriverState / BlockBackend.
> - it is currently being rewritten:
>   https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
>
> If you can explain your use case with a bit more detail we can try to
> see what can be done about it.
>
>
We want to use  virtio-9p for block io instead of virtio-blk-pci. But in
case of
virtio-9p we can just use fsdev devices, so we want to apply throttling
(QoS)
on these devices and as of now the io throttling only possible with the
-drive option.

As a work around we are doing the throttling using cgroup. It has its own
costs.
So, we want to have throttling for fsdev devices inside the qemu itself. I
am just
trying to understand and estimate time required for implementing it for the
fsdevices.


-Pradeep

[-- Attachment #2: Type: text/html, Size: 1931 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Greg Kurz]

On Wed, 27 Apr 2016 16:39:58 +0200
Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:

> On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com> wrote:
> 
> > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:
> >
> > > Thanks for the reply. I am still in the early phase, I will let you
> > > know if any changes are needed for the APIs.
> > >
> > > We might also have to implement throttle-group.c for 9p devices, if
> > > we want to apply throttle for group of devices.
> >
> > Fair enough, but again please note that:
> >
> > - throttle-group.c is not meant to be generic, but it's tied to
> >   BlockDriverState / BlockBackend.
> > - it is currently being rewritten:
> >   https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> >
> > If you can explain your use case with a bit more detail we can try to
> > see what can be done about it.
> >
> >
> We want to use  virtio-9p for block io instead of virtio-blk-pci. But in
> case of

9p is mostly aimed at sharing files... why would you want to use it for
block io instead of a true block device ? And how would you do that ?

> virtio-9p we can just use fsdev devices, so we want to apply throttling
> (QoS)
> on these devices and as of now the io throttling only possible with the
> -drive option.
> 

Indeed.

> As a work around we are doing the throttling using cgroup. It has its own
> costs.

Can you elaborate ?

> So, we want to have throttling for fsdev devices inside the qemu itself. I
> am just
> trying to understand and estimate time required for implementing it for the
> fsdevices.
> 

I still don't clearly understand what you are trying to do... maybe provide
a more detailed scenario.

> 
> -Pradeep

Cheers.

--
Greg

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 1955 bytes --]

On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:

> On Wed, 27 Apr 2016 16:39:58 +0200
> Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
>
> > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com> wrote:
> >
> > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:
> > >
> > > > Thanks for the reply. I am still in the early phase, I will let you
> > > > know if any changes are needed for the APIs.
> > > >
> > > > We might also have to implement throttle-group.c for 9p devices, if
> > > > we want to apply throttle for group of devices.
> > >
> > > Fair enough, but again please note that:
> > >
> > > - throttle-group.c is not meant to be generic, but it's tied to
> > >   BlockDriverState / BlockBackend.
> > > - it is currently being rewritten:
> > >   https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > >
> > > If you can explain your use case with a bit more detail we can try to
> > > see what can be done about it.
> > >
> > >
> > We want to use  virtio-9p for block io instead of virtio-blk-pci. But in
> > case of
>
> 9p is mostly aimed at sharing files... why would you want to use it for
> block io instead of a true block device ? And how would you do that ?
>

*Yes, we want to share the files itself. So we are using the virtio-9p.*
*We want to have QoS on these files access for every VM.*


>
> > virtio-9p we can just use fsdev devices, so we want to apply throttling
> > (QoS)
> > on these devices and as of now the io throttling only possible with the
> > -drive option.
> >
>
> Indeed.
>
> > As a work around we are doing the throttling using cgroup. It has its own
> > costs.
>
> Can you elaborate ?
>

*We saw that we need to create cgroups and set it and also we observed lot
of iowaits *
*compared to implementing the throttling inside the qemu.*
*This we did observe by using the virtio-blk-pci devices. (Using cgroups Vs
qemu throttling)*


Thanks,
Pradeep

[-- Attachment #2: Type: text/html, Size: 3143 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Greg Kurz]

On Thu, 28 Apr 2016 11:45:41 +0200
Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:

> On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> 
> > On Wed, 27 Apr 2016 16:39:58 +0200
> > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> >
> > > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com> wrote:
> > >
> > > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:
> > > >
> > > > > Thanks for the reply. I am still in the early phase, I will let you
> > > > > know if any changes are needed for the APIs.
> > > > >
> > > > > We might also have to implement throttle-group.c for 9p devices, if
> > > > > we want to apply throttle for group of devices.
> > > >
> > > > Fair enough, but again please note that:
> > > >
> > > > - throttle-group.c is not meant to be generic, but it's tied to
> > > >   BlockDriverState / BlockBackend.
> > > > - it is currently being rewritten:
> > > >   https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > > >
> > > > If you can explain your use case with a bit more detail we can try to
> > > > see what can be done about it.
> > > >
> > > >
> > > We want to use  virtio-9p for block io instead of virtio-blk-pci. But in
> > > case of
> >
> > 9p is mostly aimed at sharing files... why would you want to use it for
> > block io instead of a true block device ? And how would you do that ?
> >
> 
> *Yes, we want to share the files itself. So we are using the virtio-9p.*

You want to pass a disk image to the guest as a plain file on a 9p mount ?
And then, what do you do in the guest ? Attach it to a loop device ?

> *We want to have QoS on these files access for every VM.*
> 

You won't be able to have QoS on selected files, but it may be possible to
introduce limits at the fsdev level: control all write accesses to all files
and all read accesses to all files for a 9p device.

> 
> >
> > > virtio-9p we can just use fsdev devices, so we want to apply throttling
> > > (QoS)
> > > on these devices and as of now the io throttling only possible with the
> > > -drive option.
> > >
> >
> > Indeed.
> >
> > > As a work around we are doing the throttling using cgroup. It has its own
> > > costs.
> >
> > Can you elaborate ?
> >
> 
> *We saw that we need to create cgroups and set it and also we observed lot
> of iowaits *
> *compared to implementing the throttling inside the qemu.*
> *This we did observe by using the virtio-blk-pci devices. (Using cgroups Vs
> qemu throttling)*
> 

Just to be sure I get it right.

You tried both:
1) run QEMU with -device virtio-blk-pci and -drive throttling.*
2) run QEMU with -device virtio-blk-pci in its own cgroup

And 1) has better performance and is easier to use than 2) ?

And what do you expect with 9p compared to 1) ?

> 
> Thanks,
> Pradeep

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 3645 bytes --]

On 2 May 2016 at 14:57, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:

> On Thu, 28 Apr 2016 11:45:41 +0200
> Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
>
> > On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> >
> > > On Wed, 27 Apr 2016 16:39:58 +0200
> > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > >
> > > > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com> wrote:
> > > >
> > > > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:
> > > > >
> > > > > > Thanks for the reply. I am still in the early phase, I will let
> you
> > > > > > know if any changes are needed for the APIs.
> > > > > >
> > > > > > We might also have to implement throttle-group.c for 9p devices,
> if
> > > > > > we want to apply throttle for group of devices.
> > > > >
> > > > > Fair enough, but again please note that:
> > > > >
> > > > > - throttle-group.c is not meant to be generic, but it's tied to
> > > > >   BlockDriverState / BlockBackend.
> > > > > - it is currently being rewritten:
> > > > >
> https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > > > >
> > > > > If you can explain your use case with a bit more detail we can try
> to
> > > > > see what can be done about it.
> > > > >
> > > > >
> > > > We want to use  virtio-9p for block io instead of virtio-blk-pci.
> But in
> > > > case of
> > >
> > > 9p is mostly aimed at sharing files... why would you want to use it for
> > > block io instead of a true block device ? And how would you do that ?
> > >
> >
> > *Yes, we want to share the files itself. So we are using the virtio-9p.*
>
> You want to pass a disk image to the guest as a plain file on a 9p mount ?
> And then, what do you do in the guest ? Attach it to a loop device ?
>

Yes, would like to mount as  a 9p drive and create file inside that and
read/write.
This was the experiment we are doing, actual use case no idea. My work is
to do
a feasibility test does it work or not.


>
> > *We want to have QoS on these files access for every VM.*
> >
>
> You won't be able to have QoS on selected files, but it may be possible to
> introduce limits at the fsdev level: control all write accesses to all
> files
> and all read accesses to all files for a 9p device.
>

That is right, I do not want to have QoS for individual files but to whole
fsdev device.


> >
> > >
> > > > virtio-9p we can just use fsdev devices, so we want to apply
> throttling
> > > > (QoS)
> > > > on these devices and as of now the io throttling only possible with
> the
> > > > -drive option.
> > > >
> > >
> > > Indeed.
> > >
> > > > As a work around we are doing the throttling using cgroup. It has
> its own
> > > > costs.
> > >
> > > Can you elaborate ?
> > >
> >
> > *We saw that we need to create cgroups and set it and also we observed
> lot
> > of iowaits *
> > *compared to implementing the throttling inside the qemu.*
> > *This we did observe by using the virtio-blk-pci devices. (Using cgroups
> Vs
> > qemu throttling)*
> >
>


>
> Just to be sure I get it right.
>
> You tried both:
> 1) run QEMU with -device virtio-blk-pci and -drive throttling.*
> 2) run QEMU with -device virtio-blk-pci in its own cgroup
>
> And 1) has better performance and is easier to use than 2) ?
>
> And what do you expect with 9p compared to 1) ?
>
>
That was just to understand the cost of cpu
 io throttling inside the qemu vs using cgroup.

The bench-marking we did to reproduce the numbers and understand the cost
mentioned in

http://www.linux-kvm.org/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf

Thanks,
Pradeep


> >
> > Thanks,
> > Pradeep
>
>

[-- Attachment #2: Type: text/html, Size: 6150 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Greg Kurz]

On Mon, 2 May 2016 17:49:26 +0200
Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:

> On 2 May 2016 at 14:57, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> 
> > On Thu, 28 Apr 2016 11:45:41 +0200
> > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> >
> > > On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> > >
> > > > On Wed, 27 Apr 2016 16:39:58 +0200
> > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > >
> > > > > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com> wrote:
> > > > >
> > > > > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale wrote:
> > > > > >
> > > > > > > Thanks for the reply. I am still in the early phase, I will let
> > you
> > > > > > > know if any changes are needed for the APIs.
> > > > > > >
> > > > > > > We might also have to implement throttle-group.c for 9p devices,
> > if
> > > > > > > we want to apply throttle for group of devices.
> > > > > >
> > > > > > Fair enough, but again please note that:
> > > > > >
> > > > > > - throttle-group.c is not meant to be generic, but it's tied to
> > > > > >   BlockDriverState / BlockBackend.
> > > > > > - it is currently being rewritten:
> > > > > >
> > https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > > > > >
> > > > > > If you can explain your use case with a bit more detail we can try
> > to
> > > > > > see what can be done about it.
> > > > > >
> > > > > >
> > > > > We want to use  virtio-9p for block io instead of virtio-blk-pci.
> > But in
> > > > > case of
> > > >
> > > > 9p is mostly aimed at sharing files... why would you want to use it for
> > > > block io instead of a true block device ? And how would you do that ?
> > > >
> > >
> > > *Yes, we want to share the files itself. So we are using the virtio-9p.*
> >
> > You want to pass a disk image to the guest as a plain file on a 9p mount ?
> > And then, what do you do in the guest ? Attach it to a loop device ?
> >
> 
> Yes, would like to mount as  a 9p drive and create file inside that and
> read/write.
> This was the experiment we are doing, actual use case no idea. My work is
> to do
> a feasibility test does it work or not.
> 
> 
> >
> > > *We want to have QoS on these files access for every VM.*
> > >
> >
> > You won't be able to have QoS on selected files, but it may be possible to
> > introduce limits at the fsdev level: control all write accesses to all
> > files
> > and all read accesses to all files for a 9p device.
> >
> 
> That is right, I do not want to have QoS for individual files but to whole
> fsdev device.
> 
> 
> > >
> > > >
> > > > > virtio-9p we can just use fsdev devices, so we want to apply
> > throttling
> > > > > (QoS)
> > > > > on these devices and as of now the io throttling only possible with
> > the
> > > > > -drive option.
> > > > >
> > > >
> > > > Indeed.
> > > >
> > > > > As a work around we are doing the throttling using cgroup. It has
> > its own
> > > > > costs.
> > > >
> > > > Can you elaborate ?
> > > >
> > >
> > > *We saw that we need to create cgroups and set it and also we observed
> > lot
> > > of iowaits *
> > > *compared to implementing the throttling inside the qemu.*
> > > *This we did observe by using the virtio-blk-pci devices. (Using cgroups
> > Vs
> > > qemu throttling)*
> > >
> >
> 
> 
> >
> > Just to be sure I get it right.
> >
> > You tried both:
> > 1) run QEMU with -device virtio-blk-pci and -drive throttling.*
> > 2) run QEMU with -device virtio-blk-pci in its own cgroup
> >
> > And 1) has better performance and is easier to use than 2) ?
> >
> > And what do you expect with 9p compared to 1) ?
> >
> >
> That was just to understand the cost of cpu
>  io throttling inside the qemu vs using cgroup.
> 
> The bench-marking we did to reproduce the numbers and understand the cost
> mentioned in
> 
> http://www.linux-kvm.org/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf
> 
> Thanks,
> Pradeep
> 

Ok. So you did compare current QEMU block I/O throttling with cgroup ? And you observed numbers
similar to the link above ?

And now you would like to run the same test on a file in a 9p mount with experimental 9p QoS ?

Maybe possible to reuse the throttle.h API and hack v9fs_write() and v9fs_read() in 9p.c then.

Cheers.

--
Greg

> 
> > >
> > > Thanks,
> > > Pradeep
> >
> >

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 5014 bytes --]

On 4 May 2016 at 17:40, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:

> On Mon, 2 May 2016 17:49:26 +0200
> Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
>
> > On 2 May 2016 at 14:57, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> >
> > > On Thu, 28 Apr 2016 11:45:41 +0200
> > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > >
> > > > On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com>
> wrote:
> > > >
> > > > > On Wed, 27 Apr 2016 16:39:58 +0200
> > > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > > >
> > > > > > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com>
> wrote:
> > > > > >
> > > > > > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale
> wrote:
> > > > > > >
> > > > > > > > Thanks for the reply. I am still in the early phase, I will
> let
> > > you
> > > > > > > > know if any changes are needed for the APIs.
> > > > > > > >
> > > > > > > > We might also have to implement throttle-group.c for 9p
> devices,
> > > if
> > > > > > > > we want to apply throttle for group of devices.
> > > > > > >
> > > > > > > Fair enough, but again please note that:
> > > > > > >
> > > > > > > - throttle-group.c is not meant to be generic, but it's tied to
> > > > > > >   BlockDriverState / BlockBackend.
> > > > > > > - it is currently being rewritten:
> > > > > > >
> > > https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > > > > > >
> > > > > > > If you can explain your use case with a bit more detail we can
> try
> > > to
> > > > > > > see what can be done about it.
> > > > > > >
> > > > > > >
> > > > > > We want to use  virtio-9p for block io instead of virtio-blk-pci.
> > > But in
> > > > > > case of
> > > > >
> > > > > 9p is mostly aimed at sharing files... why would you want to use
> it for
> > > > > block io instead of a true block device ? And how would you do
> that ?
> > > > >
> > > >
> > > > *Yes, we want to share the files itself. So we are using the
> virtio-9p.*
> > >
> > > You want to pass a disk image to the guest as a plain file on a 9p
> mount ?
> > > And then, what do you do in the guest ? Attach it to a loop device ?
> > >
> >
> > Yes, would like to mount as  a 9p drive and create file inside that and
> > read/write.
> > This was the experiment we are doing, actual use case no idea. My work is
> > to do
> > a feasibility test does it work or not.
> >
> >
> > >
> > > > *We want to have QoS on these files access for every VM.*
> > > >
> > >
> > > You won't be able to have QoS on selected files, but it may be
> possible to
> > > introduce limits at the fsdev level: control all write accesses to all
> > > files
> > > and all read accesses to all files for a 9p device.
> > >
> >
> > That is right, I do not want to have QoS for individual files but to
> whole
> > fsdev device.
> >
> >
> > > >
> > > > >
> > > > > > virtio-9p we can just use fsdev devices, so we want to apply
> > > throttling
> > > > > > (QoS)
> > > > > > on these devices and as of now the io throttling only possible
> with
> > > the
> > > > > > -drive option.
> > > > > >
> > > > >
> > > > > Indeed.
> > > > >
> > > > > > As a work around we are doing the throttling using cgroup. It has
> > > its own
> > > > > > costs.
> > > > >
> > > > > Can you elaborate ?
> > > > >
> > > >
> > > > *We saw that we need to create cgroups and set it and also we
> observed
> > > lot
> > > > of iowaits *
> > > > *compared to implementing the throttling inside the qemu.*
> > > > *This we did observe by using the virtio-blk-pci devices. (Using
> cgroups
> > > Vs
> > > > qemu throttling)*
> > > >
> > >
> >
> >
> > >
> > > Just to be sure I get it right.
> > >
> > > You tried both:
> > > 1) run QEMU with -device virtio-blk-pci and -drive throttling.*
> > > 2) run QEMU with -device virtio-blk-pci in its own cgroup
> > >
> > > And 1) has better performance and is easier to use than 2) ?
> > >
> > > And what do you expect with 9p compared to 1) ?
> > >
> > >
> > That was just to understand the cost of cpu
> >  io throttling inside the qemu vs using cgroup.
> >
> > The bench-marking we did to reproduce the numbers and understand the cost
> > mentioned in
> >
> >
> http://www.linux-kvm.org/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf
> >
> > Thanks,
> > Pradeep
> >
>
> Ok. So you did compare current QEMU block I/O throttling with cgroup ? And
> you observed numbers
> similar to the link above ?
>

*Yes, I did, I did run DD command in guest to do IO. The recent QEMU is in
par with cgroups in terms *
*of CPU utilization.*

>
> And now you would like to run the same test on a file in a 9p mount with
> experimental 9p QoS ?
>
> *Yes, you are right.*


> Maybe possible to reuse the throttle.h API and hack v9fs_write() and
> v9fs_read() in 9p.c then.
>
>
*OK, I am looking into it. Are there any sample test cases or something
about how to apply the*
*throttling APIs to a device?*


Regards,
Pradeep



> Cheers.
>
> --
> Greg
>
> >
> > > >
> > > > Thanks,
> > > > Pradeep
> > >
> > >
>
>
>

[-- Attachment #2: Type: text/html, Size: 8279 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Greg Kurz]

On Fri, 6 May 2016 08:01:09 +0200
Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:

> On 4 May 2016 at 17:40, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> 
> > On Mon, 2 May 2016 17:49:26 +0200
> > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> >
> > > On 2 May 2016 at 14:57, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> > >
> > > > On Thu, 28 Apr 2016 11:45:41 +0200
> > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > >
> > > > > On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com>
> > wrote:
> > > > >
> > > > > > On Wed, 27 Apr 2016 16:39:58 +0200
> > > > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > > > >
> > > > > > > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com>
> > wrote:
> > > > > > >
> > > > > > > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale
> > wrote:
> > > > > > > >
> > > > > > > > > Thanks for the reply. I am still in the early phase, I will
> > let
> > > > you
> > > > > > > > > know if any changes are needed for the APIs.
> > > > > > > > >
> > > > > > > > > We might also have to implement throttle-group.c for 9p
> > devices,
> > > > if
> > > > > > > > > we want to apply throttle for group of devices.
> > > > > > > >
> > > > > > > > Fair enough, but again please note that:
> > > > > > > >
> > > > > > > > - throttle-group.c is not meant to be generic, but it's tied to
> > > > > > > >   BlockDriverState / BlockBackend.
> > > > > > > > - it is currently being rewritten:
> > > > > > > >
> > > > https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > > > > > > >
> > > > > > > > If you can explain your use case with a bit more detail we can
> > try
> > > > to
> > > > > > > > see what can be done about it.
> > > > > > > >
> > > > > > > >
> > > > > > > We want to use  virtio-9p for block io instead of virtio-blk-pci.
> > > > But in
> > > > > > > case of
> > > > > >
> > > > > > 9p is mostly aimed at sharing files... why would you want to use
> > it for
> > > > > > block io instead of a true block device ? And how would you do
> > that ?
> > > > > >
> > > > >
> > > > > *Yes, we want to share the files itself. So we are using the
> > virtio-9p.*
> > > >
> > > > You want to pass a disk image to the guest as a plain file on a 9p
> > mount ?
> > > > And then, what do you do in the guest ? Attach it to a loop device ?
> > > >
> > >
> > > Yes, would like to mount as  a 9p drive and create file inside that and
> > > read/write.
> > > This was the experiment we are doing, actual use case no idea. My work is
> > > to do
> > > a feasibility test does it work or not.
> > >
> > >
> > > >
> > > > > *We want to have QoS on these files access for every VM.*
> > > > >
> > > >
> > > > You won't be able to have QoS on selected files, but it may be
> > possible to
> > > > introduce limits at the fsdev level: control all write accesses to all
> > > > files
> > > > and all read accesses to all files for a 9p device.
> > > >
> > >
> > > That is right, I do not want to have QoS for individual files but to
> > whole
> > > fsdev device.
> > >
> > >
> > > > >
> > > > > >
> > > > > > > virtio-9p we can just use fsdev devices, so we want to apply
> > > > throttling
> > > > > > > (QoS)
> > > > > > > on these devices and as of now the io throttling only possible
> > with
> > > > the
> > > > > > > -drive option.
> > > > > > >
> > > > > >
> > > > > > Indeed.
> > > > > >
> > > > > > > As a work around we are doing the throttling using cgroup. It has
> > > > its own
> > > > > > > costs.
> > > > > >
> > > > > > Can you elaborate ?
> > > > > >
> > > > >
> > > > > *We saw that we need to create cgroups and set it and also we
> > observed
> > > > lot
> > > > > of iowaits *
> > > > > *compared to implementing the throttling inside the qemu.*
> > > > > *This we did observe by using the virtio-blk-pci devices. (Using
> > cgroups
> > > > Vs
> > > > > qemu throttling)*
> > > > >
> > > >
> > >
> > >
> > > >
> > > > Just to be sure I get it right.
> > > >
> > > > You tried both:
> > > > 1) run QEMU with -device virtio-blk-pci and -drive throttling.*
> > > > 2) run QEMU with -device virtio-blk-pci in its own cgroup
> > > >
> > > > And 1) has better performance and is easier to use than 2) ?
> > > >
> > > > And what do you expect with 9p compared to 1) ?
> > > >
> > > >
> > > That was just to understand the cost of cpu
> > >  io throttling inside the qemu vs using cgroup.
> > >
> > > The bench-marking we did to reproduce the numbers and understand the cost
> > > mentioned in
> > >
> > >
> > http://www.linux-kvm.org/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf
> > >
> > > Thanks,
> > > Pradeep
> > >
> >
> > Ok. So you did compare current QEMU block I/O throttling with cgroup ? And
> > you observed numbers
> > similar to the link above ?
> >
> 
> *Yes, I did, I did run DD command in guest to do IO. The recent QEMU is in
> par with cgroups in terms *
> *of CPU utilization.*
> 
> >
> > And now you would like to run the same test on a file in a 9p mount with
> > experimental 9p QoS ?
> >
> > *Yes, you are right.*
> 
> 
> > Maybe possible to reuse the throttle.h API and hack v9fs_write() and
> > v9fs_read() in 9p.c then.
> >
> >
> *OK, I am looking into it. Are there any sample test cases or something
> about how to apply the*
> *throttling APIs to a device?*
> 

The throttling API is currently only used by block devices, and the only
documentation out-there is the code itself...

> 
> Regards,
> Pradeep
> 
> 
> 
> > Cheers.
> >
> > --
> > Greg
> >
> > >
> > > > >
> > > > > Thanks,
> > > > > Pradeep
> > > >
> > > >
> >
> >
> >

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

[-- Attachment #1: Type: text/plain, Size: 6205 bytes --]

On 6 May 2016 at 09:02, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:

> On Fri, 6 May 2016 08:01:09 +0200
> Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
>
> > On 4 May 2016 at 17:40, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> >
> > > On Mon, 2 May 2016 17:49:26 +0200
> > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > >
> > > > On 2 May 2016 at 14:57, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> > > >
> > > > > On Thu, 28 Apr 2016 11:45:41 +0200
> > > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > > >
> > > > > > On 27 April 2016 at 19:12, Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > wrote:
> > > > > >
> > > > > > > On Wed, 27 Apr 2016 16:39:58 +0200
> > > > > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > > > > >
> > > > > > > > On 27 April 2016 at 10:38, Alberto Garcia <berto@igalia.com>
> > > wrote:
> > > > > > > >
> > > > > > > > > On Wed, Apr 27, 2016 at 09:29:02AM +0200, Pradeep Kiruvale
> > > wrote:
> > > > > > > > >
> > > > > > > > > > Thanks for the reply. I am still in the early phase, I
> will
> > > let
> > > > > you
> > > > > > > > > > know if any changes are needed for the APIs.
> > > > > > > > > >
> > > > > > > > > > We might also have to implement throttle-group.c for 9p
> > > devices,
> > > > > if
> > > > > > > > > > we want to apply throttle for group of devices.
> > > > > > > > >
> > > > > > > > > Fair enough, but again please note that:
> > > > > > > > >
> > > > > > > > > - throttle-group.c is not meant to be generic, but it's
> tied to
> > > > > > > > >   BlockDriverState / BlockBackend.
> > > > > > > > > - it is currently being rewritten:
> > > > > > > > >
> > > > >
> https://lists.gnu.org/archive/html/qemu-block/2016-04/msg00645.html
> > > > > > > > >
> > > > > > > > > If you can explain your use case with a bit more detail we
> can
> > > try
> > > > > to
> > > > > > > > > see what can be done about it.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > We want to use  virtio-9p for block io instead of
> virtio-blk-pci.
> > > > > But in
> > > > > > > > case of
> > > > > > >
> > > > > > > 9p is mostly aimed at sharing files... why would you want to
> use
> > > it for
> > > > > > > block io instead of a true block device ? And how would you do
> > > that ?
> > > > > > >
> > > > > >
> > > > > > *Yes, we want to share the files itself. So we are using the
> > > virtio-9p.*
> > > > >
> > > > > You want to pass a disk image to the guest as a plain file on a 9p
> > > mount ?
> > > > > And then, what do you do in the guest ? Attach it to a loop device
> ?
> > > > >
> > > >
> > > > Yes, would like to mount as  a 9p drive and create file inside that
> and
> > > > read/write.
> > > > This was the experiment we are doing, actual use case no idea. My
> work is
> > > > to do
> > > > a feasibility test does it work or not.
> > > >
> > > >
> > > > >
> > > > > > *We want to have QoS on these files access for every VM.*
> > > > > >
> > > > >
> > > > > You won't be able to have QoS on selected files, but it may be
> > > possible to
> > > > > introduce limits at the fsdev level: control all write accesses to
> all
> > > > > files
> > > > > and all read accesses to all files for a 9p device.
> > > > >
> > > >
> > > > That is right, I do not want to have QoS for individual files but to
> > > whole
> > > > fsdev device.
> > > >
> > > >
> > > > > >
> > > > > > >
> > > > > > > > virtio-9p we can just use fsdev devices, so we want to apply
> > > > > throttling
> > > > > > > > (QoS)
> > > > > > > > on these devices and as of now the io throttling only
> possible
> > > with
> > > > > the
> > > > > > > > -drive option.
> > > > > > > >
> > > > > > >
> > > > > > > Indeed.
> > > > > > >
> > > > > > > > As a work around we are doing the throttling using cgroup.
> It has
> > > > > its own
> > > > > > > > costs.
> > > > > > >
> > > > > > > Can you elaborate ?
> > > > > > >
> > > > > >
> > > > > > *We saw that we need to create cgroups and set it and also we
> > > observed
> > > > > lot
> > > > > > of iowaits *
> > > > > > *compared to implementing the throttling inside the qemu.*
> > > > > > *This we did observe by using the virtio-blk-pci devices. (Using
> > > cgroups
> > > > > Vs
> > > > > > qemu throttling)*
> > > > > >
> > > > >
> > > >
> > > >
> > > > >
> > > > > Just to be sure I get it right.
> > > > >
> > > > > You tried both:
> > > > > 1) run QEMU with -device virtio-blk-pci and -drive throttling.*
> > > > > 2) run QEMU with -device virtio-blk-pci in its own cgroup
> > > > >
> > > > > And 1) has better performance and is easier to use than 2) ?
> > > > >
> > > > > And what do you expect with 9p compared to 1) ?
> > > > >
> > > > >
> > > > That was just to understand the cost of cpu
> > > >  io throttling inside the qemu vs using cgroup.
> > > >
> > > > The bench-marking we did to reproduce the numbers and understand the
> cost
> > > > mentioned in
> > > >
> > > >
> > >
> http://www.linux-kvm.org/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf
> > > >
> > > > Thanks,
> > > > Pradeep
> > > >
> > >
> > > Ok. So you did compare current QEMU block I/O throttling with cgroup ?
> And
> > > you observed numbers
> > > similar to the link above ?
> > >
> >
> > *Yes, I did, I did run DD command in guest to do IO. The recent QEMU is
> in
> > par with cgroups in terms *
> > *of CPU utilization.*
> >
> > >
> > > And now you would like to run the same test on a file in a 9p mount
> with
> > > experimental 9p QoS ?
> > >
> > > *Yes, you are right.*
> >
> >
> > > Maybe possible to reuse the throttle.h API and hack v9fs_write() and
> > > v9fs_read() in 9p.c then.
> > >
> > >
> > *OK, I am looking into it. Are there any sample test cases or something
> > about how to apply the*
> > *throttling APIs to a device?*
> >
>
> The throttling API is currently only used by block devices, and the only
> documentation out-there is the code itself...
>

Thanks, I will have a look and get back to you if I have any further
questions regarding this.

Regards,
Pradeep



>
> >
> > Regards,
> > Pradeep
> >
> >
> >
> > > Cheers.
> > >
> > > --
> > > Greg
> > >
> > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Pradeep
> > > > >
> > > > >
> > >
> > >
> > >
>
>

[-- Attachment #2: Type: text/html, Size: 10321 bytes --]

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Alberto Garcia]

On Fri 06 May 2016 09:39:13 AM CEST, Pradeep Kiruvale wrote:

>> The throttling API is currently only used by block devices, and the
>> only documentation out-there is the code itself...
>
> Thanks, I will have a look and get back to you if I have any further
> questions regarding this.

As I said in an earlier e-mail:

1. the common API is in throttle.h.

2. Once you configure the throttling settings you essentially only need
   to call throttle_schedule_timer() to see if a request needs to be
   throttled, and afterwards throttle_account() to register the I/O that
   has been peformed.

Berto

Re: [Qemu-devel] [Qemu-discuss] iolimits for virtio-9p

[Pradeep Kiruvale]

Thanks Alberto.

Regards,
Pradeep

On 6 May 2016 at 14:16, Alberto Garcia <berto@igalia.com> wrote:

> On Fri 06 May 2016 09:39:13 AM CEST, Pradeep Kiruvale wrote:
>
> >> The throttling API is currently only used by block devices, and the
> >> only documentation out-there is the code itself...
> >
> > Thanks, I will have a look and get back to you if I have any further
> > questions regarding this.
>
> As I said in an earlier e-mail:
>
> 1. the common API is in throttle.h.
>
> 2. Once you configure the throttling settings you essentially only need
>    to call throttle_schedule_timer() to see if a request needs to be
>    throttled, and afterwards throttle_account() to register the I/O that
>    has been peformed.
>
> Berto
>