From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49127) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1apcgK-0003tm-DX for qemu-devel@nongnu.org; Mon, 11 Apr 2016 10:19:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1apcgF-0003SH-KQ for qemu-devel@nongnu.org; Mon, 11 Apr 2016 10:19:12 -0400 MIME-Version: 1.0 In-Reply-To: <20160408165801.1d93b372@bahia.huguette.org> References: <20160408101054.14b77747@bahia.huguette.org> <20160408144056.3793c712@bahia.huguette.org> <20160408165801.1d93b372@bahia.huguette.org> Date: Mon, 11 Apr 2016 16:18:53 +0200 Message-ID: From: Pradeep Kiruvale Content-Type: multipart/alternative; boundary=001a11422abc76c69b0530363911 Subject: Re: [Qemu-devel] Virtio-9p and cgroup io-throttling List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Greg Kurz Cc: qemu-devel@nongnu.org, "qemu-discuss@nongnu.org" --001a11422abc76c69b0530363911 Content-Type: text/plain; charset=UTF-8 Hi Greg, Yes, it was nothing to do with the virtio-9p. I was writing at 4k Blocks now I changed it to 1K Block. This works fine for me. Thanks for your help. Regards, Pradeep On 8 April 2016 at 16:58, Greg Kurz wrote: > On Fri, 8 Apr 2016 14:55:29 +0200 > Pradeep Kiruvale wrote: > > > Hi Greg, > > > > FInd my replies inline > > > > > > > > > Below is the way how I add to blkio > > > > > > > > echo "8:16 8388608" > > > > > /sys/fs/cgroup/blkio/test/blkio.throttle.write_bps_device > > > > > > > > > > Ok, this just puts a limit of 8MB/s when writing to /dev/sdb for all > > > tasks in the test cgroup... but what about the tasks themselves ? > > > > > > > The problem I guess is adding these task ids to the "tasks" file in > > > cgroup > > > > > > > > > > Exactly. :) > > > > > > > These threads are started randomly and even then I add the PIDs to > the > > > > tasks file the cgroup still does not do IO control. > > > > > > > > > > How did you get the PIDs ? Are you sure these threads you have added > to the > > > cgroup are the ones that write to /dev/sdb ? > > > > > > > *Yes, I get PIDs from /proc/Qemu_PID/task* > > > > And then you echoed the PIDs to /sys/fs/cgroup/blkio/test/tasks ? > > This is racy... another IO thread may be started to do some work on > /dev/sdb > just after you've read PIDs from /proc/Qemu_PID/task, and it won't be part > of the cgroup. > > > > > > > > > > > > Is it possible to reduce these number of threads? I see different > number > > > of > > > > threads doing IO at different runs. > > > > > > > > > > AFAIK, no. > > > > > > Why don't you simply start QEMU in the cgroup ? Unless I miss > something, > > > all > > > children threads, including the 9p ones, will be in the cgroup and > honor > > > the > > > throttle setttings. > > > > > > > > > *I started the qemu with cgroup as below* > > > > *cgexec -g blkio:/test qemu.......* > > *Is there any other way of starting the qemu in cgroup?* > > > > Maybe you can pass --sticky to cgexec to prevent cgred from moving > children tasks to other cgroups... > > There's also the old fashion method: > > # echo $$ > /sys/fs/cgroup/blkio/test/tasks > # qemu..... > > This being said, QEMU is a regular userspace program that is completely > cgroup > agnostic. It won't behave differently than 'dd if=/dev/sdb of=/dev/null'. > > This really doesn't look like a QEMU related issue to me. > > > Regards, > > Pradeep > > > > Cheers. > > -- > Greg > > > > > > > > > > Regards, > > > > Pradeep > > > > > > > > > > Cheers. > > > > > > -- > > > Greg > > > > > > > > > > > On 8 April 2016 at 10:10, Greg Kurz > wrote: > > > > > > > > > On Thu, 7 Apr 2016 11:48:27 +0200 > > > > > Pradeep Kiruvale wrote: > > > > > > > > > > > Hi All, > > > > > > > > > > > > I am using virtio-9p for sharing the file between host and > guest. To > > > test > > > > > > the shared file I do read/write options in the guest.To have > > > controlled > > > > > io, > > > > > > I am using cgroup blkio. > > > > > > > > > > > > While using cgroup I am facing two issues,Please find the issues > > > below. > > > > > > > > > > > > 1. When I do IO throttling using the cgroup the read throttling > works > > > > > fine > > > > > > but the write throttling does not wok. It still bypasses these > > > throttling > > > > > > control and does the default, am I missing something here? > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > Can you provide details on your blkio setup ? > > > > > > > > > > > I use the following commands to create VM, share the files and to > > > > > > read/write from guest. > > > > > > > > > > > > *Create vm* > > > > > > qemu-system-x86_64 -balloon none .......-name vm0 -cpu host -m > 128 > > > -smp 1 > > > > > > -enable-kvm -parallel .... -fsdev > > > > > > > local,id=sdb1,path=/mnt/sdb1,security_model=none,writeout=immediate > > > > > -device > > > > > > virtio-9p-pci,fsdev=sdb1,mount_tag=sdb1 > > > > > > > > > > > > *Mount file* > > > > > > mount -t 9p -o trans=virtio,version=9p2000.L sdb1 /sdb1_ext4 > > > 2>>dd.log && > > > > > > sync > > > > > > > > > > > > touch /sdb1_ext4/dddrive > > > > > > > > > > > > *Write test* > > > > > > dd if=/dev/zero of=/sdb1_ext4/dddrive bs=4k count=800000 > > > oflag=direct >> > > > > > > dd.log 2>&1 && sync > > > > > > > > > > > > *Read test* > > > > > > dd if=/sdb1_ext4/dddrive of=/dev/null >> dd.log 2>&1 && sync > > > > > > > > > > > > 2. The other issue is when I run "dd" command inside guest it > > > creates > > > > > > multiple threads to write/read. I can see those on host using > iotop > > > is > > > > > this > > > > > > expected behavior? > > > > > > > > > > > > > > > > Yes. QEMU uses a thread pool to handle 9p requests. > > > > > > > > > > > Regards, > > > > > > Pradeep > > > > > > > > > > Cheers. > > > > > > > > > > -- > > > > > Greg > > > > > > > > > > > > > > > > > > > --001a11422abc76c69b0530363911 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Greg,

Yes, it was nothing to do with= the virtio-9p. I was writing at 4k Blocks now I changed it to 1K Block. Th= is works fine for me.

Thanks for your help.
<= div>
Regards,
Pradeep

On 8 April 2016 at 16:58, Greg Kurz= <gkurz@linux.vnet.ibm.com> wrote:
On Fri, 8 Apr 2016 14:55:29 +0200
Pradeep Kiruvale <pradeepki= ruvale@gmail.com> wrote:

> Hi Greg,
>
> FInd my replies inline
>
> >
> > > Below is the way how I add to blkio
> > >
> > > echo "8:16 8388608" >
> > > /sys/fs/cgroup/blkio/test/blkio.throttle.write_bps_device > > >
> >
> > Ok, this just puts a limit of 8MB/s when writing to /dev/sdb for = all
> > tasks in the test cgroup... but what about the tasks themselves ?=
> >
> > > The problem I guess is adding these task ids to the "ta= sks" file in
> > cgroup
> > >
> >
> > Exactly. :)
> >
> > > These threads are started randomly and even then I add the P= IDs to the
> > > tasks file the cgroup still does not do IO control.
> > >
> >
> > How did you get the PIDs ? Are you sure these threads you have ad= ded to the
> > cgroup are the ones that write to /dev/sdb ?
> >
>
> *Yes, I get PIDs from /proc/Qemu_PID/task*
>

And then you echoed the PIDs to /sys/fs/cgroup/blkio/test/tasks ?

This is racy... another IO thread may be started to do some work on /dev/sd= b
just after you've read PIDs from /proc/Qemu_PID/task, and it won't = be part
of the cgroup.

>
>
> >
> > > Is it possible to reduce these number of threads? I see diff= erent number
> > of
> > > threads doing IO at different runs.
> > >
> >
> > AFAIK, no.
> >
> > Why don't you simply start QEMU in the cgroup ? Unless I miss= something,
> > all
> > children threads, including the 9p ones, will be in the cgroup an= d honor
> > the
> > throttle setttings.
> >
>
>
> *I started the qemu with cgroup as below*
>
> *cgexec -g blkio:/test qemu.......*
> *Is there any other way of starting the qemu in cgroup?*
>

Maybe you can pass --sticky to cgexec to prevent cgred from moving
children tasks to other cgroups...

There's also the old fashion method:

# echo $$ > /sys/fs/cgroup/blkio/test/tasks
# qemu.....

This being said, QEMU is a regular userspace program that is completely cgr= oup
agnostic. It won't behave differently than 'dd if=3D/dev/sdb of=3D/= dev/null'.

This really doesn't look like a QEMU related issue to me.

> Regards,
> Pradeep
>

Cheers.

--
Greg

>
> >
> > > Regards,
> > > Pradeep
> > >
> >
> > Cheers.
> >
> > --
> > Greg
> >
> > >
> > > On 8 April 2016 at 10:10, Greg Kurz <gkurz@linux.vnet.ibm.com> wrote:
> > >
> > > > On Thu, 7 Apr 2016 11:48:27 +0200
> > > > Pradeep Kiruvale <pradeepkiruvale@gmail.com> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > > I am using virtio-9p for sharing the file between = host and guest. To
> > test
> > > > > the shared file I do read/write options in the gue= st.To have
> > controlled
> > > > io,
> > > > > I am using cgroup blkio.
> > > > >
> > > > > While using cgroup I am facing two issues,Please f= ind the issues
> > below.
> > > > >
> > > > > 1. When I do IO throttling using the cgroup the re= ad throttling works
> > > > fine
> > > > > but the write throttling does not wok. It still by= passes these
> > throttling
> > > > > control and does the default, am I missing somethi= ng here?
> > > > >
> > > >
> > > > Hi,
> > > >
> > > > Can you provide details on your blkio setup ?
> > > >
> > > > > I use the following commands to create VM, share t= he files and to
> > > > > read/write from guest.
> > > > >
> > > > > *Create vm*
> > > > > qemu-system-x86_64 -balloon none .......-name vm0 = -cpu host -m 128
> > -smp 1
> > > > > -enable-kvm -parallel .... -fsdev
> > > > > local,id=3Dsdb1,path=3D/mnt/sdb1,security_model=3D= none,writeout=3Dimmediate
> > > > -device
> > > > > virtio-9p-pci,fsdev=3Dsdb1,mount_tag=3Dsdb1
> > > > >
> > > > > *Mount file*
> > > > > mount -t 9p -o trans=3Dvirtio,version=3D9p2000.L s= db1 /sdb1_ext4
> > 2>>dd.log &&
> > > > > sync
> > > > >
> > > > > touch /sdb1_ext4/dddrive
> > > > >
> > > > > *Write test*
> > > > > dd if=3D/dev/zero of=3D/sdb1_ext4/dddrive bs=3D4k = count=3D800000
> > oflag=3Ddirect >>
> > > > > dd.log 2>&1 && sync
> > > > >
> > > > > *Read test*
> > > > > dd if=3D/sdb1_ext4/dddrive of=3D/dev/null >>= dd.log 2>&1 && sync
> > > > >
> > > > > 2. The other issue is when I run "dd" co= mmand inside guest=C2=A0 it
> > creates
> > > > > multiple threads to write/read. I can see those on= host using iotop
> > is
> > > > this
> > > > > expected behavior?
> > > > >
> > > >
> > > > Yes. QEMU uses a thread pool to handle 9p requests.
> > > >
> > > > > Regards,
> > > > > Pradeep
> > > >
> > > > Cheers.
> > > >
> > > > --
> > > > Greg
> > > >
> > > >
> >
> >



--001a11422abc76c69b0530363911--