[Qemu-devel] poor virtio-scsi performance

[Qemu-devel] poor virtio-scsi performance

[Vasiliy Tolstov]

Hi all!

I suspected poor performance of virtio-scsi driver.
I did a few tests:
   Host machine: linux 3.19.1, QEMU emulator version 2.3.0
   Guest machine: linux 4.0.4

   part of domain xml:
    <emulator>/usr/bin/kvm</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native' discard='unmap'/>
      <source dev='/dev/ram0'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <alias name='scsi0-0-0-1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>

/dev/ram0 I got by running `modprobe brd rd_size=$((5*1024*1024))` on
host machine.

fio conf:
  [readtest]
  blocksize=4k
  filename=/dev/sdb (/dev/ram0 whe test from host machine)
  rw=randread
  direct=1
  buffered=0
  ioengine=libaio
  iodepth=32


results:
  from host:
    bw=1594.6MB/s, iops=408196, clat=76usec
  from guest:
    bw=398MB/s, iops=99720, clat=316usec

Both host and guest system I boot with `scsi_mod.use_blk_mq=Y`.

Why difference in 4 times?!

-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru

Re: [Qemu-devel] poor virtio-scsi performance

[Alexandre DERUMIER]

Hi,
if you want to use multiqueues in guest, you need to enabled it on virtio-scsi controller.

<controller type='scsi' index='0' model='virtio-scsi' num_queues='8'/>

for example.


----- Mail original -----
De: "Vasiliy Tolstov" <v.tolstov@selfip.ru>
À: "qemu-devel" <qemu-devel@nongnu.org>, libvir-list@redhat.com
Envoyé: Lundi 8 Juin 2015 12:30:59
Objet: [Qemu-devel] poor virtio-scsi performance

Hi all! 

I suspected poor performance of virtio-scsi driver. 
I did a few tests: 
Host machine: linux 3.19.1, QEMU emulator version 2.3.0 
Guest machine: linux 4.0.4 

part of domain xml: 
<emulator>/usr/bin/kvm</emulator> 
<disk type='block' device='disk'> 
<driver name='qemu' type='raw' cache='none' io='native' discard='unmap'/> 
<source dev='/dev/ram0'/> 
<backingStore/> 
<target dev='sda' bus='scsi'/> 
<alias name='scsi0-0-0-1'/> 
<address type='drive' controller='0' bus='0' target='0' unit='1'/> 
</disk> 

/dev/ram0 I got by running `modprobe brd rd_size=$((5*1024*1024))` on 
host machine. 

fio conf: 
[readtest] 
blocksize=4k 
filename=/dev/sdb (/dev/ram0 whe test from host machine) 
rw=randread 
direct=1 
buffered=0 
ioengine=libaio 
iodepth=32 


results: 
from host: 
bw=1594.6MB/s, iops=408196, clat=76usec 
from guest: 
bw=398MB/s, iops=99720, clat=316usec 

Both host and guest system I boot with `scsi_mod.use_blk_mq=Y`. 

Why difference in 4 times?! 

-- 
Vasiliy Tolstov, 
e-mail: v.tolstov@selfip.ru 

Re: [Qemu-devel] poor virtio-scsi performance

[Paolo Bonzini]

On 08/06/2015 16:00, Vasiliy Tolstov wrote:
> 2015-06-08 16:46 GMT+03:00 Paolo Bonzini <pbonzini@redhat.com>:
>> > virtio-scsi dataplane is not yet 100% thread-safe, but in practice it
>> > should work in 2.3+.  It's definitely good enough for benchmarking.
> 
> Minimal qemu version is 2.3 or something from git?

2.3

> Also how can i enable dataplane ?

Replace "-device virtio-scsi-pci" with "-object iothread,id=io1 -device
virtio-scsi-pci,iothread=io1".

Paolo

Re: [Qemu-devel] poor virtio-scsi performance

[Vasiliy Tolstov]

2015-06-08 16:46 GMT+03:00 Paolo Bonzini <pbonzini@redhat.com>:
> virtio-scsi dataplane is not yet 100% thread-safe, but in practice it
> should work in 2.3+.  It's definitely good enough for benchmarking.


Minimal qemu version is 2.3 or something from git?
Also how can i enable dataplane ?

-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru

Re: [Qemu-devel] poor virtio-scsi performance

[Vasiliy Tolstov]

2015-06-08 16:10 GMT+03:00 Andrey Korolyov <andrey@xdel.ru>:
> You are probably hitting the (more or less fundamental) ceiling which
> was the reason for introducing dataplane backend recently, in other
> words, it does not matter how fast your backend is, the operation
> number will be limited by 50..100kIOPS without dataplane. As far as I
> remember the recent development made possible using it with
> virtio-scsi as well as with virtio-blk, so you`d possibly want to try
> it.


I can't find dataplane for virtio-scsi, only for virtio-blk.

-- 
Vasiliy Tolstov,
e-mail: v.tolstov@selfip.ru

Re: [Qemu-devel] poor virtio-scsi performance

[Paolo Bonzini]

On 08/06/2015 15:10, Andrey Korolyov wrote:
> 
> You are probably hitting the (more or less fundamental) ceiling which
> was the reason for introducing dataplane backend recently, in other
> words, it does not matter how fast your backend is, the operation
> number will be limited by 50..100kIOPS without dataplane. As far as I
> remember the recent development made possible using it with
> virtio-scsi as well as with virtio-blk, so you`d possibly want to try
> it.

virtio-scsi dataplane is not yet 100% thread-safe, but in practice it
should work in 2.3+.  It's definitely good enough for benchmarking.

Paolo

Re: [Qemu-devel] poor virtio-scsi performance

[Andrey Korolyov]

On Mon, Jun 8, 2015 at 12:52 PM, Alexey <alukardd@alukardd.org> wrote:
> Hi all!
>
> I suspected poor performance of virtio-scsi driver.
> I did a few tests:
>    Host machine: linux 3.19.1, QEMU emulator version 2.3.0
>    Guest machine: linux 4.0.4
>
>    part of domain xml:
>     <emulator>/usr/bin/kvm</emulator>
>     <disk type='block' device='disk'>
>       <driver name='qemu' type='raw' cache='none' io='native'
> discard='unmap'/>
>       <source dev='/dev/ram0'/>
>       <backingStore/>
>       <target dev='sda' bus='scsi'/>
>       <alias name='scsi0-0-0-1'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='1'/>
>     </disk>
>
> /dev/ram0 I got by running `modprobe brd rd_size=$((5*1024*1024))` on host
> machine.
>
> fio conf:
>   [readtest]
>   blocksize=4k
>   filename=/dev/sdb (/dev/ram0 whe test from host machine)
>   rw=randread
>   direct=1
>   buffered=0
>   ioengine=libaio
>   iodepth=32
>
>
> results:
>   from host:
>     bw=1594.6MB/s, iops=408196, clat=76usec
>   from guest:
>     bw=398MB/s, iops=99720, clat=316usec
>
> Both host and guest system I boot with `scsi_mod.use_blk_mq=Y`.
>
> Why difference in 4 times?!
>
>
> Regards,
> Alexey Mochkin
>


You are probably hitting the (more or less fundamental) ceiling which
was the reason for introducing dataplane backend recently, in other
words, it does not matter how fast your backend is, the operation
number will be limited by 50..100kIOPS without dataplane. As far as I
remember the recent development made possible using it with
virtio-scsi as well as with virtio-blk, so you`d possibly want to try
it.

[Qemu-devel] poor virtio-scsi performance

[Alexey]

Hi all!

I suspected poor performance of virtio-scsi driver.
I did a few tests:
    Host machine: linux 3.19.1, QEMU emulator version 2.3.0
    Guest machine: linux 4.0.4

    part of domain xml:
     <emulator>/usr/bin/kvm</emulator>
     <disk type='block' device='disk'>
       <driver name='qemu' type='raw' cache='none' io='native' 
discard='unmap'/>
       <source dev='/dev/ram0'/>
       <backingStore/>
       <target dev='sda' bus='scsi'/>
       <alias name='scsi0-0-0-1'/>
       <address type='drive' controller='0' bus='0' target='0' unit='1'/>
     </disk>

/dev/ram0 I got by running `modprobe brd rd_size=$((5*1024*1024))` on 
host machine.

fio conf:
   [readtest]
   blocksize=4k
   filename=/dev/sdb (/dev/ram0 whe test from host machine)
   rw=randread
   direct=1
   buffered=0
   ioengine=libaio
   iodepth=32


results:
   from host:
     bw=1594.6MB/s, iops=408196, clat=76usec
   from guest:
     bw=398MB/s, iops=99720, clat=316usec

Both host and guest system I boot with `scsi_mod.use_blk_mq=Y`.

Why difference in 4 times?!


Regards,
Alexey Mochkin

[Qemu-devel] poor virtio-scsi performance

[Alexey]

Hi all!

I suspected poor performance of virtio-scsi driver.
I did a few tests:
    Host machine: linux 3.19.1, QEMU emulator version 2.3.0
    Guest machine: linux 4.0.4

    part of domain xml:
     <emulator>/usr/bin/kvm</emulator>
     <disk type='block' device='disk'>
       <driver name='qemu' type='raw' cache='none' io='native' 
discard='unmap'/>
       <source dev='/dev/ram0'/>
       <backingStore/>
       <target dev='sda' bus='scsi'/>
       <alias name='scsi0-0-0-1'/>
       <address type='drive' controller='0' bus='0' target='0' unit='1'/>
     </disk>

/dev/ram0 I got by running `modprobe brd rd_size=$((5*1024*1024))` on 
host machine.

fio conf:
   [readtest]
   blocksize=4k
   filename=/dev/sdb (/dev/ram0 whe test from host machine)
   rw=randread
   direct=1
   buffered=0
   ioengine=libaio
   iodepth=32


results:
   from host:
     bw=1594.6MB/s, iops=408196, clat=76usec
   from guest:
     bw=398MB/s, iops=99720, clat=316usec

Both host and guest system I boot with `scsi_mod.use_blk_mq=Y`.

Why difference in 4 times?!


Regards,
Alexey Mochkin