[SPDK] Re: About single virtio-blk device performance

[SPDK] Re: About single virtio-blk device performance

[Liu, Changpeng]

[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

Hi Bin Yang,

I think that's the only benefit by using multiple cores for a virtio-blk device, if your tests case is for a VM, 
Actually you can add multiple devices to this VM.


> -----Original Message-----
> From: Bin Yang <bin.yang(a)jaguarmicro.com>
> Sent: Wednesday, October 12, 2022 3:02 PM
> To: spdk(a)lists.01.org
> Subject: [SPDK] About single virtio-blk device performance
> 
> Hello, Everyone!
> 
>     For spdk vhost_blk，can we distribute multi queues requests in different cpu
> to improve single virtio-blk device performance?
> for example，create request poller for each queue and distribute the pollers in
> different threads.  Does this feasible？
> 
> Thanks!
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: About single virtio-blk device performance

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 4572 bytes --]

The SPDK vhost target only supports one thread per device because virtio-scsi requires shared state between queues (it's the scsi part at fault), so in order to use multiple queues they'd need to take locks and that's less efficient than just running it all on one thread. The virtio-blk part really could take advantage of multiple virtqueues and fan out, but since it's lumped together with virtio-scsi that makes it quite difficult.

I think, in hindsight, we likely should not have grouped virtio-scsi in with virtio-blk, and instead made a virtio-blk target separate from a virtio-scsi target, both of which had vhost transports.

Thanks,
Ben

> -----Original Message-----
> From: Rui Chang <Rui.Chang(a)arm.com>
> Sent: Friday, October 14, 2022 12:55 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Re: About single virtio-blk device performance
> 
> An interesting topic. Personally, I think we should not assume people are
> using SPDK vhost with VMs that only have very few cores. If the core number
> is a little bit larger, it could easily saturate vhost which only have one core.
> Further, since we support multiple VMs to connect one vhost controller, if
> the VM count is 4, 8, or more, it could also saturate one vhost controller
> easily. Isn't it?
> 
> Regards,
> Rui
> 
> -----Original Message-----
> From: Liu, Changpeng <changpeng.liu(a)intel.com>
> Sent: Friday, October 14, 2022 3:03 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Re: About single virtio-blk device performance
> 
> From my own perspective, I don't like the idea to poll vrings from different
> cores for one virtio device, I don't see benefits when supporting multiple
> VMs with limited number of cores, this is a very rare test case for SPDK vhost.
> 
> > -----Original Message-----
> > From: Bin Yang <bin.yang(a)jaguarmicro.com>
> > Sent: Friday, October 14, 2022 11:19 AM
> > To: Storage Performance Development Kit <spdk(a)lists.01.org>
> > Subject: [SPDK] 回复: About single virtio-blk device performance
> >
> > Hi Changpeng:
> >       Thanks for your reply!
> >       Will community accept the commits as using multiple cores for a
> > virtio-blk device?
> >       If not,  what are the main concerns?
> >
> > ________________________________
> > 发件人: Liu, Changpeng <changpeng.liu(a)intel.com>
> > 发送时间: 2022年10月13日 15:16
> > 收件人: Storage Performance Development Kit <spdk(a)lists.01.org>
> > 主题: [SPDK] Re: About single virtio-blk device performance
> >
> > Hi Bin Yang,
> >
> > I think that's the only benefit by using multiple cores for a
> > virtio-blk device, if your tests case is for a VM, Actually you can
> > add multiple devices to this VM.
> >
> >
> > > -----Original Message-----
> > > From: Bin Yang <bin.yang(a)jaguarmicro.com>
> > > Sent: Wednesday, October 12, 2022 3:02 PM
> > > To: spdk(a)lists.01.org
> > > Subject: [SPDK] About single virtio-blk device performance
> > >
> > > Hello, Everyone!
> > >
> > >     For spdk vhost_blk，can we distribute multi queues requests in
> > > different cpu to improve single virtio-blk device performance?
> > > for example，create request poller for each queue and distribute the
> > > pollers in different threads.  Does this feasible？
> > >
> > > Thanks!
> > > _______________________________________________
> > > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email
> > > to spdk-leave(a)lists.01.org
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> > spdk-leave(a)lists.01.org
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> > spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org IMPORTANT
> NOTICE: The contents of this email and any attachments are confidential and
> may also be privileged. If you are not the intended recipient, please notify
> the sender immediately and do not disclose the contents to any other
> person, use it for any purpose, or store or copy the information in any
> medium. Thank you.
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: About single virtio-blk device performance

[Rui Chang]

[-- Attachment #1: Type: text/plain, Size: 3348 bytes --]

An interesting topic. Personally, I think we should not assume people are using SPDK vhost with VMs that only have very few cores. If the core number is a little bit larger, it could easily saturate vhost which only have one core. Further, since we support multiple VMs to connect one vhost controller, if the VM count is 4, 8, or more, it could also saturate one vhost controller easily. Isn't it?

Regards,
Rui

-----Original Message-----
From: Liu, Changpeng <changpeng.liu(a)intel.com>
Sent: Friday, October 14, 2022 3:03 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: About single virtio-blk device performance

From my own perspective, I don't like the idea to poll vrings from different cores for one virtio device, I don't see benefits when supporting multiple VMs with limited number of cores, this is a very rare test case for SPDK vhost.

> -----Original Message-----
> From: Bin Yang <bin.yang(a)jaguarmicro.com>
> Sent: Friday, October 14, 2022 11:19 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] 回复: About single virtio-blk device performance
>
> Hi Changpeng:
>       Thanks for your reply!
>       Will community accept the commits as using multiple cores for a
> virtio-blk device?
>       If not,  what are the main concerns?
>
> ________________________________
> 发件人: Liu, Changpeng <changpeng.liu(a)intel.com>
> 发送时间: 2022年10月13日 15:16
> 收件人: Storage Performance Development Kit <spdk(a)lists.01.org>
> 主题: [SPDK] Re: About single virtio-blk device performance
>
> Hi Bin Yang,
>
> I think that's the only benefit by using multiple cores for a
> virtio-blk device, if your tests case is for a VM, Actually you can
> add multiple devices to this VM.
>
>
> > -----Original Message-----
> > From: Bin Yang <bin.yang(a)jaguarmicro.com>
> > Sent: Wednesday, October 12, 2022 3:02 PM
> > To: spdk(a)lists.01.org
> > Subject: [SPDK] About single virtio-blk device performance
> >
> > Hello, Everyone!
> >
> >     For spdk vhost_blk，can we distribute multi queues requests in
> > different cpu to improve single virtio-blk device performance?
> > for example，create request poller for each queue and distribute the
> > pollers in different threads.  Does this feasible？
> >
> > Thanks!
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email
> > to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

[SPDK] Re: About single virtio-blk device performance

[Liu, Changpeng]

[-- Attachment #1: Type: text/plain, Size: 2216 bytes --]

From my own perspective, I don't like the idea to poll vrings from different
cores for one virtio device, I don't see benefits when supporting multiple VMs
with limited number of cores, this is a very rare test case for SPDK vhost.

> -----Original Message-----
> From: Bin Yang <bin.yang(a)jaguarmicro.com>
> Sent: Friday, October 14, 2022 11:19 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] 回复: About single virtio-blk device performance
> 
> Hi Changpeng:
>       Thanks for your reply!
>       Will community accept the commits as using multiple cores for a virtio-blk
> device?
>       If not,  what are the main concerns?
> 
> ________________________________
> 发件人: Liu, Changpeng <changpeng.liu(a)intel.com>
> 发送时间: 2022年10月13日 15:16
> 收件人: Storage Performance Development Kit <spdk(a)lists.01.org>
> 主题: [SPDK] Re: About single virtio-blk device performance
> 
> Hi Bin Yang,
> 
> I think that's the only benefit by using multiple cores for a virtio-blk device, if your
> tests case is for a VM,
> Actually you can add multiple devices to this VM.
> 
> 
> > -----Original Message-----
> > From: Bin Yang <bin.yang(a)jaguarmicro.com>
> > Sent: Wednesday, October 12, 2022 3:02 PM
> > To: spdk(a)lists.01.org
> > Subject: [SPDK] About single virtio-blk device performance
> >
> > Hello, Everyone!
> >
> >     For spdk vhost_blk，can we distribute multi queues requests in different cpu
> > to improve single virtio-blk device performance?
> > for example，create request poller for each queue and distribute the pollers in
> > different threads.  Does this feasible？
> >
> > Thanks!
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org
> > To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org