[SPDK] NVME-OF RDMA scalability

[SPDK] NVME-OF RDMA scalability

[Sasha Kotchubievsky]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

Hi,

I submitted a while ago a patch adding SRQ (Shared receive queue) to 
NVME-OF RDMA solution.

https://review.gerrithub.io/c/spdk/spdk/+/428458

SRQ improves RDMA solution scalability. All QPs, associated with the 
same polling group, can use shared pool of IBV work requests. In 
scenarios when multiple senders send date to the same receiver it allow 
significantly reduce memory consumption in the receiver. Actually, 
incoming amount of data is limited by receiver port bandwidth and the 
receiver can allocate buffers based on this bandwidth for all QPs.

You can find some results and calculations in the patch's message.

Is it possible to mark this patch to 19.01 release and move forward with 
code review?

Linux kernel implementation for NVME-OF RDMA use SRQ as well.

SRQ support, at this stage is conditional (compilation flag) and 
shouldn't break existing approach.


Best regards

Sasha

Re: [SPDK] NVME-OF RDMA scalability

[Sasha Kotchubievsky]

[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]

Hi Seth,

Thanks.

I'll answer questions and comment on concerns raised in code review in 
Gerrit.

BR

Sasha

On 1/18/2019 5:29 PM, Howell, Seth wrote:
> Hi Sasha,
>
> I have added the hashtag to your changes in the patch series related to shared receive queue as requested and will give them a review today.
>
> Thanks,
>
> Seth
>
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Sasha Kotchubievsky
> Sent: Thursday, January 17, 2019 1:50 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] NVME-OF RDMA scalability
>
> Hi,
>
> I submitted a while ago a patch adding SRQ (Shared receive queue) to NVME-OF RDMA solution.
>
> https://review.gerrithub.io/c/spdk/spdk/+/428458
>
> SRQ improves RDMA solution scalability. All QPs, associated with the same polling group, can use shared pool of IBV work requests. In scenarios when multiple senders send date to the same receiver it allow significantly reduce memory consumption in the receiver. Actually, incoming amount of data is limited by receiver port bandwidth and the receiver can allocate buffers based on this bandwidth for all QPs.
>
> You can find some results and calculations in the patch's message.
>
> Is it possible to mark this patch to 19.01 release and move forward with code review?
>
> Linux kernel implementation for NVME-OF RDMA use SRQ as well.
>
> SRQ support, at this stage is conditional (compilation flag) and shouldn't break existing approach.
>
>
> Best regards
>
> Sasha
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] NVME-OF RDMA scalability

[Howell, Seth]

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

Hi Sasha,

I have added the hashtag to your changes in the patch series related to shared receive queue as requested and will give them a review today.

Thanks,

Seth

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Sasha Kotchubievsky
Sent: Thursday, January 17, 2019 1:50 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] NVME-OF RDMA scalability

Hi,

I submitted a while ago a patch adding SRQ (Shared receive queue) to NVME-OF RDMA solution.

https://review.gerrithub.io/c/spdk/spdk/+/428458

SRQ improves RDMA solution scalability. All QPs, associated with the same polling group, can use shared pool of IBV work requests. In scenarios when multiple senders send date to the same receiver it allow significantly reduce memory consumption in the receiver. Actually, incoming amount of data is limited by receiver port bandwidth and the receiver can allocate buffers based on this bandwidth for all QPs.

You can find some results and calculations in the patch's message.

Is it possible to mark this patch to 19.01 release and move forward with code review?

Linux kernel implementation for NVME-OF RDMA use SRQ as well.

SRQ support, at this stage is conditional (compilation flag) and shouldn't break existing approach.


Best regards

Sasha

_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk