* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 7:45 Andrey Kuzmin
0 siblings, 0 replies; 11+ messages in thread
From: Andrey Kuzmin @ 2018-03-22 7:45 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1789 bytes --]
On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
> How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev
> will call it multiple times as the spdk_nvme_cmd_cb callback will move
> buffer pointers forward, if I got the code right.
>
Then I'm clearly wrong with the intention behind the *v command flavor. In
NVMe, those could be natively turned into PRP or SGL filled in, followed by
the single submission, enjoying a single cimpletion event. If the code is
like you say, it doesn't matter who loops over the iovs, the caller or the
callee.
Regards,
A.
>
> On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>>
>>
>> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> In case I have to read/write iovecs, is there any advantage of using
>>> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
>>> call spdk_nvme_ns_cmd_write?
>>>
>>
>> The apparent proprotional reduction of the number of
>> submission/completion events per I/O. Although, with SPDK, those are much
>> cheaper than with the kernel, every penny counts ;).
>>
>> Regards,
>> Andrey
>>
>>>
>>> Sincerely,
>>> E.
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>> --
>>
>> Regards,
>> Andrey
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
--
Regards,
Andrey
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 4638 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 13:41 Ernest Zed
0 siblings, 0 replies; 11+ messages in thread
From: Ernest Zed @ 2018-03-22 13:41 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4389 bytes --]
Thanks!
On Thu, Mar 22, 2018 at 3:09 PM, Wodkowski, PawelX <
pawelx.wodkowski(a)intel.com> wrote:
> Correct.
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Ernest Zed
> *Sent:* Thursday, March 22, 2018 12:31 PM
>
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over
> spdk_nvme_ns_cmd_write
>
>
>
> Pawel, do you mean all these _nvme_ns_cmd_split_request_*
> in _nvme_ns_cmd_rw surrounded by if/else? As I understand this is the only
> case when it will be split into multiple commands, otherwise it will be
> issued as one command, correct?
>
>
>
>
>
> On Thu, Mar 22, 2018 at 10:46 AM, Wodkowski, PawelX <
> pawelx.wodkowski(a)intel.com> wrote:
>
> As you said, those *v turn iovec into PRP or SGL. Requests are split into
> multiple sub requests only if they can’t be send as single request (eg
> iovec[N-1] and iovec[N] is not PRP complaint). You can check the
> spdk_nvme_ns_cmd_writev() function. The next_sge_fn is a callback used
> inside nvme driver to produce next sgl entry.
>
>
>
> Pawel
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Andrey
> Kuzmin
> *Sent:* Thursday, March 22, 2018 9:12 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over
> spdk_nvme_ns_cmd_write
>
>
>
>
>
> On Thu, Mar 22, 2018, 11:03 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> Lets wait and see what Intel guys have to say.
>
>
>
> Being in your shoes, I'd still stick to spdk_*v flavor in hope that, if
> the downstream driver supports vectored I/O like nvme does, this still
> happens inside the respective bdev.
>
>
>
> Regards,
>
> A.
>
>
>
>
>
> On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>
>
> On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev
> will call it multiple times as the spdk_nvme_cmd_cb callback will move
> buffer pointers forward, if I got the code right.
>
>
>
> Then I'm clearly wrong with the intention behind the *v command flavor. In
> NVMe, those could be natively turned into PRP or SGL filled in, followed by
> the single submission, enjoying a single cimpletion event. If the code is
> like you say, it doesn't matter who loops over the iovs, the caller or the
> callee.
>
>
>
> Regards,
>
> A.
>
>
>
>
>
>
>
> On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>
>
> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> Hi All,
>
>
>
> In case I have to read/write iovecs, is there any advantage of using
> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
> call spdk_nvme_ns_cmd_write?
>
>
>
> The apparent proprotional reduction of the number of submission/completion
> events per I/O. Although, with SPDK, those are much cheaper than with the
> kernel, every penny counts ;).
>
>
>
> Regards,
>
> Andrey
>
>
>
> Sincerely,
>
> E.
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 12923 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 13:09 Wodkowski, PawelX
0 siblings, 0 replies; 11+ messages in thread
From: Wodkowski, PawelX @ 2018-03-22 13:09 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4128 bytes --]
Correct.
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Ernest Zed
Sent: Thursday, March 22, 2018 12:31 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
Pawel, do you mean all these _nvme_ns_cmd_split_request_* in _nvme_ns_cmd_rw surrounded by if/else? As I understand this is the only case when it will be split into multiple commands, otherwise it will be issued as one command, correct?
On Thu, Mar 22, 2018 at 10:46 AM, Wodkowski, PawelX <pawelx.wodkowski(a)intel.com<mailto:pawelx.wodkowski(a)intel.com>> wrote:
As you said, those *v turn iovec into PRP or SGL. Requests are split into multiple sub requests only if they can’t be send as single request (eg iovec[N-1] and iovec[N] is not PRP complaint). You can check the spdk_nvme_ns_cmd_writev() function. The next_sge_fn is a callback used inside nvme driver to produce next sgl entry.
Pawel
From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Andrey Kuzmin
Sent: Thursday, March 22, 2018 9:12 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>
Subject: Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
On Thu, Mar 22, 2018, 11:03 Ernest Zed <kreuzerkrieg(a)gmail.com<mailto:kreuzerkrieg(a)gmail.com>> wrote:
Lets wait and see what Intel guys have to say.
Being in your shoes, I'd still stick to spdk_*v flavor in hope that, if the downstream driver supports vectored I/O like nvme does, this still happens inside the respective bdev.
Regards,
A.
On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com<mailto:andrey.v.kuzmin(a)gmail.com>> wrote:
On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com<mailto:kreuzerkrieg(a)gmail.com>> wrote:
How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev will call it multiple times as the spdk_nvme_cmd_cb callback will move buffer pointers forward, if I got the code right.
Then I'm clearly wrong with the intention behind the *v command flavor. In NVMe, those could be natively turned into PRP or SGL filled in, followed by the single submission, enjoying a single cimpletion event. If the code is like you say, it doesn't matter who loops over the iovs, the caller or the callee.
Regards,
A.
On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com<mailto:andrey.v.kuzmin(a)gmail.com>> wrote:
On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com<mailto:kreuzerkrieg(a)gmail.com>> wrote:
Hi All,
In case I have to read/write iovecs, is there any advantage of using spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and call spdk_nvme_ns_cmd_write?
The apparent proprotional reduction of the number of submission/completion events per I/O. Although, with SPDK, those are much cheaper than with the kernel, every penny counts ;).
Regards,
Andrey
Sincerely,
E.
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
--
Regards,
Andrey
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
--
Regards,
Andrey
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
--
Regards,
Andrey
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 16938 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 11:30 Ernest Zed
0 siblings, 0 replies; 11+ messages in thread
From: Ernest Zed @ 2018-03-22 11:30 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3789 bytes --]
Pawel, do you mean all these _nvme_ns_cmd_split_request_*
in _nvme_ns_cmd_rw surrounded by if/else? As I understand this is the only
case when it will be split into multiple commands, otherwise it will be
issued as one command, correct?
On Thu, Mar 22, 2018 at 10:46 AM, Wodkowski, PawelX <
pawelx.wodkowski(a)intel.com> wrote:
> As you said, those *v turn iovec into PRP or SGL. Requests are split into
> multiple sub requests only if they can’t be send as single request (eg
> iovec[N-1] and iovec[N] is not PRP complaint). You can check the
> spdk_nvme_ns_cmd_writev() function. The next_sge_fn is a callback used
> inside nvme driver to produce next sgl entry.
>
>
>
> Pawel
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Andrey
> Kuzmin
> *Sent:* Thursday, March 22, 2018 9:12 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over
> spdk_nvme_ns_cmd_write
>
>
>
>
>
> On Thu, Mar 22, 2018, 11:03 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> Lets wait and see what Intel guys have to say.
>
>
>
> Being in your shoes, I'd still stick to spdk_*v flavor in hope that, if
> the downstream driver supports vectored I/O like nvme does, this still
> happens inside the respective bdev.
>
>
>
> Regards,
>
> A.
>
>
>
>
>
> On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>
>
> On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev
> will call it multiple times as the spdk_nvme_cmd_cb callback will move
> buffer pointers forward, if I got the code right.
>
>
>
> Then I'm clearly wrong with the intention behind the *v command flavor. In
> NVMe, those could be natively turned into PRP or SGL filled in, followed by
> the single submission, enjoying a single cimpletion event. If the code is
> like you say, it doesn't matter who loops over the iovs, the caller or the
> callee.
>
>
>
> Regards,
>
> A.
>
>
>
>
>
>
>
> On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>
>
> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> Hi All,
>
>
>
> In case I have to read/write iovecs, is there any advantage of using
> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
> call spdk_nvme_ns_cmd_write?
>
>
>
> The apparent proprotional reduction of the number of submission/completion
> events per I/O. Although, with SPDK, those are much cheaper than with the
> kernel, every penny counts ;).
>
>
>
> Regards,
>
> Andrey
>
>
>
> Sincerely,
>
> E.
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 10394 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 11:26 Andrey Kuzmin
0 siblings, 0 replies; 11+ messages in thread
From: Andrey Kuzmin @ 2018-03-22 11:26 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3616 bytes --]
Thanks Pavel, that makes far more sense.
Regards,
A.
On Thu, Mar 22, 2018, 11:46 Wodkowski, PawelX <pawelx.wodkowski(a)intel.com>
wrote:
> As you said, those *v turn iovec into PRP or SGL. Requests are split into
> multiple sub requests only if they can’t be send as single request (eg
> iovec[N-1] and iovec[N] is not PRP complaint). You can check the
> spdk_nvme_ns_cmd_writev() function. The next_sge_fn is a callback used
> inside nvme driver to produce next sgl entry.
>
>
>
> Pawel
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Andrey
> Kuzmin
> *Sent:* Thursday, March 22, 2018 9:12 AM
> *To:* Storage Performance Development Kit <spdk(a)lists.01.org>
> *Subject:* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over
> spdk_nvme_ns_cmd_write
>
>
>
>
>
> On Thu, Mar 22, 2018, 11:03 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> Lets wait and see what Intel guys have to say.
>
>
>
> Being in your shoes, I'd still stick to spdk_*v flavor in hope that, if
> the downstream driver supports vectored I/O like nvme does, this still
> happens inside the respective bdev.
>
>
>
> Regards,
>
> A.
>
>
>
>
>
> On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>
>
> On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev
> will call it multiple times as the spdk_nvme_cmd_cb callback will move
> buffer pointers forward, if I got the code right.
>
>
>
> Then I'm clearly wrong with the intention behind the *v command flavor. In
> NVMe, those could be natively turned into PRP or SGL filled in, followed by
> the single submission, enjoying a single cimpletion event. If the code is
> like you say, it doesn't matter who loops over the iovs, the caller or the
> callee.
>
>
>
> Regards,
>
> A.
>
>
>
>
>
>
>
> On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>
>
> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
> Hi All,
>
>
>
> In case I have to read/write iovecs, is there any advantage of using
> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
> call spdk_nvme_ns_cmd_write?
>
>
>
> The apparent proprotional reduction of the number of submission/completion
> events per I/O. Although, with SPDK, those are much cheaper than with the
> kernel, every penny counts ;).
>
>
>
> Regards,
>
> Andrey
>
>
>
> Sincerely,
>
> E.
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
> --
>
> Regards,
> Andrey
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
--
Regards,
Andrey
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 10369 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 8:46 Wodkowski, PawelX
0 siblings, 0 replies; 11+ messages in thread
From: Wodkowski, PawelX @ 2018-03-22 8:46 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3251 bytes --]
As you said, those *v turn iovec into PRP or SGL. Requests are split into multiple sub requests only if they can’t be send as single request (eg iovec[N-1] and iovec[N] is not PRP complaint). You can check the spdk_nvme_ns_cmd_writev() function. The next_sge_fn is a callback used inside nvme driver to produce next sgl entry.
Pawel
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Andrey Kuzmin
Sent: Thursday, March 22, 2018 9:12 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
On Thu, Mar 22, 2018, 11:03 Ernest Zed <kreuzerkrieg(a)gmail.com<mailto:kreuzerkrieg(a)gmail.com>> wrote:
Lets wait and see what Intel guys have to say.
Being in your shoes, I'd still stick to spdk_*v flavor in hope that, if the downstream driver supports vectored I/O like nvme does, this still happens inside the respective bdev.
Regards,
A.
On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com<mailto:andrey.v.kuzmin(a)gmail.com>> wrote:
On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com<mailto:kreuzerkrieg(a)gmail.com>> wrote:
How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev will call it multiple times as the spdk_nvme_cmd_cb callback will move buffer pointers forward, if I got the code right.
Then I'm clearly wrong with the intention behind the *v command flavor. In NVMe, those could be natively turned into PRP or SGL filled in, followed by the single submission, enjoying a single cimpletion event. If the code is like you say, it doesn't matter who loops over the iovs, the caller or the callee.
Regards,
A.
On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com<mailto:andrey.v.kuzmin(a)gmail.com>> wrote:
On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com<mailto:kreuzerkrieg(a)gmail.com>> wrote:
Hi All,
In case I have to read/write iovecs, is there any advantage of using spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and call spdk_nvme_ns_cmd_write?
The apparent proprotional reduction of the number of submission/completion events per I/O. Although, with SPDK, those are much cheaper than with the kernel, every penny counts ;).
Regards,
Andrey
Sincerely,
E.
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
--
Regards,
Andrey
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
--
Regards,
Andrey
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org<mailto:SPDK(a)lists.01.org>
https://lists.01.org/mailman/listinfo/spdk
--
Regards,
Andrey
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 11302 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 8:11 Andrey Kuzmin
0 siblings, 0 replies; 11+ messages in thread
From: Andrey Kuzmin @ 2018-03-22 8:11 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2698 bytes --]
On Thu, Mar 22, 2018, 11:03 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
> Lets wait and see what Intel guys have to say.
>
Being in your shoes, I'd still stick to spdk_*v flavor in hope that, if the
downstream driver supports vectored I/O like nvme does, this still happens
inside the respective bdev.
Regards,
A.
>
>
> On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
> wrote:
>
>>
>>
>> On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>>
>>> How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev
>>> will call it multiple times as the spdk_nvme_cmd_cb callback will move
>>> buffer pointers forward, if I got the code right.
>>>
>>
>> Then I'm clearly wrong with the intention behind the *v command flavor.
>> In NVMe, those could be natively turned into PRP or SGL filled in, followed
>> by the single submission, enjoying a single cimpletion event. If the code
>> is like you say, it doesn't matter who loops over the iovs, the caller or
>> the callee.
>>
>> Regards,
>> A.
>>
>>
>>>
>>> On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <
>>> andrey.v.kuzmin(a)gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>>>>
>>>>> Hi All,
>>>>>
>>>>> In case I have to read/write iovecs, is there any advantage of using
>>>>> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
>>>>> call spdk_nvme_ns_cmd_write?
>>>>>
>>>>
>>>> The apparent proprotional reduction of the number of
>>>> submission/completion events per I/O. Although, with SPDK, those are much
>>>> cheaper than with the kernel, every penny counts ;).
>>>>
>>>> Regards,
>>>> Andrey
>>>>
>>>>>
>>>>> Sincerely,
>>>>> E.
>>>>> _______________________________________________
>>>>> SPDK mailing list
>>>>> SPDK(a)lists.01.org
>>>>> https://lists.01.org/mailman/listinfo/spdk
>>>>>
>>>> --
>>>>
>>>> Regards,
>>>> Andrey
>>>>
>>>> _______________________________________________
>>>> SPDK mailing list
>>>> SPDK(a)lists.01.org
>>>> https://lists.01.org/mailman/listinfo/spdk
>>>>
>>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>> --
>>
>> Regards,
>> Andrey
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
--
Regards,
Andrey
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 6804 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 8:03 Ernest Zed
0 siblings, 0 replies; 11+ messages in thread
From: Ernest Zed @ 2018-03-22 8:03 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 2163 bytes --]
Lets wait and see what Intel guys have to say.
On Thu, Mar 22, 2018 at 9:45 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
wrote:
>
>
> On Thu, Mar 22, 2018, 10:37 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
>> How come, Andrey? Both functions use _nvme_ns_cmd_rw, the spdk_nvme_ns_cmd_writev
>> will call it multiple times as the spdk_nvme_cmd_cb callback will move
>> buffer pointers forward, if I got the code right.
>>
>
> Then I'm clearly wrong with the intention behind the *v command flavor. In
> NVMe, those could be natively turned into PRP or SGL filled in, followed by
> the single submission, enjoying a single cimpletion event. If the code is
> like you say, it doesn't matter who loops over the iovs, the caller or the
> callee.
>
> Regards,
> A.
>
>
>>
>> On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com
>> > wrote:
>>
>>>
>>>
>>> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> In case I have to read/write iovecs, is there any advantage of using
>>>> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
>>>> call spdk_nvme_ns_cmd_write?
>>>>
>>>
>>> The apparent proprotional reduction of the number of
>>> submission/completion events per I/O. Although, with SPDK, those are much
>>> cheaper than with the kernel, every penny counts ;).
>>>
>>> Regards,
>>> Andrey
>>>
>>>>
>>>> Sincerely,
>>>> E.
>>>> _______________________________________________
>>>> SPDK mailing list
>>>> SPDK(a)lists.01.org
>>>> https://lists.01.org/mailman/listinfo/spdk
>>>>
>>> --
>>>
>>> Regards,
>>> Andrey
>>>
>>> _______________________________________________
>>> SPDK mailing list
>>> SPDK(a)lists.01.org
>>> https://lists.01.org/mailman/listinfo/spdk
>>>
>>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
> --
>
> Regards,
> Andrey
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 5573 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 7:37 Ernest Zed
0 siblings, 0 replies; 11+ messages in thread
From: Ernest Zed @ 2018-03-22 7:37 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]
How come, Andrey? Both functions use _nvme_ns_cmd_rw, the
spdk_nvme_ns_cmd_writev
will call it multiple times as the spdk_nvme_cmd_cb callback will move
buffer pointers forward, if I got the code right.
On Thu, Mar 22, 2018 at 9:06 AM, Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com>
wrote:
>
>
> On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
>
>> Hi All,
>>
>> In case I have to read/write iovecs, is there any advantage of using
>> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
>> call spdk_nvme_ns_cmd_write?
>>
>
> The apparent proprotional reduction of the number of submission/completion
> events per I/O. Although, with SPDK, those are much cheaper than with the
> kernel, every penny counts ;).
>
> Regards,
> Andrey
>
>>
>> Sincerely,
>> E.
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
> --
>
> Regards,
> Andrey
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 3305 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 7:06 Andrey Kuzmin
0 siblings, 0 replies; 11+ messages in thread
From: Andrey Kuzmin @ 2018-03-22 7:06 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 669 bytes --]
On Thu, Mar 22, 2018, 09:34 Ernest Zed <kreuzerkrieg(a)gmail.com> wrote:
> Hi All,
>
> In case I have to read/write iovecs, is there any advantage of using
> spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
> call spdk_nvme_ns_cmd_write?
>
The apparent proprotional reduction of the number of submission/completion
events per I/O. Although, with SPDK, those are much cheaper than with the
kernel, every penny counts ;).
Regards,
Andrey
>
> Sincerely,
> E.
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
--
Regards,
Andrey
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 1416 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write
@ 2018-03-22 6:34 Ernest Zed
0 siblings, 0 replies; 11+ messages in thread
From: Ernest Zed @ 2018-03-22 6:34 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 200 bytes --]
Hi All,
In case I have to read/write iovecs, is there any advantage of using
spdk_nvme_ns_cmd_writev instead of just going over iovecs in loop and
call spdk_nvme_ns_cmd_write?
Sincerely,
E.
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 272 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2018-03-22 13:41 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-22 7:45 [SPDK] Advantages of spdk_nvme_ns_cmd_writev over spdk_nvme_ns_cmd_write Andrey Kuzmin
-- strict thread matches above, loose matches on Subject: below --
2018-03-22 13:41 Ernest Zed
2018-03-22 13:09 Wodkowski, PawelX
2018-03-22 11:30 Ernest Zed
2018-03-22 11:26 Andrey Kuzmin
2018-03-22 8:46 Wodkowski, PawelX
2018-03-22 8:11 Andrey Kuzmin
2018-03-22 8:03 Ernest Zed
2018-03-22 7:37 Ernest Zed
2018-03-22 7:06 Andrey Kuzmin
2018-03-22 6:34 Ernest Zed
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.