* Re: [SPDK] ioat copy submitted but not copy any bytes
@ 2018-07-01 8:27 Cao, Gang
0 siblings, 0 replies; 4+ messages in thread
From: Cao, Gang @ 2018-07-01 8:27 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]
Are there any errors reported? Also could you try to build the debug version and dump related trace to take a look?
Thanks,
Gang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of 杨晔
Sent: Saturday, June 30, 2018 8:08 PM
To: spdk(a)lists.01.org
Subject: [SPDK] ioat copy submitted but not copy any bytes
Hello,
What other issues should I pay attention to when using ioat copy?
I just write an demo like the example "perf" did, but it can't copy any bytes.
Here is my code structure:
...(spdk env init)
...(ioat init)
malloc about 300 src strings and fill them(the malloc function I use is kernel function)
malloc 300 dst strings
for(i=0;i<BATCHNUM;i++)
{
for(j=0;j<BATCHNSIZE;j++)
{
spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]->dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
t->current_queue_depth++;
}
}
while(t->current_queue_depth>0)
{
spdk_ioat_process_events(t->chan);
}
And the ioat_done function is as follow:
static void
ioat_done(void *cb_arg)
{
struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
t->current_queue_depth--;
}
Another problem is the ioat channel can never be used if a program exits abnormally.
Thanks.
--
Yang Ye
Network Research Center, Institute of Computing Technology, Chinese Academy of Sciences,
NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing, China (100190)
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 6102 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* [SPDK] ioat copy submitted but not copy any bytes
@ 2019-09-28 2:06
0 siblings, 0 replies; 4+ messages in thread
From: @ 2019-09-28 2:06 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]
Hello,
What other issues should I pay attention to when using ioat copy?
I just write an demo like the example "perf" did, but it can't copy any bytes.
Here is my code structure:
...(spdk env init)
...(ioat init)
malloc about 300 src strings and fill them(the malloc function I use is kernel function)
malloc 300 dst strings
for(i=0;i<BATCHNUM;i++)
{
for(j=0;j<BATCHNSIZE;j++)
{
spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]->dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
t->current_queue_depth++;
}
}
while(t->current_queue_depth>0)
{
spdk_ioat_process_events(t->chan);
}
And the ioat_done function is as follow:
static void
ioat_done(void *cb_arg)
{
struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
t->current_queue_depth--;
}
Another problem is the ioat channel can never be used if a program exits abnormally.
Thanks.
--
Yang Ye
Network Research Center, Institute of Computing Technology, Chinese Academy of Sciences,
NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing, China (100190)
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 1873 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [SPDK] ioat copy submitted but not copy any bytes
@ 2018-07-02 15:17 Stojaczyk, DariuszX
0 siblings, 0 replies; 4+ messages in thread
From: Stojaczyk, DariuszX @ 2018-07-02 15:17 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 4383 bytes --]
> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Cao, Gang
> Sent: Monday, July 2, 2018 3:19 PM
> To: 杨晔 <yangye(a)ict.ac.cn>
> Cc: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
>
> Can you try below function and refer to its example usage for the memory
> registration? And see whether it works in your case?
>
>
>
> spdk_mem_register()
It's not that simple. The minimum page size SPDK supports is 2MB. You can use manually-registered kernel memory alongside spdk_*malloc() only if it comes from hugepages.
D.
>
>
>
> Thanks,
>
> Gang
>
>
>
> From: 杨晔 [mailto:yangye(a)ict.ac.cn]
> Sent: Monday, July 2, 2018 6:46 PM
> To: Cao, Gang <gang.cao(a)intel.com>
> Subject: Re: RE: [SPDK] ioat copy submitted but not copy any bytes
>
>
>
> That works for me.
>
> But that means I can only use ioat to copy the buffers in hugepages?
>
> And if I want use ioat to copy the buffers allocated by kernel, what shall I do? Use
> the kernel module rather than SPDK?
>
> Thanks.
>
> -----原始邮件-----
> 发件人:"Cao, Gang" <gang.cao(a)intel.com <mailto:gang.cao(a)intel.com>
> >
> 发送时间:2018-07-02 16:07:11 (星期一)
> 收件人: "yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn> "
> <yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn> >
> 抄送:
> 主题: RE: [SPDK] ioat copy submitted but not copy any bytes
>
> Have you tried the spdk_malloc() instead of the kernel version malloc for
> this testing?
>
>
>
> From: yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn>
> [mailto:yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn> ]
> Sent: Sunday, July 1, 2018 4:50 PM
> To: Cao, Gang <gang.cao(a)intel.com <mailto:gang.cao(a)intel.com> >
> Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
>
>
>
> No, there's no error report.
>
> It just stopped in the while loop. And I used gdb to memcmp the src and
> dst buffer, finding it not copying any bytes.
>
> I want to know how it works. I call the function spdk_ioat_submit_copy
> and the ioat channel will perform the operation, right?
>
>
>
> ---Original---
>
> From: "Cao, Gang"<gang.cao(a)intel.com <mailto:gang.cao(a)intel.com> >
>
> Date: Sun, Jul 1, 2018 16:27 PM
>
> To: "Storage Performance Development Kit"<spdk(a)lists.01.org
> <mailto:spdk(a)lists.01.org> >;
>
> Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
>
>
>
> Are there any errors reported? Also could you try to build the debug
> version and dump related trace to take a look?
>
>
>
> Thanks,
>
> Gang
>
>
>
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of 杨晔
> Sent: Saturday, June 30, 2018 8:08 PM
> To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> Subject: [SPDK] ioat copy submitted but not copy any bytes
>
>
>
> Hello,
>
> What other issues should I pay attention to when using ioat copy?
>
> I just write an demo like the example "perf" did, but it can't copy any
> bytes.
>
> Here is my code structure:
>
>
>
> ...(spdk env init)
>
> ...(ioat init)
>
> malloc about 300 src strings and fill them(the malloc function I use is
> kernel function)
>
> malloc 300 dst strings
>
> for(i=0;i<BATCHNUM;i++)
> {
> for(j=0;j<BATCHNSIZE;j++)
> {
> spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]-
> >dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
> t->current_queue_depth++;
> }
> }
>
> while(t->current_queue_depth>0)
> {
> spdk_ioat_process_events(t->chan);
> }
>
>
>
> And the ioat_done function is as follow:
>
>
> static void
> ioat_done(void *cb_arg)
> {
> struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
> t->current_queue_depth--;
> }
>
>
>
> Another problem is the ioat channel can never be used if a program exits
> abnormally.
>
> Thanks.
>
> --
>
> Yang Ye
>
> Network Research Center, Institute of Computing Technology, Chinese
> Academy of Sciences,
>
> NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing,
> China (100190)
>
>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [SPDK] ioat copy submitted but not copy any bytes
@ 2018-07-02 13:19 Cao, Gang
0 siblings, 0 replies; 4+ messages in thread
From: Cao, Gang @ 2018-07-02 13:19 UTC (permalink / raw)
To: spdk
[-- Attachment #1: Type: text/plain, Size: 3412 bytes --]
Can you try below function and refer to its example usage for the memory registration? And see whether it works in your case?
spdk_mem_register()
Thanks,
Gang
From: 杨晔 [mailto:yangye(a)ict.ac.cn]
Sent: Monday, July 2, 2018 6:46 PM
To: Cao, Gang <gang.cao(a)intel.com>
Subject: Re: RE: [SPDK] ioat copy submitted but not copy any bytes
That works for me.
But that means I can only use ioat to copy the buffers in hugepages?
And if I want use ioat to copy the buffers allocated by kernel, what shall I do? Use the kernel module rather than SPDK?
Thanks.
-----原始邮件-----
发件人:"Cao, Gang" <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>
发送时间:2018-07-02 16:07:11 (星期一)
收件人: "yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn>" <yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn>>
抄送:
主题: RE: [SPDK] ioat copy submitted but not copy any bytes
Have you tried the spdk_malloc() instead of the kernel version malloc for this testing?
From: yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn> [mailto:yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn>]
Sent: Sunday, July 1, 2018 4:50 PM
To: Cao, Gang <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>
Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
No, there's no error report.
It just stopped in the while loop. And I used gdb to memcmp the src and dst buffer, finding it not copying any bytes.
I want to know how it works. I call the function spdk_ioat_submit_copy and the ioat channel will perform the operation, right?
---Original---
From: "Cao, Gang"<gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>
Date: Sun, Jul 1, 2018 16:27 PM
To: "Storage Performance Development Kit"<spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>;
Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
Are there any errors reported? Also could you try to build the debug version and dump related trace to take a look?
Thanks,
Gang
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of 杨晔
Sent: Saturday, June 30, 2018 8:08 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] ioat copy submitted but not copy any bytes
Hello,
What other issues should I pay attention to when using ioat copy?
I just write an demo like the example "perf" did, but it can't copy any bytes.
Here is my code structure:
...(spdk env init)
...(ioat init)
malloc about 300 src strings and fill them(the malloc function I use is kernel function)
malloc 300 dst strings
for(i=0;i<BATCHNUM;i++)
{
for(j=0;j<BATCHNSIZE;j++)
{
spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]->dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
t->current_queue_depth++;
}
}
while(t->current_queue_depth>0)
{
spdk_ioat_process_events(t->chan);
}
And the ioat_done function is as follow:
static void
ioat_done(void *cb_arg)
{
struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
t->current_queue_depth--;
}
Another problem is the ioat channel can never be used if a program exits abnormally.
Thanks.
--
Yang Ye
Network Research Center, Institute of Computing Technology, Chinese Academy of Sciences,
NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing, China (100190)
[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 13365 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-09-28 2:06 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-01 8:27 [SPDK] ioat copy submitted but not copy any bytes Cao, Gang
2018-07-02 13:19 Cao, Gang
2018-07-02 15:17 Stojaczyk, DariuszX
2019-09-28 2:06
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.