All of lore.kernel.org
 help / color / mirror / Atom feed
* Re: [SPDK] ioat copy submitted but not copy any bytes
@ 2018-07-01  8:27 Cao, Gang
  0 siblings, 0 replies; 4+ messages in thread
From: Cao, Gang @ 2018-07-01  8:27 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 1546 bytes --]

Are there any errors reported? Also could you try to build the debug version and dump related trace to take a look?

Thanks,
Gang

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of 杨晔
Sent: Saturday, June 30, 2018 8:08 PM
To: spdk(a)lists.01.org
Subject: [SPDK] ioat copy submitted but not copy any bytes


Hello,

 What other issues should I pay attention to when using ioat copy?

 I just write an demo like the example "perf" did, but it can't copy any bytes.

 Here is my code structure:



  ...(spdk env init)

  ...(ioat init)

  malloc about 300 src strings and fill them(the malloc function I use is kernel function)

  malloc 300 dst strings

  for(i=0;i<BATCHNUM;i++)
  {
     for(j=0;j<BATCHNSIZE;j++)
     {
        spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]->dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
        t->current_queue_depth++;
     }
  }

  while(t->current_queue_depth>0)
  {
     spdk_ioat_process_events(t->chan);
  }



And the ioat_done function is as follow:


  static void
  ioat_done(void *cb_arg)
  {
     struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
     t->current_queue_depth--;
  }



Another problem is the ioat channel can never be used if a program exits abnormally.

Thanks.

--

Yang Ye

Network Research Center, Institute of Computing Technology, Chinese Academy of Sciences,

NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing, China (100190)

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 6102 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* [SPDK] ioat copy submitted but not copy any bytes
@ 2019-09-28  2:06 
  0 siblings, 0 replies; 4+ messages in thread
From:  @ 2019-09-28  2:06 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

Hello,

 What other issues should I pay attention to when using ioat copy?

 I just write an demo like the example "perf" did, but it can't copy any bytes.

 Here is my code structure:

 

  ...(spdk env init)

  ...(ioat init)

  malloc about 300 src strings and fill them(the malloc function I use is kernel function)

  malloc 300 dst strings

  for(i=0;i<BATCHNUM;i++)
  {
     for(j=0;j<BATCHNSIZE;j++)
     {
        spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]->dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
        t->current_queue_depth++;
     }
  }
 
  while(t->current_queue_depth>0)
  {
     spdk_ioat_process_events(t->chan);
  }

  

And the ioat_done function is as follow:

 
  static void
  ioat_done(void *cb_arg)
  {
     struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
     t->current_queue_depth--;
  }




Another problem is the ioat channel can never be used if a program exits abnormally.

Thanks.


--


Yang Ye

Network Research Center, Institute of Computing Technology, Chinese Academy of Sciences,

NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing, China (100190)

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 1873 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [SPDK] ioat copy submitted but not copy any bytes
@ 2018-07-02 15:17 Stojaczyk, DariuszX
  0 siblings, 0 replies; 4+ messages in thread
From: Stojaczyk, DariuszX @ 2018-07-02 15:17 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 4383 bytes --]



> -----Original Message-----
> From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Cao, Gang
> Sent: Monday, July 2, 2018 3:19 PM
> To: 杨晔 <yangye(a)ict.ac.cn>
> Cc: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
> 
> Can you try below function and refer to its example usage for the memory
> registration? And see whether it works in your case?
> 
> 
> 
> spdk_mem_register()

It's not that simple. The minimum page size SPDK supports is 2MB. You can use manually-registered kernel memory alongside spdk_*malloc() only if it comes from hugepages.

D.

> 
> 
> 
> Thanks,
> 
> Gang
> 
> 
> 
> From: 杨晔 [mailto:yangye(a)ict.ac.cn]
> Sent: Monday, July 2, 2018 6:46 PM
> To: Cao, Gang <gang.cao(a)intel.com>
> Subject: Re: RE: [SPDK] ioat copy submitted but not copy any bytes
> 
> 
> 
> That works for me.
> 
> But that means I can only use ioat to copy the buffers in hugepages?
> 
> And if I want use ioat to copy the buffers allocated by kernel, what shall I do? Use
> the kernel module rather than SPDK?
> 
> Thanks.
> 
> 	-----原始邮件-----
> 	发件人:"Cao, Gang" <gang.cao(a)intel.com <mailto:gang.cao(a)intel.com>
> >
> 	发送时间:2018-07-02 16:07:11 (星期一)
> 	收件人: "yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn> "
> <yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn> >
> 	抄送:
> 	主题: RE: [SPDK] ioat copy submitted but not copy any bytes
> 
> 	Have you tried the spdk_malloc() instead of the kernel version malloc for
> this testing?
> 
> 
> 
> 	From: yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn>
> [mailto:yangye(a)ict.ac.cn <mailto:yangye(a)ict.ac.cn> ]
> 	Sent: Sunday, July 1, 2018 4:50 PM
> 	To: Cao, Gang <gang.cao(a)intel.com <mailto:gang.cao(a)intel.com> >
> 	Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
> 
> 
> 
> 	No, there's no error report.
> 
> 	It just stopped in the while loop. And I used gdb to memcmp the src and
> dst buffer, finding it not copying any bytes.
> 
> 	I want to know how it works. I call the function spdk_ioat_submit_copy
> and the ioat channel will perform the operation, right?
> 
> 
> 
> 	---Original---
> 
> 	From: "Cao, Gang"<gang.cao(a)intel.com <mailto:gang.cao(a)intel.com> >
> 
> 	Date: Sun, Jul 1, 2018 16:27 PM
> 
> 	To: "Storage Performance Development Kit"<spdk(a)lists.01.org
> <mailto:spdk(a)lists.01.org> >;
> 
> 	Subject: Re: [SPDK] ioat copy submitted but not copy any bytes
> 
> 
> 
> 	Are there any errors reported? Also could you try to build the debug
> version and dump related trace to take a look?
> 
> 
> 
> 	Thanks,
> 
> 	Gang
> 
> 
> 
> 	From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of 杨晔
> 	Sent: Saturday, June 30, 2018 8:08 PM
> 	To: spdk(a)lists.01.org <mailto:spdk(a)lists.01.org>
> 	Subject: [SPDK] ioat copy submitted but not copy any bytes
> 
> 
> 
> 	Hello,
> 
> 	 What other issues should I pay attention to when using ioat copy?
> 
> 	 I just write an demo like the example "perf" did, but it can't copy any
> bytes.
> 
> 	 Here is my code structure:
> 
> 
> 
> 	  ...(spdk env init)
> 
> 	  ...(ioat init)
> 
> 	  malloc about 300 src strings and fill them(the malloc function I use is
> kernel function)
> 
> 	  malloc 300 dst strings
> 
> 	  for(i=0;i<BATCHNUM;i++)
> 	  {
> 	     for(j=0;j<BATCHNSIZE;j++)
> 	     {
> 	        spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]-
> >dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
> 	        t->current_queue_depth++;
> 	     }
> 	  }
> 
> 	  while(t->current_queue_depth>0)
> 	  {
> 	     spdk_ioat_process_events(t->chan);
> 	  }
> 
> 
> 
> 	And the ioat_done function is as follow:
> 
> 
> 	  static void
> 	  ioat_done(void *cb_arg)
> 	  {
> 	     struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
> 	     t->current_queue_depth--;
> 	  }
> 
> 
> 
> 	Another problem is the ioat channel can never be used if a program exits
> abnormally.
> 
> 	Thanks.
> 
> 	--
> 
> 	Yang Ye
> 
> 	Network Research Center, Institute of Computing Technology, Chinese
> Academy of Sciences,
> 
> 	NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing,
> China (100190)
> 
> 


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [SPDK] ioat copy submitted but not copy any bytes
@ 2018-07-02 13:19 Cao, Gang
  0 siblings, 0 replies; 4+ messages in thread
From: Cao, Gang @ 2018-07-02 13:19 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 3412 bytes --]

Can you try below function and refer to its example usage for the memory registration? And see whether it works in your case?

spdk_mem_register()

Thanks,
Gang

From: 杨晔 [mailto:yangye(a)ict.ac.cn]
Sent: Monday, July 2, 2018 6:46 PM
To: Cao, Gang <gang.cao(a)intel.com>
Subject: Re: RE: [SPDK] ioat copy submitted but not copy any bytes


That works for me.

But that means I can only use ioat to copy the buffers in hugepages?

And if I want use ioat to copy the buffers allocated by kernel, what shall I do? Use the kernel module rather than SPDK?

Thanks.
-----原始邮件-----
发件人:"Cao, Gang" <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>
发送时间:2018-07-02 16:07:11 (星期一)
收件人: "yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn>" <yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn>>
抄送:
主题: RE: [SPDK] ioat copy submitted but not copy any bytes
Have you tried the spdk_malloc() instead of the kernel version malloc for this testing?

From: yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn> [mailto:yangye(a)ict.ac.cn<mailto:yangye(a)ict.ac.cn>]
Sent: Sunday, July 1, 2018 4:50 PM
To: Cao, Gang <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>
Subject: Re: [SPDK] ioat copy submitted but not copy any bytes

No, there's no error report.
It just stopped in the while loop. And I used gdb to memcmp the src and dst buffer, finding it not copying any bytes.
I want to know how it works. I call the function spdk_ioat_submit_copy and the ioat channel will perform the operation, right?

---Original---
From: "Cao, Gang"<gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>
Date: Sun, Jul 1, 2018 16:27 PM
To: "Storage Performance Development Kit"<spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>;
Subject: Re: [SPDK] ioat copy submitted but not copy any bytes

Are there any errors reported? Also could you try to build the debug version and dump related trace to take a look?

Thanks,
Gang

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of 杨晔
Sent: Saturday, June 30, 2018 8:08 PM
To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>
Subject: [SPDK] ioat copy submitted but not copy any bytes


Hello,

 What other issues should I pay attention to when using ioat copy?

 I just write an demo like the example "perf" did, but it can't copy any bytes.

 Here is my code structure:



  ...(spdk env init)

  ...(ioat init)

  malloc about 300 src strings and fill them(the malloc function I use is kernel function)

  malloc 300 dst strings

  for(i=0;i<BATCHNUM;i++)
  {
     for(j=0;j<BATCHNSIZE;j++)
     {
        spdk_ioat_submit_copy(t->chan, t, ioat_done, batchcopy[i]->elem[j]->dst, batchcopy[i]->elem[j]->src, batchcopy[i]->elem[j]->len);
        t->current_queue_depth++;
     }
  }

  while(t->current_queue_depth>0)
  {
     spdk_ioat_process_events(t->chan);
  }



And the ioat_done function is as follow:


  static void
  ioat_done(void *cb_arg)
  {
     struct ioat_chan_entry *t = (struct ioat_chan_entry *)cb_arg;
     t->current_queue_depth--;
  }



Another problem is the ioat channel can never be used if a program exits abnormally.

Thanks.

--

Yang Ye

Network Research Center, Institute of Computing Technology, Chinese Academy of Sciences,

NO.6 Kexueyuan South Road, Zhongguancun, Haidian District, Beijing, China (100190)


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 13365 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2019-09-28  2:06 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-01  8:27 [SPDK] ioat copy submitted but not copy any bytes Cao, Gang
2018-07-02 13:19 Cao, Gang
2018-07-02 15:17 Stojaczyk, DariuszX
2019-09-28  2:06 

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.