[RFC] User space to kernel space copy optimization

[RFC] User space to kernel space copy optimization

[Sinan Kaya]

One of the things I'm interested in is to use a memcpy capable DMA
engine HW to optimize user space and kernel space parameter copying.

Of course, this will not be viable for all parameter sizes but I can see
this being useful. The goal is to save power in the cost of little
performance.

Before I go and invent my own mechanism, I'd like to have ideas on what
an acceptable solution would look like. Especially, I'm looking for a
generic implementation that could plug into any DMA engine HW.

-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project

Re: [RFC] User space to kernel space copy optimization

[Jiang, Dave]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1351 bytes --]



On Mon, 2015-11-16 at 21:53 +0000, Sinan Kaya wrote:
> One of the things I'm interested in is to use a memcpy capable DMA
> engine HW to optimize user space and kernel space parameter copying.

Have you looked at why NET_DMA was deprecated and using DMA engine to
do kernel->user copy could be a problem?
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/
?id=77873803363c

> 
> Of course, this will not be viable for all parameter sizes but I can
> see
> this being useful. The goal is to save power in the cost of little
> performance.
> 
> Before I go and invent my own mechanism, I'd like to have ideas on
> what
> an acceptable solution would look like. Especially, I'm looking for a
> generic implementation that could plug into any DMA engine HW.
> 
> -- 
> Sinan Kaya
> Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center,
> Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
> --
> To unsubscribe from this list: send the line "unsubscribe dmaengine"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.htmlÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [RFC] User space to kernel space copy optimization

[Sinan Kaya]

On 11/16/2015 5:57 PM, Jiang, Dave wrote:
>> One of the things I'm interested in is to use a memcpy capable DMA
>> > engine HW to optimize user space and kernel space parameter copying.
> Have you looked at why NET_DMA was deprecated and using DMA engine to
> do kernel->user copy could be a problem?
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/
> ?id=77873803363c
> 

As far as I know, the problem was the TCP/IP stack trying to work on a
buffer that was given to the DMA engine for moving. It was causing stale
data problems.

The goal was to let DMA engine move the data while TCP/IP stack was
working on something else. Then, synchronize on a known point. Of
course, when above violation happens; things break.

Are you expecting a similar problem on user space to kernel space
interaction? I think the ownership buffers are pretty much defined,
isn't it?


-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project

Re: [RFC] User space to kernel space copy optimization

[Jiang, Dave]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1474 bytes --]

On Mon, 2015-11-16 at 23:19 +0000, Sinan Kaya wrote:
> On 11/16/2015 5:57 PM, Jiang, Dave wrote:
> > > One of the things I'm interested in is to use a memcpy capable
> > > DMA
> > > > engine HW to optimize user space and kernel space parameter
> > > > copying.
> > Have you looked at why NET_DMA was deprecated and using DMA engine
> > to
> > do kernel->user copy could be a problem?
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/com
> > mit/
> > ?id=77873803363c
> > 
> 
> As far as I know, the problem was the TCP/IP stack trying to work on
> a
> buffer that was given to the DMA engine for moving. It was causing
> stale
> data problems.
> 
> The goal was to let DMA engine move the data while TCP/IP stack was
> working on something else. Then, synchronize on a known point. Of
> course, when above violation happens; things break.
> 
> Are you expecting a similar problem on user space to kernel space
> interaction? I think the ownership buffers are pretty much defined,
> isn't it?

No. I thought you were investigating both ways so thought you might
need to be aware. 

> 
> 
> -- 
> Sinan Kaya
> Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center,
> Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Projectÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [RFC] User space to kernel space copy optimization

[Vinod Koul]

On Mon, Nov 16, 2015 at 04:53:27PM -0500, Sinan Kaya wrote:
> One of the things I'm interested in is to use a memcpy capable DMA
> engine HW to optimize user space and kernel space parameter copying.
> 
> Of course, this will not be viable for all parameter sizes but I can see
> this being useful. The goal is to save power in the cost of little
> performance.
> 
> Before I go and invent my own mechanism, I'd like to have ideas on what
> an acceptable solution would look like. Especially, I'm looking for a
> generic implementation that could plug into any DMA engine HW.

Do you have a proposal on how this generic solution should look like, broad
contours of the design to start off with and we can start from that point

-- 
~Vinod

Re: [RFC] User space to kernel space copy optimization

[Sinan Kaya]

On 12/5/2015 3:10 AM, Vinod Koul wrote:
> On Mon, Nov 16, 2015 at 04:53:27PM -0500, Sinan Kaya wrote:
>> One of the things I'm interested in is to use a memcpy capable DMA
>> engine HW to optimize user space and kernel space parameter copying.
>>
>> Of course, this will not be viable for all parameter sizes but I can see
>> this being useful. The goal is to save power in the cost of little
>> performance.
>>
>> Before I go and invent my own mechanism, I'd like to have ideas on what
>> an acceptable solution would look like. Especially, I'm looking for a
>> generic implementation that could plug into any DMA engine HW.
> 
> Do you have a proposal on how this generic solution should look like, broad
> contours of the design to start off with and we can start from that point
> 

No, not yet. AFAIK, some of these optimization ideas have been tried by
the Intel IO/AT in the past (kernel 2.6) and were rejected by the
upstream. I wanted to see if anybody had better ideas these days that is
upstream friendly.

-- 
Sinan Kaya
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project