scatter-gather doubt?

scatter-gather doubt?

[Muni Sekhar]

Hi All,

I am working on a xilinx PCIe endpoint with DMA reference block.

The DMA reference block design has 2 Scatter-Gather engines, one for
each DMA channel.

Channel 0 is for HostMemory -> DMA_REF FIFO transfers
Channel 1 is for DMA_REF FIFO -> HostMemory transfers

Each scatter-gather engine works through a linked list of Descriptors
from which it generates the required DMA activity.


The format of these descriptors is depicted as below:

Offset @ 0x00 - LSBs of pointer to DMA data

Offset @ 0x04 - MSBs of pointer to DMA data

Offset @ 0x08 - Number of data bytes to be transferred. (note: only 8
byte aligned transfers supported)

Offset @ 0x0C - LSBs of pointer to next Descriptor  (Set this field &
MSBs to zero to indicate end of descriptor list)

Offset @ 0x10 - MSBs of pointer to next Descriptor



Does the Linux kernel has any data structure to support the above
mentioned scatter-gather descriptor?



Will it be possible to use the kernel scatterlist API?s for this hardware?

-- 
Thanks,
Sekhar

scatter-gather doubt?

[Greg Freemyer]

On Sat, Sep 24, 2016 at 9:33 AM, Muni Sekhar <munisekharrms@gmail.com> wrote:
> Hi All,
>
> I am working on a xilinx PCIe endpoint with DMA reference block.
>
> The DMA reference block design has 2 Scatter-Gather engines, one for
> each DMA channel.
>
> Channel 0 is for HostMemory -> DMA_REF FIFO transfers
> Channel 1 is for DMA_REF FIFO -> HostMemory transfers
>
> Each scatter-gather engine works through a linked list of Descriptors
> from which it generates the required DMA activity.
>
>
> The format of these descriptors is depicted as below:
>
> Offset @ 0x00 - LSBs of pointer to DMA data
>
> Offset @ 0x04 - MSBs of pointer to DMA data
>
> Offset @ 0x08 - Number of data bytes to be transferred. (note: only 8
> byte aligned transfers supported)
>
> Offset @ 0x0C - LSBs of pointer to next Descriptor  (Set this field &
> MSBs to zero to indicate end of descriptor list)
>
> Offset @ 0x10 - MSBs of pointer to next Descriptor
>
>
>
> Does the Linux kernel has any data structure to support the above
> mentioned scatter-gather descriptor?
>
>
>
> Will it be possible to use the kernel scatterlist API?s for this hardware?

Muni,

Have you checked that they don't have a pre-existing driver?

http://www.wiki.xilinx.com/Linux+Drivers

There are 5 different DMA drivers there.  Assuming you have and the
DMA engine doesn't yet have a linux driver:

I'm not sure I've ever written the low level scatter-gather DMA logic
for a DMA engine, but scatter-gather is decades old technology.

The basic scatterlist is defined in: linux/scatterlist.h

see http://lxr.free-electrons.com/source/include/linux/scatterlist.h#L10

===
 10 struct scatterlist {
 11 #ifdef CONFIG_DEBUG_SG
 12         unsigned long   sg_magic;
 13 #endif
 14         unsigned long   page_link;
 15         unsigned int    offset;
 16         unsigned int    length;
 17         dma_addr_t      dma_address;
 18 #ifdef CONFIG_NEED_SG_DMA_LENGTH
 19         unsigned int    dma_length;
 20 #endif
 21 };
===

The upper levels of the linux kernel will populate that and pass it
into the DMA engine driver.

I believe your DMA engine driver will have to accept that scatterlist
coming from the upper layers of the kernel and map it to a new
structure to pass to your IOMMU.

But don't trust my rather vague knowledge.  Have you read:

https://www.kernel.org/doc/Documentation/DMA-API.txt
https://www.kernel.org/doc/Documentation/DMA-API-HOWTO.txt

Search through those for "scatter" and "sg".  You will find the main
kernel structs and API.

And of course, look at the existing xilinx DMA engine drivers and see
if there isn't one close to what you need.

Hope that helps,
Greg

scatter-gather doubt?

[Muni Sekhar]

On Sun, Sep 25, 2016 at 1:02 AM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
> On Sat, Sep 24, 2016 at 9:33 AM, Muni Sekhar <munisekharrms@gmail.com> wrote:
>> Hi All,
>>
>> I am working on a xilinx PCIe endpoint with DMA reference block.
>>
>> The DMA reference block design has 2 Scatter-Gather engines, one for
>> each DMA channel.
>>
>> Channel 0 is for HostMemory -> DMA_REF FIFO transfers
>> Channel 1 is for DMA_REF FIFO -> HostMemory transfers
>>
>> Each scatter-gather engine works through a linked list of Descriptors
>> from which it generates the required DMA activity.
>>
>>
>> The format of these descriptors is depicted as below:
>>
>> Offset @ 0x00 - LSBs of pointer to DMA data
>>
>> Offset @ 0x04 - MSBs of pointer to DMA data
>>
>> Offset @ 0x08 - Number of data bytes to be transferred. (note: only 8
>> byte aligned transfers supported)
>>
>> Offset @ 0x0C - LSBs of pointer to next Descriptor  (Set this field &
>> MSBs to zero to indicate end of descriptor list)
>>
>> Offset @ 0x10 - MSBs of pointer to next Descriptor
>>
>>
>>
>> Does the Linux kernel has any data structure to support the above
>> mentioned scatter-gather descriptor?
>>
>>
>>
>> Will it be possible to use the kernel scatterlist API?s for this hardware?
>
> Muni,
>
> Have you checked that they don't have a pre-existing driver?
>
> http://www.wiki.xilinx.com/Linux+Drivers
>
> There are 5 different DMA drivers there.  Assuming you have and the
> DMA engine doesn't yet have a linux driver:
>
> I'm not sure I've ever written the low level scatter-gather DMA logic
> for a DMA engine, but scatter-gather is decades old technology.
>
> The basic scatterlist is defined in: linux/scatterlist.h
>
> see http://lxr.free-electrons.com/source/include/linux/scatterlist.h#L10
>
> ===
>  10 struct scatterlist {
>  11 #ifdef CONFIG_DEBUG_SG
>  12         unsigned long   sg_magic;
>  13 #endif
>  14         unsigned long   page_link;
>  15         unsigned int    offset;
>  16         unsigned int    length;
>  17         dma_addr_t      dma_address;
>  18 #ifdef CONFIG_NEED_SG_DMA_LENGTH
>  19         unsigned int    dma_length;
>  20 #endif
>  21 };
> ===
>
> The upper levels of the linux kernel will populate that and pass it
> into the DMA engine driver.
>
> I believe your DMA engine driver will have to accept that scatterlist
> coming from the upper layers of the kernel and map it to a new
> structure to pass to your IOMMU.
>
Hi Greg,
Thanks for the useful information, I will look at the existing xilinx
DMA engine drivers and documentation for DMA-API's.

> But don't trust my rather vague knowledge.  Have you read:
>
> https://www.kernel.org/doc/Documentation/DMA-API.txt
> https://www.kernel.org/doc/Documentation/DMA-API-HOWTO.txt
>
> Search through those for "scatter" and "sg".  You will find the main
> kernel structs and API.
>
> And of course, look at the existing xilinx DMA engine drivers and see
> if there isn't one close to what you need.
>
> Hope that helps,
> Greg



-- 
Thanks,
Sekhar