DSP/IOMMU needs 128-byte alignment from user-space buffers?

DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Felipe Contreras]

Hi,

I received information from TI saying that user-space buffers need to
be 128-byte aligned for the DSP to work properly.

Is this true or is it some kind of limitation on their MMU code that
might be solved by Hiroshi's iommu?

This impacts quite drastically user-space applications like GStreamer
since a memory copy would be required to achieve that 128-byte
alignment.

They also claim that adding 128-byte padding at the beginning and at
the end achieves the same purpose.

-- 
Felipe Contreras

RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Kanigeri, Hari]

Hi Felipe,

> I received information from TI saying that user-space buffers 
> need to be 128-byte aligned for the DSP to work properly.
> 
> Is this true or is it some kind of limitation on their MMU 
> code that might be solved by Hiroshi's iommu?
> 
> This impacts quite drastically user-space applications like 
> GStreamer since a memory copy would be required to achieve 
> that 128-byte alignment.
> 
> They also claim that adding 128-byte padding at the beginning 
> and at the end achieves the same purpose.

This is to address the memory corruption that might arise when using DMM (Dynamic memory mapping) feature.
When the DSP processes data, the DSP cache controller loads 128-Byte chunks (lines) from SDRAM and writes the data back in 128-Byte chunks. If a DMM buffer does not start and end on a 128-Byte boundary, the data preceding the start address from the 128-Byte boundary to the Start Address and the data at addresses trailing the end address from the End Address to the next 128-Byte boundary will be loaded and written back as well. If the DMM buffer was allocated on the heap of a process running on the ARM, the preceding and trailing data, if modified by the ARM (or other non-DSP entity) during DSP processing, will be overwritten by the DSP cache controller, when the 128-Byte chunks are written back to SDRAM. This can lead to heap corruption.
I think if you request the buffer from OMX component, this issue will be taken as there is an additional 256 bytes padding.

Thank you,
Best regards,
Hari
 

Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Felipe Contreras]

On Wed, Dec 17, 2008 at 10:37 PM, Kanigeri, Hari <h-kanigeri2@ti.com> wrote:
> Hi Felipe,
>
>> I received information from TI saying that user-space buffers
>> need to be 128-byte aligned for the DSP to work properly.
>>
>> Is this true or is it some kind of limitation on their MMU
>> code that might be solved by Hiroshi's iommu?
>>
>> This impacts quite drastically user-space applications like
>> GStreamer since a memory copy would be required to achieve
>> that 128-byte alignment.
>>
>> They also claim that adding 128-byte padding at the beginning
>> and at the end achieves the same purpose.
>
> This is to address the memory corruption that might arise when using DMM (Dynamic memory mapping) feature.
> When the DSP processes data, the DSP cache controller loads 128-Byte chunks (lines) from SDRAM and writes the data back in 128-Byte chunks. If a DMM buffer does not start and end on a 128-Byte boundary, the data preceding the start address from the 128-Byte boundary to the Start Address and the data at addresses trailing the end address from the End Address to the next 128-Byte boundary will be loaded and written back as well. If the DMM buffer was allocated on the heap of a process running on the ARM, the preceding and trailing data, if modified by the ARM (or other non-DSP entity) during DSP processing, will be overwritten by the DSP cache controller, when the 128-Byte chunks are written back to SDRAM. This can lead to heap corruption.

Thanks for the explanation. So it doesn't have anything to do with
IOMMU; it's the software running on the DSP that has this limitation.

> I think if you request the buffer from OMX component, this issue will be taken as there is an additional 256 bytes padding.

Yes, but in that case a memcpy would be unavoidable in many cases,
like trying to render the result of video decoding on Xv. If the Xv
buffer is unaligned; a dedicated DMM would need to be allocated, and
then the data coming from DSP would need to be copied to the Xv
buffer. That has a huge performance penalty.

So if I'm understanding correctly there's no problem with unaligned
DMM buffers if the DSP software is just reading; the problem is when
writing data back to ARM.

-- 
Felipe Contreras

RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Krishna Kishore]

Hi Felipe,

>So if I'm understanding correctly there's no problem with unaligned DMM
buffers if the DSP software is just reading; the >problem is when
writing data back to ARM.

    If DSP software is just reading, you may have to do cache
invalidate. This is to 
enable subsequent read operations get latest data (if ARM has written
into the buffer).
So, even in this case, 128 byte alignment is required.

Regards,
Kishore. 

-----Original Message-----
From: linux-omap-owner@vger.kernel.org
[mailto:linux-omap-owner@vger.kernel.org] On Behalf Of Felipe Contreras
Sent: Thursday, December 18, 2008 2:51 AM
To: Kanigeri, Hari
Cc: linux-omap@vger.kernel.org; Hiroshi DOYU
Subject: Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?

On Wed, Dec 17, 2008 at 10:37 PM, Kanigeri, Hari <h-kanigeri2@ti.com>
wrote:
> Hi Felipe,
>
>> I received information from TI saying that user-space buffers need to

>> be 128-byte aligned for the DSP to work properly.
>>
>> Is this true or is it some kind of limitation on their MMU code that 
>> might be solved by Hiroshi's iommu?
>>
>> This impacts quite drastically user-space applications like GStreamer

>> since a memory copy would be required to achieve that 128-byte 
>> alignment.
>>
>> They also claim that adding 128-byte padding at the beginning and at 
>> the end achieves the same purpose.
>
> This is to address the memory corruption that might arise when using
DMM (Dynamic memory mapping) feature.
> When the DSP processes data, the DSP cache controller loads 128-Byte
chunks (lines) from SDRAM and writes the data back in 128-Byte chunks.
If a DMM buffer does not start and end on a 128-Byte boundary, the data
preceding the start address from the 128-Byte boundary to the Start
Address and the data at addresses trailing the end address from the End
Address to the next 128-Byte boundary will be loaded and written back as
well. If the DMM buffer was allocated on the heap of a process running
on the ARM, the preceding and trailing data, if modified by the ARM (or
other non-DSP entity) during DSP processing, will be overwritten by the
DSP cache controller, when the 128-Byte chunks are written back to
SDRAM. This can lead to heap corruption.

Thanks for the explanation. So it doesn't have anything to do with
IOMMU; it's the software running on the DSP that has this limitation.

> I think if you request the buffer from OMX component, this issue will
be taken as there is an additional 256 bytes padding.

Yes, but in that case a memcpy would be unavoidable in many cases, like
trying to render the result of video decoding on Xv. If the Xv buffer is
unaligned; a dedicated DMM would need to be allocated, and then the data
coming from DSP would need to be copied to the Xv buffer. That has a
huge performance penalty.

So if I'm understanding correctly there's no problem with unaligned DMM
buffers if the DSP software is just reading; the problem is when writing
data back to ARM.

--
Felipe Contreras
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org More majordomo info
at  http://vger.kernel.org/majordomo-info.html
SASKEN BUSINESS DISCLAIMER
-------------------------
This message may contain confidential, proprietary or legally privileged information. In 
case you are not the original intended Recipient of the message, you must not, directly or 
indirectly, use, Disclose, distribute, print, or copy any part of this message and you are 
requested to delete it and inform the sender. Any views expressed in this message are 
those of the individual sender unless otherwise stated. Nothing contained in this message 
shall be construed as an offer or acceptance of any offer by Sasken Communication 
Technologies Limited ("Sasken") unless sent with that express intent and with due 
authority of Sasken. Sasken has taken enough precautions to prevent the spread of 
viruses. However the company accepts no liability for any damage caused by any virus 
transmitted by this email

RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Kanigeri, Hari]

Hi Felipe,

> So if I'm understanding correctly there's no problem with 
> unaligned DMM buffers if the DSP software is just reading; 
> the problem is when writing data back to ARM.

--- That is my understanding too. For read only buffer there shouldn't be an issue with data corruption.

I heard that some folks at TI are working on documenting this cache alignement issues to share with you. Please hold on.

Thank you,
Best regards,
Hari
 

> -----Original Message-----
> From: Felipe Contreras [mailto:felipe.contreras@gmail.com] 
> Sent: Wednesday, December 17, 2008 3:21 PM
> To: Kanigeri, Hari
> Cc: linux-omap@vger.kernel.org; Hiroshi DOYU
> Subject: Re: DSP/IOMMU needs 128-byte alignment from 
> user-space buffers?
> 
> On Wed, Dec 17, 2008 at 10:37 PM, Kanigeri, Hari 
> <h-kanigeri2@ti.com> wrote:
> > Hi Felipe,
> >
> >> I received information from TI saying that user-space 
> buffers need to 
> >> be 128-byte aligned for the DSP to work properly.
> >>
> >> Is this true or is it some kind of limitation on their MMU 
> code that 
> >> might be solved by Hiroshi's iommu?
> >>
> >> This impacts quite drastically user-space applications 
> like GStreamer 
> >> since a memory copy would be required to achieve that 128-byte 
> >> alignment.
> >>
> >> They also claim that adding 128-byte padding at the 
> beginning and at 
> >> the end achieves the same purpose.
> >
> > This is to address the memory corruption that might arise 
> when using DMM (Dynamic memory mapping) feature.
> > When the DSP processes data, the DSP cache controller loads 
> 128-Byte chunks (lines) from SDRAM and writes the data back 
> in 128-Byte chunks. If a DMM buffer does not start and end on 
> a 128-Byte boundary, the data preceding the start address 
> from the 128-Byte boundary to the Start Address and the data 
> at addresses trailing the end address from the End Address to 
> the next 128-Byte boundary will be loaded and written back as 
> well. If the DMM buffer was allocated on the heap of a 
> process running on the ARM, the preceding and trailing data, 
> if modified by the ARM (or other non-DSP entity) during DSP 
> processing, will be overwritten by the DSP cache controller, 
> when the 128-Byte chunks are written back to SDRAM. This can 
> lead to heap corruption.
> 
> Thanks for the explanation. So it doesn't have anything to do 
> with IOMMU; it's the software running on the DSP that has 
> this limitation.
> 
> > I think if you request the buffer from OMX component, this 
> issue will be taken as there is an additional 256 bytes padding.
> 
> Yes, but in that case a memcpy would be unavoidable in many 
> cases, like trying to render the result of video decoding on 
> Xv. If the Xv buffer is unaligned; a dedicated DMM would need 
> to be allocated, and then the data coming from DSP would need 
> to be copied to the Xv buffer. That has a huge performance penalty.
> 
> So if I'm understanding correctly there's no problem with 
> unaligned DMM buffers if the DSP software is just reading; 
> the problem is when writing data back to ARM.
> 
> --
> Felipe Contreras
> 
> 

Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Hiroshi DOYU]

Hi Hari,

Would it be possible for the bridge to check this alignment and if
it's not, it returns some error to userland and upper layers in a
chain?

      Hiroshi DOYU

From: "ext Kanigeri, Hari" <h-kanigeri2@ti.com>
Subject: RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?
Date: Wed, 17 Dec 2008 14:37:01 -0600

> Hi Felipe,
> 
> > I received information from TI saying that user-space buffers 
> > need to be 128-byte aligned for the DSP to work properly.
> > 
> > Is this true or is it some kind of limitation on their MMU 
> > code that might be solved by Hiroshi's iommu?
> > 
> > This impacts quite drastically user-space applications like 
> > GStreamer since a memory copy would be required to achieve 
> > that 128-byte alignment.
> > 
> > They also claim that adding 128-byte padding at the beginning 
> > and at the end achieves the same purpose.
> 
> This is to address the memory corruption that might arise when using DMM (Dynamic memory mapping) feature.
> When the DSP processes data, the DSP cache controller loads 128-Byte chunks (lines) from SDRAM and writes the data back in 128-Byte chunks. If a DMM buffer does not start and end on a 128-Byte boundary, the data preceding the start address from the 128-Byte boundary to the Start Address and the data at addresses trailing the end address from the End Address to the next 128-Byte boundary will be loaded and written back as well. If the DMM buffer was allocated on the heap of a process running on the ARM, the preceding and trailing data, if modified by the ARM (or other non-DSP entity) during DSP processing, will be overwritten by the DSP cache controller, when the 128-Byte chunks are written back to SDRAM. This can lead to heap corruption.
> I think if you request the buffer from OMX component, this issue will be taken as there is an additional 256 bytes padding.
> 
> Thank you,
> Best regards,
> Hari
>  

Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[David Brownell]

On Wednesday 17 December 2008, Kanigeri, Hari wrote:
> When the DSP processes data, the DSP cache controller
> loads 128-Byte chunks (lines) from SDRAM and writes the
> data back in 128-Byte chunks.

In short:  it's the same kind of cache line sharing issue
that any other DMA-using software needs to be aware of,
except that (a) the lines are 128 bytes vs whatever ARM
uses, and (b) the other bus master is a CPU instead of
something relatively dumb.

Any software that runs on non-x86 hardware will likely
have had to deal with such issues already, since most
hardware other than x86 relies on software to prevent
cache coherency goofs:  no bus snooping, MOESI, etc.

- Dave



> If a DMM buffer does not start and end on a 128-Byte
> boundary, the data preceding the start address from
> the 128-Byte boundary to the Start Address and the
> data at addresses trailing the end address from the
> End Address to the next 128-Byte boundary will be
> loaded and written back as well.         

Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Felipe Contreras]

On Mon, Dec 22, 2008 at 4:19 AM, David Brownell <david-b@pacbell.net> wrote:
> On Wednesday 17 December 2008, Kanigeri, Hari wrote:
>> When the DSP processes data, the DSP cache controller
>> loads 128-Byte chunks (lines) from SDRAM and writes the
>> data back in 128-Byte chunks.
>
> In short:  it's the same kind of cache line sharing issue
> that any other DMA-using software needs to be aware of,
> except that (a) the lines are 128 bytes vs whatever ARM
> uses, and (b) the other bus master is a CPU instead of
> something relatively dumb.
>
> Any software that runs on non-x86 hardware will likely
> have had to deal with such issues already, since most
> hardware other than x86 relies on software to prevent
> cache coherency goofs:  no bus snooping, MOESI, etc.

Do you know of any software doing some kind of processing that sends
the data back? My guess is that there isn't a lot of software doing
that.

I think existing software would need to be updated to render video
output efficiently (allocate Xv buffers properly aligned).

-- 
Felipe Contreras

Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[David Brownell]

On Sunday 21 December 2008, Felipe Contreras wrote:
> On Mon, Dec 22, 2008 at 4:19 AM, David Brownell wrote:
> > In short:  it's the same kind of cache line sharing issue
> > that any other DMA-using software needs to be aware of,
> > ...
> 
> Do you know of any software doing some kind of processing that sends
> the data back? My guess is that there isn't a lot of software doing
> that.

I wouldn't know about such hardware, but I'd be shocked to
learn that TI's products are the first to offload image
processing tasks to hardware.  Workstations have done such
stuff for at least 25 years now, which is enough time to
have tried a LOT of hardware variations.  Surely at least
one non-TI variation wrote data back, in those years...

I'm just saying this is really a bog-standard issue.

Maybe it's a surprise in this context for some reason;
current Open Source tools might easily have made unwise
x86-oriented assumptions, like caches being DMA-coherent.
Tools moving for the first time to new platforms tend to
uncover a lot of portability glitches.  They get fixed.

- Dave

RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Kanigeri, Hari]

Hi Doyu-san,

> Hi Hari,
> 
> Would it be possible for the bridge to check this alignment and if
> it's not, it returns some error to userland and upper layers in a
> chain?

-- We can do this in Bridge. I think the userland layers (LCML, OMX, GStreamer,...) should be then aware on how to handle this error. This will require some discussion with these components owners.

Thank you,
Best regards,
Hari

> -----Original Message-----
> From: Hiroshi DOYU [mailto:Hiroshi.DOYU@nokia.com]
> Sent: Saturday, December 20, 2008 12:27 AM
> To: Kanigeri, Hari
> Cc: felipe.contreras@gmail.com; linux-omap@vger.kernel.org
> Subject: Re: DSP/IOMMU needs 128-byte alignment from user-space buffers?
> 
> Hi Hari,
> 
> Would it be possible for the bridge to check this alignment and if
> it's not, it returns some error to userland and upper layers in a
> chain?
> 
>       Hiroshi DOYU
> 
> From: "ext Kanigeri, Hari" <h-kanigeri2@ti.com>
> Subject: RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?
> Date: Wed, 17 Dec 2008 14:37:01 -0600
> 
> > Hi Felipe,
> >
> > > I received information from TI saying that user-space buffers
> > > need to be 128-byte aligned for the DSP to work properly.
> > >
> > > Is this true or is it some kind of limitation on their MMU
> > > code that might be solved by Hiroshi's iommu?
> > >
> > > This impacts quite drastically user-space applications like
> > > GStreamer since a memory copy would be required to achieve
> > > that 128-byte alignment.
> > >
> > > They also claim that adding 128-byte padding at the beginning
> > > and at the end achieves the same purpose.
> >
> > This is to address the memory corruption that might arise when using DMM
> (Dynamic memory mapping) feature.
> > When the DSP processes data, the DSP cache controller loads 128-Byte
> chunks (lines) from SDRAM and writes the data back in 128-Byte chunks. If
> a DMM buffer does not start and end on a 128-Byte boundary, the data
> preceding the start address from the 128-Byte boundary to the Start
> Address and the data at addresses trailing the end address from the End
> Address to the next 128-Byte boundary will be loaded and written back as
> well. If the DMM buffer was allocated on the heap of a process running on
> the ARM, the preceding and trailing data, if modified by the ARM (or other
> non-DSP entity) during DSP processing, will be overwritten by the DSP
> cache controller, when the 128-Byte chunks are written back to SDRAM. This
> can lead to heap corruption.
> > I think if you request the buffer from OMX component, this issue will be
> taken as there is an additional 256 bytes padding.
> >
> > Thank you,
> > Best regards,
> > Hari
> >

RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Krishna Kishore]

Hi Hari,

   Is this document ready? Can you please send this for information?

Regards,
Kishore. 

-----Original Message-----
From: linux-omap-owner@vger.kernel.org
[mailto:linux-omap-owner@vger.kernel.org] On Behalf Of Kanigeri, Hari
Sent: Thursday, December 18, 2008 7:01 PM
To: Felipe Contreras
Cc: linux-omap@vger.kernel.org; Hiroshi DOYU
Subject: RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

Hi Felipe,

> So if I'm understanding correctly there's no problem with unaligned 
> DMM buffers if the DSP software is just reading; the problem is when 
> writing data back to ARM.

--- That is my understanding too. For read only buffer there shouldn't
be an issue with data corruption.

I heard that some folks at TI are working on documenting this cache
alignement issues to share with you. Please hold on.

Thank you,
Best regards,
Hari
 

> -----Original Message-----
> From: Felipe Contreras [mailto:felipe.contreras@gmail.com]
> Sent: Wednesday, December 17, 2008 3:21 PM
> To: Kanigeri, Hari
> Cc: linux-omap@vger.kernel.org; Hiroshi DOYU
> Subject: Re: DSP/IOMMU needs 128-byte alignment from user-space 
> buffers?
> 
> On Wed, Dec 17, 2008 at 10:37 PM, Kanigeri, Hari <h-kanigeri2@ti.com> 
> wrote:
> > Hi Felipe,
> >
> >> I received information from TI saying that user-space
> buffers need to
> >> be 128-byte aligned for the DSP to work properly.
> >>
> >> Is this true or is it some kind of limitation on their MMU
> code that
> >> might be solved by Hiroshi's iommu?
> >>
> >> This impacts quite drastically user-space applications
> like GStreamer
> >> since a memory copy would be required to achieve that 128-byte 
> >> alignment.
> >>
> >> They also claim that adding 128-byte padding at the
> beginning and at
> >> the end achieves the same purpose.
> >
> > This is to address the memory corruption that might arise
> when using DMM (Dynamic memory mapping) feature.
> > When the DSP processes data, the DSP cache controller loads
> 128-Byte chunks (lines) from SDRAM and writes the data back in 
> 128-Byte chunks. If a DMM buffer does not start and end on a 128-Byte 
> boundary, the data preceding the start address from the 128-Byte 
> boundary to the Start Address and the data at addresses trailing the 
> end address from the End Address to the next 128-Byte boundary will be

> loaded and written back as well. If the DMM buffer was allocated on 
> the heap of a process running on the ARM, the preceding and trailing 
> data, if modified by the ARM (or other non-DSP entity) during DSP 
> processing, will be overwritten by the DSP cache controller, when the 
> 128-Byte chunks are written back to SDRAM. This can lead to heap 
> corruption.
> 
> Thanks for the explanation. So it doesn't have anything to do with 
> IOMMU; it's the software running on the DSP that has this limitation.
> 
> > I think if you request the buffer from OMX component, this
> issue will be taken as there is an additional 256 bytes padding.
> 
> Yes, but in that case a memcpy would be unavoidable in many cases, 
> like trying to render the result of video decoding on Xv. If the Xv 
> buffer is unaligned; a dedicated DMM would need to be allocated, and 
> then the data coming from DSP would need to be copied to the Xv 
> buffer. That has a huge performance penalty.
> 
> So if I'm understanding correctly there's no problem with unaligned 
> DMM buffers if the DSP software is just reading; the problem is when 
> writing data back to ARM.
> 
> --
> Felipe Contreras
> 
> --
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org More majordomo info
at  http://vger.kernel.org/majordomo-info.html
SASKEN BUSINESS DISCLAIMER
-------------------------
This message may contain confidential, proprietary or legally privileged information. In 
case you are not the original intended Recipient of the message, you must not, directly or 
indirectly, use, Disclose, distribute, print, or copy any part of this message and you are 
requested to delete it and inform the sender. Any views expressed in this message are 
those of the individual sender unless otherwise stated. Nothing contained in this message 
shall be construed as an offer or acceptance of any offer by Sasken Communication 
Technologies Limited ("Sasken") unless sent with that express intent and with due 
authority of Sasken. Sasken has taken enough precautions to prevent the spread of 
viruses. However the company accepts no liability for any damage caused by any virus 
transmitted by this email

RE: DSP/IOMMU needs 128-byte alignment from user-space buffers?

[Kanigeri, Hari]

Hi Kishore,

>    Is this document ready? Can you please send this for information?

You can get this document from omapzoom website
https://omapzoom.org/gf/download/docmanfileversion/52/985/DSP_cache.pdf


Thank you,
Best regards,
Hari