* [U-Boot] PCIE_MEM_BUS for Freescale SoC
@ 2015-11-21 22:55 York Sun
2015-11-21 23:29 ` York Sun
0 siblings, 1 reply; 7+ messages in thread
From: York Sun @ 2015-11-21 22:55 UTC (permalink / raw)
To: u-boot
Roy,
Do you remember the reason why we use different virtual memory address from pci
bus address with 36-bit? For example
include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000
include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000
include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc40000000ull
include/configs/P1022DS.h-500-#else
include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000
include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000
include/configs/P1022DS.h-503-#endif
As far as I can tell, the following is the mapping
TLB: MEM_VIRT=>MEM_PHYS
PCI: MEM_BUS=>MEM_PHYS
LAW: MEM_PHYS=>pcie interface
Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci
header" command to show the BARs, I expect I can use "md" to access the BAR
address. That's not the case if MEM_BUS is different from MEM_VIRT.
I forget why we did this for 36-bit addressing. The MEM_VIRT is the same as
MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address for
all PCIe hose? I know they are not conflicting, but is it necessary?
Let me know if you have any concerns. Otherwise I will change all platforms to
use consistent addresses.
York
^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC
2015-11-21 22:55 [U-Boot] PCIE_MEM_BUS for Freescale SoC York Sun
@ 2015-11-21 23:29 ` York Sun
2015-11-23 16:24 ` Scott Wood
0 siblings, 1 reply; 7+ messages in thread
From: York Sun @ 2015-11-21 23:29 UTC (permalink / raw)
To: u-boot
On 11/21/2015 02:55 PM, York Sun wrote:
> Roy,
>
> Do you remember the reason why we use different virtual memory address from pci
> bus address with 36-bit? For example
>
> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000
> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000
> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc40000000ull
> include/configs/P1022DS.h-500-#else
> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000
> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000
> include/configs/P1022DS.h-503-#endif
>
> As far as I can tell, the following is the mapping
>
> TLB: MEM_VIRT=>MEM_PHYS
> PCI: MEM_BUS=>MEM_PHYS
> LAW: MEM_PHYS=>pcie interface
>
> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci
> header" command to show the BARs, I expect I can use "md" to access the BAR
> address. That's not the case if MEM_BUS is different from MEM_VIRT.
>
> I forget why we did this for 36-bit addressing. The MEM_VIRT is the same as
> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address for
> all PCIe hose? I know they are not conflicting, but is it necessary?
(I hope Becky and Kumar still follow this mailing list)
I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by Becky Bruce.
She believed overlapping the bus address for PCI controllers leaves more space.
That's true. But we haven't use more than 512MB in u-boot. If we do need more
space, we can easily move things around if we have PHYS_64BIT. If no objection,
I'd like to change this back.
York
^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC
2015-11-21 23:29 ` York Sun
@ 2015-11-23 16:24 ` Scott Wood
2015-11-23 17:00 ` York Sun
0 siblings, 1 reply; 7+ messages in thread
From: Scott Wood @ 2015-11-23 16:24 UTC (permalink / raw)
To: u-boot
On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
>
> On 11/21/2015 02:55 PM, York Sun wrote:
> > Roy,
> >
> > Do you remember the reason why we use different virtual memory address
> > from pci
> > bus address with 36-bit? For example
> >
> > include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000
> > include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
> > include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000
> > include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
> > 0xc40000000ull
> > include/configs/P1022DS.h-500-#else
> > include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000
> > include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000
> > include/configs/P1022DS.h-503-#endif
> >
> > As far as I can tell, the following is the mapping
> >
> > TLB: MEM_VIRT=>MEM_PHYS
> > PCI: MEM_BUS=>MEM_PHYS
> > LAW: MEM_PHYS=>pcie interface
> >
> > Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci
> > header" command to show the BARs, I expect I can use "md" to access the
> > BAR
> > address. That's not the case if MEM_BUS is different from MEM_VIRT.
> >
> > I forget why we did this for 36-bit addressing. The MEM_VIRT is the same
> > as
> > MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address
> > for
> > all PCIe hose? I know they are not conflicting, but is it necessary?
>
> (I hope Becky and Kumar still follow this mailing list)
>
> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by Becky
> Bruce.
> She believed overlapping the bus address for PCI controllers leaves more
> space.
> That's true. But we haven't use more than 512MB in u-boot. If we do need
> more
> space, we can easily move things around if we have PHYS_64BIT. If no
> objection,
> I'd like to change this back.
I object. It's not about how much RAM is used in U-Boot; it's about how much
memory the OS needs to bounce-buffer for DMA. The addresses set up by U-Boot
should match what's in the device tree. Yes, on 85xx Linux reprograms the
ATMU based on the device tree rather than trusting U-Boot, but that doesn't
mean every OS does.
-Scott
^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC
2015-11-23 16:24 ` Scott Wood
@ 2015-11-23 17:00 ` York Sun
2015-11-23 17:07 ` Scott Wood
0 siblings, 1 reply; 7+ messages in thread
From: York Sun @ 2015-11-23 17:00 UTC (permalink / raw)
To: u-boot
On 11/23/2015 08:24 AM, Scott Wood wrote:
> On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
>>
>> On 11/21/2015 02:55 PM, York Sun wrote:
>>> Roy,
>>>
>>> Do you remember the reason why we use different virtual memory address
>>> from pci
>>> bus address with 36-bit? For example
>>>
>>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT 0xc0000000
>>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
>>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS 0xe0000000
>>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>> 0xc40000000ull
>>> include/configs/P1022DS.h-500-#else
>>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS 0xc0000000
>>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS 0xc0000000
>>> include/configs/P1022DS.h-503-#endif
>>>
>>> As far as I can tell, the following is the mapping
>>>
>>> TLB: MEM_VIRT=>MEM_PHYS
>>> PCI: MEM_BUS=>MEM_PHYS
>>> LAW: MEM_PHYS=>pcie interface
>>>
>>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run "pci
>>> header" command to show the BARs, I expect I can use "md" to access the
>>> BAR
>>> address. That's not the case if MEM_BUS is different from MEM_VIRT.
>>>
>>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the same
>>> as
>>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS address
>>> for
>>> all PCIe hose? I know they are not conflicting, but is it necessary?
>>
>> (I hope Becky and Kumar still follow this mailing list)
>>
>> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by Becky
>> Bruce.
>> She believed overlapping the bus address for PCI controllers leaves more
>> space.
>> That's true. But we haven't use more than 512MB in u-boot. If we do need
>> more
>> space, we can easily move things around if we have PHYS_64BIT. If no
>> objection,
>> I'd like to change this back.
>
> I object. It's not about how much RAM is used in U-Boot; it's about how much
> memory the OS needs to bounce-buffer for DMA. The addresses set up by U-Boot
> should match what's in the device tree. Yes, on 85xx Linux reprograms the
> ATMU based on the device tree rather than trusting U-Boot, but that doesn't
> mean every OS does.
Isn't this backward? I mean the device tree should match u-boot, or u-boot
should fix up the device tree, right?
I checked several device tree files in kernel. We are not using more than 512MB
for each PCI. What's the benefit of using the same address 0xe0000000?
York
^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC
2015-11-23 17:00 ` York Sun
@ 2015-11-23 17:07 ` Scott Wood
2015-11-23 17:11 ` York Sun
2015-11-23 19:56 ` York Sun
0 siblings, 2 replies; 7+ messages in thread
From: Scott Wood @ 2015-11-23 17:07 UTC (permalink / raw)
To: u-boot
On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote:
>
> On 11/23/2015 08:24 AM, Scott Wood wrote:
> > On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
> > >
> > > On 11/21/2015 02:55 PM, York Sun wrote:
> > > > Roy,
> > > >
> > > > Do you remember the reason why we use different virtual memory address
> > > > from pci
> > > > bus address with 36-bit? For example
> > > >
> > > > include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT
> > > > 0xc0000000
> > > > include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
> > > > include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS
> > > > 0xe0000000
> > > > include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
> > > > 0xc40000000ull
> > > > include/configs/P1022DS.h-500-#else
> > > > include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS
> > > > 0xc0000000
> > > > include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS
> > > > 0xc0000000
> > > > include/configs/P1022DS.h-503-#endif
> > > >
> > > > As far as I can tell, the following is the mapping
> > > >
> > > > TLB: MEM_VIRT=>MEM_PHYS
> > > > PCI: MEM_BUS=>MEM_PHYS
> > > > LAW: MEM_PHYS=>pcie interface
> > > >
> > > > Being different for MEM_VIRT and MEM_BUS cause confusion. When I run
> > > > "pci
> > > > header" command to show the BARs, I expect I can use "md" to access
> > > > the
> > > > BAR
> > > > address. That's not the case if MEM_BUS is different from MEM_VIRT.
> > > >
> > > > I forget why we did this for 36-bit addressing. The MEM_VIRT is the
> > > > same
> > > > as
> > > > MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS
> > > > address
> > > > for
> > > > all PCIe hose? I know they are not conflicting, but is it necessary?
> > >
> > > (I hope Becky and Kumar still follow this mailing list)
> > >
> > > I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by
> > > Becky
> > > Bruce.
> > > She believed overlapping the bus address for PCI controllers leaves more
> > > space.
> > > That's true. But we haven't use more than 512MB in u-boot. If we do need
> > > more
> > > space, we can easily move things around if we have PHYS_64BIT. If no
> > > objection,
> > > I'd like to change this back.
> >
> > I object. It's not about how much RAM is used in U-Boot; it's about how
> > much
> > memory the OS needs to bounce-buffer for DMA. The addresses set up by U
> > -Boot
> > should match what's in the device tree. Yes, on 85xx Linux reprograms the
> > ATMU based on the device tree rather than trusting U-Boot, but that
> > doesn't
> > mean every OS does.
>
> Isn't this backward? I mean the device tree should match u-boot,
Yes, and I don't want to deal with the churn of these device trees changing.
> or u-boot should fix up the device tree, right?
In theory yes, but then you really are affecting Linux's usage.
> I checked several device tree files in kernel. We are not using more than
> 512MB
> for each PCI. What's the benefit of using the same address 0xe0000000?
Reducing the amount of RAM whose address matches a PCI bus address and thus
can't be used for PCI DMA.
-Scott
^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC
2015-11-23 17:07 ` Scott Wood
@ 2015-11-23 17:11 ` York Sun
2015-11-23 19:56 ` York Sun
1 sibling, 0 replies; 7+ messages in thread
From: York Sun @ 2015-11-23 17:11 UTC (permalink / raw)
To: u-boot
On 11/23/2015 09:07 AM, Scott Wood wrote:
> On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote:
>>
>> On 11/23/2015 08:24 AM, Scott Wood wrote:
>>> On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
>>>>
>>>> On 11/21/2015 02:55 PM, York Sun wrote:
>>>>> Roy,
>>>>>
>>>>> Do you remember the reason why we use different virtual memory address
>>>>> from pci
>>>>> bus address with 36-bit? For example
>>>>>
>>>>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
>>>>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS
>>>>> 0xe0000000
>>>>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>>>> 0xc40000000ull
>>>>> include/configs/P1022DS.h-500-#else
>>>>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-503-#endif
>>>>>
>>>>> As far as I can tell, the following is the mapping
>>>>>
>>>>> TLB: MEM_VIRT=>MEM_PHYS
>>>>> PCI: MEM_BUS=>MEM_PHYS
>>>>> LAW: MEM_PHYS=>pcie interface
>>>>>
>>>>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run
>>>>> "pci
>>>>> header" command to show the BARs, I expect I can use "md" to access
>>>>> the
>>>>> BAR
>>>>> address. That's not the case if MEM_BUS is different from MEM_VIRT.
>>>>>
>>>>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the
>>>>> same
>>>>> as
>>>>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS
>>>>> address
>>>>> for
>>>>> all PCIe hose? I know they are not conflicting, but is it necessary?
>>>>
>>>> (I hope Becky and Kumar still follow this mailing list)
>>>>
>>>> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by
>>>> Becky
>>>> Bruce.
>>>> She believed overlapping the bus address for PCI controllers leaves more
>>>> space.
>>>> That's true. But we haven't use more than 512MB in u-boot. If we do need
>>>> more
>>>> space, we can easily move things around if we have PHYS_64BIT. If no
>>>> objection,
>>>> I'd like to change this back.
>>>
>>> I object. It's not about how much RAM is used in U-Boot; it's about how
>>> much
>>> memory the OS needs to bounce-buffer for DMA. The addresses set up by U
>>> -Boot
>>> should match what's in the device tree. Yes, on 85xx Linux reprograms the
>>> ATMU based on the device tree rather than trusting U-Boot, but that
>>> doesn't
>>> mean every OS does.
>>
>> Isn't this backward? I mean the device tree should match u-boot,
>
> Yes, and I don't want to deal with the churn of these device trees changing.
>
>> or u-boot should fix up the device tree, right?
>
> In theory yes, but then you really are affecting Linux's usage.
>
>> I checked several device tree files in kernel. We are not using more than
>> 512MB
>> for each PCI. What's the benefit of using the same address 0xe0000000?
>
> Reducing the amount of RAM whose address matches a PCI bus address and thus
> can't be used for PCI DMA.
>
That sounds like a good reason. OK, I will defer my patch until we have a
stronger need to access the BARs under u-boot, or something else changes.
York
^ permalink raw reply [flat|nested] 7+ messages in thread
* [U-Boot] PCIE_MEM_BUS for Freescale SoC
2015-11-23 17:07 ` Scott Wood
2015-11-23 17:11 ` York Sun
@ 2015-11-23 19:56 ` York Sun
1 sibling, 0 replies; 7+ messages in thread
From: York Sun @ 2015-11-23 19:56 UTC (permalink / raw)
To: u-boot
On 11/23/2015 09:07 AM, Scott Wood wrote:
> On Mon, 2015-11-23 at 09:00 -0800, York Sun wrote:
>>
>> On 11/23/2015 08:24 AM, Scott Wood wrote:
>>> On Sat, 2015-11-21 at 15:29 -0800, York Sun wrote:
>>>>
>>>> On 11/21/2015 02:55 PM, York Sun wrote:
>>>>> Roy,
>>>>>
>>>>> Do you remember the reason why we use different virtual memory address
>>>>> from pci
>>>>> bus address with 36-bit? For example
>>>>>
>>>>> include/configs/P1022DS.h-496-#define CONFIG_SYS_PCIE1_MEM_VIRT
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-497-#ifdef CONFIG_PHYS_64BIT
>>>>> include/configs/P1022DS.h:498:#define CONFIG_SYS_PCIE1_MEM_BUS
>>>>> 0xe0000000
>>>>> include/configs/P1022DS.h-499-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>>>> 0xc40000000ull
>>>>> include/configs/P1022DS.h-500-#else
>>>>> include/configs/P1022DS.h:501:#define CONFIG_SYS_PCIE1_MEM_BUS
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-502-#define CONFIG_SYS_PCIE1_MEM_PHYS
>>>>> 0xc0000000
>>>>> include/configs/P1022DS.h-503-#endif
>>>>>
>>>>> As far as I can tell, the following is the mapping
>>>>>
>>>>> TLB: MEM_VIRT=>MEM_PHYS
>>>>> PCI: MEM_BUS=>MEM_PHYS
>>>>> LAW: MEM_PHYS=>pcie interface
>>>>>
>>>>> Being different for MEM_VIRT and MEM_BUS cause confusion. When I run
>>>>> "pci
>>>>> header" command to show the BARs, I expect I can use "md" to access
>>>>> the
>>>>> BAR
>>>>> address. That's not the case if MEM_BUS is different from MEM_VIRT.
>>>>>
>>>>> I forget why we did this for 36-bit addressing. The MEM_VIRT is the
>>>>> same
>>>>> as
>>>>> MEM_BUS for 32-bit addressing. And why do we use the same MEM_BUS
>>>>> address
>>>>> for
>>>>> all PCIe hose? I know they are not conflicting, but is it necessary?
>>>>
>>>> (I hope Becky and Kumar still follow this mailing list)
>>>>
>>>> I dug out an old commit 4c78d4a6c01621721b732418e1c6da684a56bbb1 by
>>>> Becky
>>>> Bruce.
>>>> She believed overlapping the bus address for PCI controllers leaves more
>>>> space.
>>>> That's true. But we haven't use more than 512MB in u-boot. If we do need
>>>> more
>>>> space, we can easily move things around if we have PHYS_64BIT. If no
>>>> objection,
>>>> I'd like to change this back.
>>>
>>> I object. It's not about how much RAM is used in U-Boot; it's about how
>>> much
>>> memory the OS needs to bounce-buffer for DMA. The addresses set up by U
>>> -Boot
>>> should match what's in the device tree. Yes, on 85xx Linux reprograms the
>>> ATMU based on the device tree rather than trusting U-Boot, but that
>>> doesn't
>>> mean every OS does.
>>
>> Isn't this backward? I mean the device tree should match u-boot,
>
> Yes, and I don't want to deal with the churn of these device trees changing.
>
>> or u-boot should fix up the device tree, right?
>
> In theory yes, but then you really are affecting Linux's usage.
>
>> I checked several device tree files in kernel. We are not using more than
>> 512MB
>> for each PCI. What's the benefit of using the same address 0xe0000000?
>
> Reducing the amount of RAM whose address matches a PCI bus address and thus
> can't be used for PCI DMA.
>
Another thought, Linux scans and enumerates PCIe regardless U-boot. The BARs are
setup according to device tree. So setting done u-boot is overwritten.
And, reusing 0xe0000000 for all PCI(e) controllers only applies to PHYS_64BIT in
current code. Isn't address space more crowed for 32-bit?
Regarding the ATMU, if OS relies on u-boot to setup correct ATMU, it should have
correct device tree as well.
Those being said, I can leave the code as is. It is not broken badly, but
confusing if one tries to access BAR space manually.
York
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-11-23 19:56 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-21 22:55 [U-Boot] PCIE_MEM_BUS for Freescale SoC York Sun
2015-11-21 23:29 ` York Sun
2015-11-23 16:24 ` Scott Wood
2015-11-23 17:00 ` York Sun
2015-11-23 17:07 ` Scott Wood
2015-11-23 17:11 ` York Sun
2015-11-23 19:56 ` York Sun
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.