* [P5600 && EVA memory caching question] PCI region
@ 2017-12-06 10:36 Yuri Frolov
2017-12-06 11:46 ` James Hogan
0 siblings, 1 reply; 18+ messages in thread
From: Yuri Frolov @ 2017-12-06 10:36 UTC (permalink / raw)
To: linux-mips; +Cc: James Hogan, paul.burton
Hi,
I'm trying to expand PCI mapped region using EVA addressing scheme for
'Baikal' P5600-based family of boards.
Currently, we've got only one PCIe mapped region, 0x08000000 -
0x1BDBFFFF (~330Mb).
This window is unsufficient for some PCIe cards, so I'm trying to expand
(or add one more) PCIe mapped region by moving 'Hi Memory DRAM block'
from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and
HIGHMEM_START are changed in
arch/mips/include/asm/mach-baikal/hardware.h
/arch/mips/include/asm/mach-baikal/spaces.h respectively) and and
allocating this region for PCIe.
So far, PCI subsystem is initialized during the boot successfully (as
far as I can see):
dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1
dw_pcie_init: PCIe error code = 0x0
dw_pcie_init: PCIe link speed GEN1
device: 'pci0000:01': device_add
PM: Adding info for No Bus:pci0000:01
device: '0000:01': device_add
PM: Adding info for No Bus:0000:01
PCI host bridge to bus 0000:01
pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff]
pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff]
pci_bus 0000:01: root bus resource [bus 01-ff]
pci_bus 0000:01: scanning bus
pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000
pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98
pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff]
pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff]
pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f]
pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff]
pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
pci 0000:01:00.0: PME# disabled
device: '0000:01:00.0': device_add
bus: 'pci': add device 0000:01:00.0
PM: Adding info for pci:0000:01:00.0
pci_bus 0000:01: fixups for bus
pci_bus 0000:01: bus scan returning with max=01
pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff]
pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref]
pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff]
pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff]
pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f]
pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c
pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228
pci 0000:01:00.0: Link Capability is GEN1, x1
but PCIe networking card (e1000e) driver fails to load:
bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver
e1000e
bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0
devices_kset: Moving 0000:01:00.0 to end of list
PCI: Enabling device 0000:01:00.0 (0000 -> 0002)
e1000e 0000:01:00.0: enabling bus mastering
e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000
e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic
conservative mode
PCI MSI: setup hwirq:0 virq:51
PCI MSI: setup hwirq:1 virq:52
PCI MSI: setup hwirq:2 virq:53
e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id);
ret_val = -2
It turns out, that PHY ID can't be read from the memory, mapped on
driver initialization function by ioremap.
__ioremap (arch/mips/mm/ioremap.c) considers all the memory above the
low 512Mb as DRAM and maps the physical addresses 0x20000000 -
0x3FFFFFFF to logical addresses starting from c0000000 and higher.
arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory
map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)'
i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0,
Cfg1: "cacheable, coherent, write-back, write-allocate, read misses
request shared attribute".
I've tried to change the caching attribute (bits 16:18 in SegCtl0) to
0x2 ("uncached, not coherent"), but
this hasn't yield any visible effect on PCIe e1000e initialization.
Could the experts suggest something?..
TIA,
Yuri
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-06 11:46 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-06 11:46 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton [-- Attachment #1: Type: text/plain, Size: 4939 bytes --] Hi Yuri, On Wed, Dec 06, 2017 at 01:36:48PM +0300, Yuri Frolov wrote: > Hi, > > I'm trying to expand PCI mapped region using EVA addressing scheme for > 'Baikal' P5600-based family of boards. > > Currently, we've got only one PCIe mapped region, 0x08000000 - > 0x1BDBFFFF (~330Mb). > > This window is unsufficient for some PCIe cards, so I'm trying to expand > (or add one more) PCIe mapped region by moving 'Hi Memory DRAM block' > from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and > HIGHMEM_START are changed in > arch/mips/include/asm/mach-baikal/hardware.h > /arch/mips/include/asm/mach-baikal/spaces.h respectively) and and > allocating this region for PCIe. > > So far, PCI subsystem is initialized during the boot successfully (as > far as I can see): > > dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1 > dw_pcie_init: PCIe error code = 0x0 > dw_pcie_init: PCIe link speed GEN1 > device: 'pci0000:01': device_add > PM: Adding info for No Bus:pci0000:01 > device: '0000:01': device_add > PM: Adding info for No Bus:0000:01 > PCI host bridge to bus 0000:01 > pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff] > pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff] > pci_bus 0000:01: root bus resource [bus 01-ff] > pci_bus 0000:01: scanning bus > pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000 > pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98 > pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff] > pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff] > pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f] > pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff] > pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref] > pci 0000:01:00.0: PME# supported from D0 D3hot D3cold > > pci 0000:01:00.0: PME# disabled > device: '0000:01:00.0': device_add > bus: 'pci': add device 0000:01:00.0 > PM: Adding info for pci:0000:01:00.0 > pci_bus 0000:01: fixups for bus > pci_bus 0000:01: bus scan returning with max=01 > pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] > pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] > pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] > pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] > pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f] > pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c > pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228 > pci 0000:01:00.0: Link Capability is GEN1, x1 > > but PCIe networking card (e1000e) driver fails to load: > > bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver > e1000e > bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0 > devices_kset: Moving 0000:01:00.0 to end of list > PCI: Enabling device 0000:01:00.0 (0000 -> 0002) > e1000e 0000:01:00.0: enabling bus mastering > e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000 > > e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic > conservative mode > > PCI MSI: setup hwirq:0 virq:51 > PCI MSI: setup hwirq:1 virq:52 > PCI MSI: setup hwirq:2 virq:53 > > e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id); > ret_val = -2 > > It turns out, that PHY ID can't be read from the memory, mapped on > driver initialization function by ioremap. > > __ioremap (arch/mips/mm/ioremap.c) considers all the memory above the > low 512Mb as DRAM and maps the physical addresses 0x20000000 - > 0x3FFFFFFF to logical addresses starting from c0000000 and higher. > > arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory > map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)' > i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0, > Cfg1: "cacheable, coherent, write-back, write-allocate, read misses > request shared attribute". > > I've tried to change the caching attribute (bits 16:18 in SegCtl0) to > 0x2 ("uncached, not coherent"), but > > this hasn't yield any visible effect on PCIe e1000e initialization. Mapped kernel segments don't use the CCA in SegCtl. Each page mapping has its own CCA specified in the TLB, which would have been loaded from page tables set up by e.g. remap_area_pte(), and the ioremap() macro does pass _CACHE_UNCACHED as the flags argument. I would recommend checking the phys_addr and flags coming in to remap_area_pte(). If that all looks correct you could also try: - Dump the TLB contents straight after accessing that address to check they are loaded correctly, i.e. call dump_tlb_all() from <asm/tlbdebug.h>. - If the TLB contents look wrong, try disabling hardware page table walker to determine whether that is misconfigured (add nohtw to kernel command line). Hope that helps Cheers James [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-06 11:46 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-06 11:46 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton [-- Attachment #1: Type: text/plain, Size: 4939 bytes --] Hi Yuri, On Wed, Dec 06, 2017 at 01:36:48PM +0300, Yuri Frolov wrote: > Hi, > > I'm trying to expand PCI mapped region using EVA addressing scheme for > 'Baikal' P5600-based family of boards. > > Currently, we've got only one PCIe mapped region, 0x08000000 - > 0x1BDBFFFF (~330Mb). > > This window is unsufficient for some PCIe cards, so I'm trying to expand > (or add one more) PCIe mapped region by moving 'Hi Memory DRAM block' > from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and > HIGHMEM_START are changed in > arch/mips/include/asm/mach-baikal/hardware.h > /arch/mips/include/asm/mach-baikal/spaces.h respectively) and and > allocating this region for PCIe. > > So far, PCI subsystem is initialized during the boot successfully (as > far as I can see): > > dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1 > dw_pcie_init: PCIe error code = 0x0 > dw_pcie_init: PCIe link speed GEN1 > device: 'pci0000:01': device_add > PM: Adding info for No Bus:pci0000:01 > device: '0000:01': device_add > PM: Adding info for No Bus:0000:01 > PCI host bridge to bus 0000:01 > pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff] > pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff] > pci_bus 0000:01: root bus resource [bus 01-ff] > pci_bus 0000:01: scanning bus > pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000 > pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98 > pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff] > pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff] > pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f] > pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff] > pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref] > pci 0000:01:00.0: PME# supported from D0 D3hot D3cold > > pci 0000:01:00.0: PME# disabled > device: '0000:01:00.0': device_add > bus: 'pci': add device 0000:01:00.0 > PM: Adding info for pci:0000:01:00.0 > pci_bus 0000:01: fixups for bus > pci_bus 0000:01: bus scan returning with max=01 > pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] > pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] > pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] > pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] > pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f] > pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c > pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228 > pci 0000:01:00.0: Link Capability is GEN1, x1 > > but PCIe networking card (e1000e) driver fails to load: > > bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver > e1000e > bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0 > devices_kset: Moving 0000:01:00.0 to end of list > PCI: Enabling device 0000:01:00.0 (0000 -> 0002) > e1000e 0000:01:00.0: enabling bus mastering > e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000 > > e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic > conservative mode > > PCI MSI: setup hwirq:0 virq:51 > PCI MSI: setup hwirq:1 virq:52 > PCI MSI: setup hwirq:2 virq:53 > > e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id); > ret_val = -2 > > It turns out, that PHY ID can't be read from the memory, mapped on > driver initialization function by ioremap. > > __ioremap (arch/mips/mm/ioremap.c) considers all the memory above the > low 512Mb as DRAM and maps the physical addresses 0x20000000 - > 0x3FFFFFFF to logical addresses starting from c0000000 and higher. > > arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory > map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)' > i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0, > Cfg1: "cacheable, coherent, write-back, write-allocate, read misses > request shared attribute". > > I've tried to change the caching attribute (bits 16:18 in SegCtl0) to > 0x2 ("uncached, not coherent"), but > > this hasn't yield any visible effect on PCIe e1000e initialization. Mapped kernel segments don't use the CCA in SegCtl. Each page mapping has its own CCA specified in the TLB, which would have been loaded from page tables set up by e.g. remap_area_pte(), and the ioremap() macro does pass _CACHE_UNCACHED as the flags argument. I would recommend checking the phys_addr and flags coming in to remap_area_pte(). If that all looks correct you could also try: - Dump the TLB contents straight after accessing that address to check they are loaded correctly, i.e. call dump_tlb_all() from <asm/tlbdebug.h>. - If the TLB contents look wrong, try disabling hardware page table walker to determine whether that is misconfigured (add nohtw to kernel command line). Hope that helps Cheers James [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region 2017-12-06 11:46 ` James Hogan (?) @ 2017-12-14 15:03 ` Yuri Frolov 2017-12-14 15:21 ` James Hogan -1 siblings, 1 reply; 18+ messages in thread From: Yuri Frolov @ 2017-12-14 15:03 UTC (permalink / raw) To: James Hogan; +Cc: linux-mips, paul.burton Hi, James. Do I understand correctly, that in case of CONFIG_EVA=y, any logical address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel space is: 1) Unmapped (no TLB translations) 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, in fact a read from physical address 0x20000000? I mean, in legacy memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in kernel space were being translated to the physical addresses from the low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or something for EVA, the mapping is 1:1? As for userspace addresses, the addresses from the 0x00000000 - 0xBFFFFFFF range are: 1) Overlayed with the range which is directly accessible from the kernel space 2) These addresses are mapped in userland, so, read from logical address 0x20000000 in userland may result in read from any physical address located in range 0x00000000 - 0xBFFFFFFF as defined by TLB for that particular userland thread? TIA, Yuri On 12/06/2017 02:46 PM, James Hogan wrote: > Hi Yuri, > > On Wed, Dec 06, 2017 at 01:36:48PM +0300, Yuri Frolov wrote: >> Hi, >> >> I'm trying to expand PCI mapped region using EVA addressing scheme for >> 'Baikal' P5600-based family of boards. >> >> Currently, we've got only one PCIe mapped region, 0x08000000 - >> 0x1BDBFFFF (~330Mb). >> >> This window is unsufficient for some PCIe cards, so I'm trying to expand >> (or add one more) PCIe mapped region by moving 'Hi Memory DRAM block' >> from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and >> HIGHMEM_START are changed in >> arch/mips/include/asm/mach-baikal/hardware.h >> /arch/mips/include/asm/mach-baikal/spaces.h respectively) and and >> allocating this region for PCIe. >> >> So far, PCI subsystem is initialized during the boot successfully (as >> far as I can see): >> >> dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1 >> dw_pcie_init: PCIe error code = 0x0 >> dw_pcie_init: PCIe link speed GEN1 >> device: 'pci0000:01': device_add >> PM: Adding info for No Bus:pci0000:01 >> device: '0000:01': device_add >> PM: Adding info for No Bus:0000:01 >> PCI host bridge to bus 0000:01 >> pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff] >> pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff] >> pci_bus 0000:01: root bus resource [bus 01-ff] >> pci_bus 0000:01: scanning bus >> pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000 >> pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98 >> pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff] >> pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff] >> pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f] >> pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff] >> pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref] >> pci 0000:01:00.0: PME# supported from D0 D3hot D3cold >> >> pci 0000:01:00.0: PME# disabled >> device: '0000:01:00.0': device_add >> bus: 'pci': add device 0000:01:00.0 >> PM: Adding info for pci:0000:01:00.0 >> pci_bus 0000:01: fixups for bus >> pci_bus 0000:01: bus scan returning with max=01 >> pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] >> pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] >> pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] >> pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] >> pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f] >> pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c >> pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228 >> pci 0000:01:00.0: Link Capability is GEN1, x1 >> >> but PCIe networking card (e1000e) driver fails to load: >> >> bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver >> e1000e >> bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0 >> devices_kset: Moving 0000:01:00.0 to end of list >> PCI: Enabling device 0000:01:00.0 (0000 -> 0002) >> e1000e 0000:01:00.0: enabling bus mastering >> e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000 >> >> e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic >> conservative mode >> >> PCI MSI: setup hwirq:0 virq:51 >> PCI MSI: setup hwirq:1 virq:52 >> PCI MSI: setup hwirq:2 virq:53 >> >> e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id); >> ret_val = -2 >> >> It turns out, that PHY ID can't be read from the memory, mapped on >> driver initialization function by ioremap. >> >> __ioremap (arch/mips/mm/ioremap.c) considers all the memory above the >> low 512Mb as DRAM and maps the physical addresses 0x20000000 - >> 0x3FFFFFFF to logical addresses starting from c0000000 and higher. >> >> arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory >> map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)' >> i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0, >> Cfg1: "cacheable, coherent, write-back, write-allocate, read misses >> request shared attribute". >> >> I've tried to change the caching attribute (bits 16:18 in SegCtl0) to >> 0x2 ("uncached, not coherent"), but >> >> this hasn't yield any visible effect on PCIe e1000e initialization. > Mapped kernel segments don't use the CCA in SegCtl. Each page mapping > has its own CCA specified in the TLB, which would have been loaded from > page tables set up by e.g. remap_area_pte(), and the ioremap() macro > does pass _CACHE_UNCACHED as the flags argument. I would recommend > checking the phys_addr and flags coming in to remap_area_pte(). > > If that all looks correct you could also try: > - Dump the TLB contents straight after accessing that address to check > they are loaded correctly, i.e. call dump_tlb_all() from > <asm/tlbdebug.h>. > - If the TLB contents look wrong, try disabling hardware page table > walker to determine whether that is misconfigured (add nohtw to kernel > command line). > > Hope that helps > > Cheers > James ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-14 15:21 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-14 15:21 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton [-- Attachment #1: Type: text/plain, Size: 7207 bytes --] Hi Yuri, On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: > Hi, James. > > Do I understand correctly, that in case of CONFIG_EVA=y, any logical > address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel > space is: > 1) Unmapped (no TLB translations) > 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, > in fact a read from physical address 0x20000000? I mean, in legacy > memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in > kernel space were being translated to the physical addresses from the > low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or > something for EVA, the mapping is 1:1? That depends on the EVA configuration. The hardware is fairly flexible (which is both useful and painful - making supporting EVA from multiplatform kernels particularly awkward), but that is one possible configuration. E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) MUSK (TLB mapped for user & kernel) so that null dereferences from kernel code are safely caught, but that costs you 1GB of directly accessible physical address space from kernel mode. > > As for userspace addresses, the addresses from the 0x00000000 - > 0xBFFFFFFF range are: > 1) Overlayed with the range which is directly accessible from the kernel > space Yes. Userland accesses access through tlb, kernel direct, and kernel eva accesses (LWE/SWE etc) access through tlb from kernel > 2) These addresses are mapped in userland, so, read from logical address > 0x20000000 in userland may result in read from any physical address > located in range 0x00000000 - 0xBFFFFFFF as defined by TLB for that > particular userland thread? Yes, though it could in theory be any physical address mapped by the TLB, not just that range. Cheers James > > TIA, > Yuri > > On 12/06/2017 02:46 PM, James Hogan wrote: > > Hi Yuri, > > > > On Wed, Dec 06, 2017 at 01:36:48PM +0300, Yuri Frolov wrote: > >> Hi, > >> > >> I'm trying to expand PCI mapped region using EVA addressing scheme for > >> 'Baikal' P5600-based family of boards. > >> > >> Currently, we've got only one PCIe mapped region, 0x08000000 - > >> 0x1BDBFFFF (~330Mb). > >> > >> This window is unsufficient for some PCIe cards, so I'm trying to expand > >> (or add one more) PCIe mapped region by moving 'Hi Memory DRAM block' > >> from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and > >> HIGHMEM_START are changed in > >> arch/mips/include/asm/mach-baikal/hardware.h > >> /arch/mips/include/asm/mach-baikal/spaces.h respectively) and and > >> allocating this region for PCIe. > >> > >> So far, PCI subsystem is initialized during the boot successfully (as > >> far as I can see): > >> > >> dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1 > >> dw_pcie_init: PCIe error code = 0x0 > >> dw_pcie_init: PCIe link speed GEN1 > >> device: 'pci0000:01': device_add > >> PM: Adding info for No Bus:pci0000:01 > >> device: '0000:01': device_add > >> PM: Adding info for No Bus:0000:01 > >> PCI host bridge to bus 0000:01 > >> pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff] > >> pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff] > >> pci_bus 0000:01: root bus resource [bus 01-ff] > >> pci_bus 0000:01: scanning bus > >> pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000 > >> pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98 > >> pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff] > >> pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff] > >> pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f] > >> pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff] > >> pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref] > >> pci 0000:01:00.0: PME# supported from D0 D3hot D3cold > >> > >> pci 0000:01:00.0: PME# disabled > >> device: '0000:01:00.0': device_add > >> bus: 'pci': add device 0000:01:00.0 > >> PM: Adding info for pci:0000:01:00.0 > >> pci_bus 0000:01: fixups for bus > >> pci_bus 0000:01: bus scan returning with max=01 > >> pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] > >> pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] > >> pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] > >> pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] > >> pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f] > >> pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c > >> pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228 > >> pci 0000:01:00.0: Link Capability is GEN1, x1 > >> > >> but PCIe networking card (e1000e) driver fails to load: > >> > >> bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver > >> e1000e > >> bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0 > >> devices_kset: Moving 0000:01:00.0 to end of list > >> PCI: Enabling device 0000:01:00.0 (0000 -> 0002) > >> e1000e 0000:01:00.0: enabling bus mastering > >> e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000 > >> > >> e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic > >> conservative mode > >> > >> PCI MSI: setup hwirq:0 virq:51 > >> PCI MSI: setup hwirq:1 virq:52 > >> PCI MSI: setup hwirq:2 virq:53 > >> > >> e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id); > >> ret_val = -2 > >> > >> It turns out, that PHY ID can't be read from the memory, mapped on > >> driver initialization function by ioremap. > >> > >> __ioremap (arch/mips/mm/ioremap.c) considers all the memory above the > >> low 512Mb as DRAM and maps the physical addresses 0x20000000 - > >> 0x3FFFFFFF to logical addresses starting from c0000000 and higher. > >> > >> arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory > >> map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)' > >> i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0, > >> Cfg1: "cacheable, coherent, write-back, write-allocate, read misses > >> request shared attribute". > >> > >> I've tried to change the caching attribute (bits 16:18 in SegCtl0) to > >> 0x2 ("uncached, not coherent"), but > >> > >> this hasn't yield any visible effect on PCIe e1000e initialization. > > Mapped kernel segments don't use the CCA in SegCtl. Each page mapping > > has its own CCA specified in the TLB, which would have been loaded from > > page tables set up by e.g. remap_area_pte(), and the ioremap() macro > > does pass _CACHE_UNCACHED as the flags argument. I would recommend > > checking the phys_addr and flags coming in to remap_area_pte(). > > > > If that all looks correct you could also try: > > - Dump the TLB contents straight after accessing that address to check > > they are loaded correctly, i.e. call dump_tlb_all() from > > <asm/tlbdebug.h>. > > - If the TLB contents look wrong, try disabling hardware page table > > walker to determine whether that is misconfigured (add nohtw to kernel > > command line). > > > > Hope that helps > > > > Cheers > > James > [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-14 15:21 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-14 15:21 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton [-- Attachment #1: Type: text/plain, Size: 7207 bytes --] Hi Yuri, On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: > Hi, James. > > Do I understand correctly, that in case of CONFIG_EVA=y, any logical > address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel > space is: > 1) Unmapped (no TLB translations) > 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, > in fact a read from physical address 0x20000000? I mean, in legacy > memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in > kernel space were being translated to the physical addresses from the > low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or > something for EVA, the mapping is 1:1? That depends on the EVA configuration. The hardware is fairly flexible (which is both useful and painful - making supporting EVA from multiplatform kernels particularly awkward), but that is one possible configuration. E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) MUSK (TLB mapped for user & kernel) so that null dereferences from kernel code are safely caught, but that costs you 1GB of directly accessible physical address space from kernel mode. > > As for userspace addresses, the addresses from the 0x00000000 - > 0xBFFFFFFF range are: > 1) Overlayed with the range which is directly accessible from the kernel > space Yes. Userland accesses access through tlb, kernel direct, and kernel eva accesses (LWE/SWE etc) access through tlb from kernel > 2) These addresses are mapped in userland, so, read from logical address > 0x20000000 in userland may result in read from any physical address > located in range 0x00000000 - 0xBFFFFFFF as defined by TLB for that > particular userland thread? Yes, though it could in theory be any physical address mapped by the TLB, not just that range. Cheers James > > TIA, > Yuri > > On 12/06/2017 02:46 PM, James Hogan wrote: > > Hi Yuri, > > > > On Wed, Dec 06, 2017 at 01:36:48PM +0300, Yuri Frolov wrote: > >> Hi, > >> > >> I'm trying to expand PCI mapped region using EVA addressing scheme for > >> 'Baikal' P5600-based family of boards. > >> > >> Currently, we've got only one PCIe mapped region, 0x08000000 - > >> 0x1BDBFFFF (~330Mb). > >> > >> This window is unsufficient for some PCIe cards, so I'm trying to expand > >> (or add one more) PCIe mapped region by moving 'Hi Memory DRAM block' > >> from 0x20000000 to 0x40000000 (dts changed, BAIKAL_HIGHMEM_START and > >> HIGHMEM_START are changed in > >> arch/mips/include/asm/mach-baikal/hardware.h > >> /arch/mips/include/asm/mach-baikal/spaces.h respectively) and and > >> allocating this region for PCIe. > >> > >> So far, PCI subsystem is initialized during the boot successfully (as > >> far as I can see): > >> > >> dw_pcie_init: DEV_ID_VEND_ID=0x80601d39 CLASS_CODE_REV_ID=0x1 > >> dw_pcie_init: PCIe error code = 0x0 > >> dw_pcie_init: PCIe link speed GEN1 > >> device: 'pci0000:01': device_add > >> PM: Adding info for No Bus:pci0000:01 > >> device: '0000:01': device_add > >> PM: Adding info for No Bus:0000:01 > >> PCI host bridge to bus 0000:01 > >> pci_bus 0000:01: root bus resource [mem 0x20000000-0x37ffffff] > >> pci_bus 0000:01: root bus resource [io 0x18020000-0x1bdaffff] > >> pci_bus 0000:01: root bus resource [bus 01-ff] > >> pci_bus 0000:01: scanning bus > >> pci 0000:01:00.0: [8086:10d3] type 00 class 0x020000 > >> pci 0000:01:00.0: calling quirk_f0_vpd_link+0x0/0x98 > >> pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0001ffff] > >> pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0007ffff] > >> pci 0000:01:00.0: reg 0x18: [io 0x0000-0x001f] > >> pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x00003fff] > >> pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0003ffff pref] > >> pci 0000:01:00.0: PME# supported from D0 D3hot D3cold > >> > >> pci 0000:01:00.0: PME# disabled > >> device: '0000:01:00.0': device_add > >> bus: 'pci': add device 0000:01:00.0 > >> PM: Adding info for pci:0000:01:00.0 > >> pci_bus 0000:01: fixups for bus > >> pci_bus 0000:01: bus scan returning with max=01 > >> pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] > >> pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] > >> pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] > >> pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] > >> pci 0000:01:00.0: BAR 2: assigned [io 0x18020000-0x1802001f] > >> pci 0000:01:00.0: calling quirk_e100_interrupt+0x0/0x20c > >> pci 0000:01:00.0: calling baikal_t1_pcie_link_speed_fixup+0x0/0x228 > >> pci 0000:01:00.0: Link Capability is GEN1, x1 > >> > >> but PCIe networking card (e1000e) driver fails to load: > >> > >> bus: 'pci': driver_probe_device: matched device 0000:01:00.0 with driver > >> e1000e > >> bus: 'pci': really_probe: probing driver e1000e with device 0000:01:00.0 > >> devices_kset: Moving 0000:01:00.0 to end of list > >> PCI: Enabling device 0000:01:00.0 (0000 -> 0002) > >> e1000e 0000:01:00.0: enabling bus mastering > >> e1000_probe:7054 mmio_start 0x200c0000 mmio_len 0x20000 > >> > >> e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic > >> conservative mode > >> > >> PCI MSI: setup hwirq:0 virq:51 > >> PCI MSI: setup hwirq:1 virq:52 > >> PCI MSI: setup hwirq:2 virq:53 > >> > >> e1000_get_phy_id_82571:444 ret_val = e1e_rphy(hw, MII_PHYSID1, &phy_id); > >> ret_val = -2 > >> > >> It turns out, that PHY ID can't be read from the memory, mapped on > >> driver initialization function by ioremap. > >> > >> __ioremap (arch/mips/mm/ioremap.c) considers all the memory above the > >> low 512Mb as DRAM and maps the physical addresses 0x20000000 - > >> 0x3FFFFFFF to logical addresses starting from c0000000 and higher. > >> > >> arch/mips/include/asm/mach-baikal/kernel-entry-init.h defines EVA memory > >> map with virtual addresses range 0xc0000000 - 0xdfffffff as 'MK (kseg2)' > >> i.e. "mapped kernel" with cashing segment attributes defined in SegCtl0, > >> Cfg1: "cacheable, coherent, write-back, write-allocate, read misses > >> request shared attribute". > >> > >> I've tried to change the caching attribute (bits 16:18 in SegCtl0) to > >> 0x2 ("uncached, not coherent"), but > >> > >> this hasn't yield any visible effect on PCIe e1000e initialization. > > Mapped kernel segments don't use the CCA in SegCtl. Each page mapping > > has its own CCA specified in the TLB, which would have been loaded from > > page tables set up by e.g. remap_area_pte(), and the ioremap() macro > > does pass _CACHE_UNCACHED as the flags argument. I would recommend > > checking the phys_addr and flags coming in to remap_area_pte(). > > > > If that all looks correct you could also try: > > - Dump the TLB contents straight after accessing that address to check > > they are loaded correctly, i.e. call dump_tlb_all() from > > <asm/tlbdebug.h>. > > - If the TLB contents look wrong, try disabling hardware page table > > walker to determine whether that is misconfigured (add nohtw to kernel > > command line). > > > > Hope that helps > > > > Cheers > > James > [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region 2017-12-14 15:21 ` James Hogan (?) @ 2017-12-15 14:05 ` Yuri Frolov 2017-12-15 23:28 ` James Hogan -1 siblings, 1 reply; 18+ messages in thread From: Yuri Frolov @ 2017-12-15 14:05 UTC (permalink / raw) To: James Hogan; +Cc: linux-mips, paul.burton Hi, James On 12/14/2017 06:21 PM, James Hogan wrote: > Hi Yuri, > > On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: >> Hi, James. >> >> Do I understand correctly, that in case of CONFIG_EVA=y, any logical >> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel >> space is: >> 1) Unmapped (no TLB translations) >> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, >> in fact a read from physical address 0x20000000? I mean, in legacy >> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in >> kernel space were being translated to the physical addresses from the >> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or >> something for EVA, the mapping is 1:1? > That depends on the EVA configuration. The hardware is fairly flexible > (which is both useful and painful - making supporting EVA from > multiplatform kernels particularly awkward), but that is one possible > configuration. > > E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) > MUSK (TLB mapped for user & kernel) so that null dereferences from > kernel code are safely caught, but that costs you 1GB of directly > accessible physical address space from kernel mode. So, do I understand correctly, that provided we have these TLB entries in kernel, Index: 71 pgmask=16kb va=c0038000 asid=00 [ri=0 xi=0 pa=200e0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=00000000 c=0 d=0 v=0 g=1] Index: 72 pgmask=16kb va=c0040000 asid=00 [ri=0 xi=0 pa=200c0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=200c4000 c=2 d=1 v=1 g=1] u32 l; l = readl((const volatile void *)(0x200c0000 + 0x20)) and l = *(u32 *)(0xc0040000+ 0x20) should return the same value? > >> As for userspace addresses, the addresses from the 0x00000000 - >> 0xBFFFFFFF range are: >> 1) Overlayed with the range which is directly accessible from the kernel >> space > Yes. Userland accesses access through tlb, kernel direct, and kernel eva > accesses (LWE/SWE etc) access through tlb from kernel > >> 2) These addresses are mapped in userland, so, read from logical address >> 0x20000000 in userland may result in read from any physical address >> located in range 0x00000000 - 0xBFFFFFFF as defined by TLB for that >> particular userland thread? > Yes, though it could in theory be any physical address mapped by the > TLB, not just that range. > > Cheers > James ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-15 23:28 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-15 23:28 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton On Fri, Dec 15, 2017 at 05:05:12PM +0300, Yuri Frolov wrote: > Hi, James > > On 12/14/2017 06:21 PM, James Hogan wrote: > > Hi Yuri, > > > > On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: > >> Hi, James. > >> > >> Do I understand correctly, that in case of CONFIG_EVA=y, any logical > >> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel > >> space is: > >> 1) Unmapped (no TLB translations) > >> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, > >> in fact a read from physical address 0x20000000? I mean, in legacy > >> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in > >> kernel space were being translated to the physical addresses from the > >> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or > >> something for EVA, the mapping is 1:1? > > That depends on the EVA configuration. The hardware is fairly flexible > > (which is both useful and painful - making supporting EVA from > > multiplatform kernels particularly awkward), but that is one possible > > configuration. > > > > E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) > > MUSK (TLB mapped for user & kernel) so that null dereferences from > > kernel code are safely caught, but that costs you 1GB of directly > > accessible physical address space from kernel mode. > So, do I understand correctly, that provided we have these TLB entries > in kernel, > > Index: 71 pgmask=16kb va=c0038000 asid=00 > [ri=0 xi=0 pa=200e0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=00000000 > c=0 d=0 v=0 g=1] > Index: 72 pgmask=16kb va=c0040000 asid=00 > [ri=0 xi=0 pa=200c0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=200c4000 > c=2 d=1 v=1 g=1] > > u32 l; > > l = readl((const volatile void *)(0x200c0000 + 0x20)) assuming you have segment5 (0x00000000..0x3FFFFFFF) set to MUSUK, with PA 0 (i.e. direct 1:1 mapping), it'd access PA 0x200c0020, but presumably the segment will be configured to be cached (CCA 3) or cached coherent (CCA 5). > and > l = *(u32 *)(0xc0040000+ 0x20) this will access physical addres 0x200c0020 uncached (CCA 2). > > should return the same value? So that would depend I think on whether there is a dirty value in the cache. Possibly not. The cached mapping would see the dirty value. The uncached mapping may see a stale value straight from RAM. Cheers James ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-15 23:28 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-15 23:28 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton On Fri, Dec 15, 2017 at 05:05:12PM +0300, Yuri Frolov wrote: > Hi, James > > On 12/14/2017 06:21 PM, James Hogan wrote: > > Hi Yuri, > > > > On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: > >> Hi, James. > >> > >> Do I understand correctly, that in case of CONFIG_EVA=y, any logical > >> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel > >> space is: > >> 1) Unmapped (no TLB translations) > >> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, > >> in fact a read from physical address 0x20000000? I mean, in legacy > >> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in > >> kernel space were being translated to the physical addresses from the > >> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or > >> something for EVA, the mapping is 1:1? > > That depends on the EVA configuration. The hardware is fairly flexible > > (which is both useful and painful - making supporting EVA from > > multiplatform kernels particularly awkward), but that is one possible > > configuration. > > > > E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) > > MUSK (TLB mapped for user & kernel) so that null dereferences from > > kernel code are safely caught, but that costs you 1GB of directly > > accessible physical address space from kernel mode. > So, do I understand correctly, that provided we have these TLB entries > in kernel, > > Index: 71 pgmask=16kb va=c0038000 asid=00 > [ri=0 xi=0 pa=200e0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=00000000 > c=0 d=0 v=0 g=1] > Index: 72 pgmask=16kb va=c0040000 asid=00 > [ri=0 xi=0 pa=200c0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=200c4000 > c=2 d=1 v=1 g=1] > > u32 l; > > l = readl((const volatile void *)(0x200c0000 + 0x20)) assuming you have segment5 (0x00000000..0x3FFFFFFF) set to MUSUK, with PA 0 (i.e. direct 1:1 mapping), it'd access PA 0x200c0020, but presumably the segment will be configured to be cached (CCA 3) or cached coherent (CCA 5). > and > l = *(u32 *)(0xc0040000+ 0x20) this will access physical addres 0x200c0020 uncached (CCA 2). > > should return the same value? So that would depend I think on whether there is a dirty value in the cache. Possibly not. The cached mapping would see the dirty value. The uncached mapping may see a stale value straight from RAM. Cheers James ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region 2017-12-15 23:28 ` James Hogan (?) @ 2017-12-20 15:11 ` Yuri Frolov 2017-12-21 12:40 ` Yuri Frolov 2017-12-21 13:41 ` James Hogan -1 siblings, 2 replies; 18+ messages in thread From: Yuri Frolov @ 2017-12-20 15:11 UTC (permalink / raw) To: James Hogan; +Cc: linux-mips, paul.burton Hi, James On 12/16/2017 02:28 AM, James Hogan wrote: > On Fri, Dec 15, 2017 at 05:05:12PM +0300, Yuri Frolov wrote: >> Hi, James >> >> On 12/14/2017 06:21 PM, James Hogan wrote: >>> Hi Yuri, >>> >>> On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: >>>> Hi, James. >>>> >>>> Do I understand correctly, that in case of CONFIG_EVA=y, any logical >>>> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel >>>> space is: >>>> 1) Unmapped (no TLB translations) >>>> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, >>>> in fact a read from physical address 0x20000000? I mean, in legacy >>>> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in >>>> kernel space were being translated to the physical addresses from the >>>> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or >>>> something for EVA, the mapping is 1:1? >>> That depends on the EVA configuration. The hardware is fairly flexible >>> (which is both useful and painful - making supporting EVA from >>> multiplatform kernels particularly awkward), but that is one possible >>> configuration. >>> >>> E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) >>> MUSK (TLB mapped for user & kernel) so that null dereferences from >>> kernel code are safely caught, but that costs you 1GB of directly >>> accessible physical address space from kernel mode. >> So, do I understand correctly, that provided we have these TLB entries >> in kernel, >> >> Index: 71 pgmask=16kb va=c0038000 asid=00 >> [ri=0 xi=0 pa=200e0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=00000000 >> c=0 d=0 v=0 g=1] >> Index: 72 pgmask=16kb va=c0040000 asid=00 >> [ri=0 xi=0 pa=200c0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=200c4000 >> c=2 d=1 v=1 g=1] >> >> u32 l; >> >> l = readl((const volatile void *)(0x200c0000 + 0x20)) > assuming you have segment5 (0x00000000..0x3FFFFFFF) set to MUSUK, with > PA 0 (i.e. direct 1:1 mapping), it'd access PA 0x200c0020, but > presumably the segment will be configured to be cached (CCA 3) or cached > coherent (CCA 5). > >> and >> l = *(u32 *)(0xc0040000+ 0x20) > this will access physical addres 0x200c0020 uncached (CCA 2). > >> should return the same value? > So that would depend I think on whether there is a dirty value in the > cache. Possibly not. The cached mapping would see the dirty value. The > uncached mapping may see a stale value straight from RAM. > > Cheers > James I'm looking at arch/mips/include/asm//mach-malta/kernel-entry-init.h and there is a definition for SegCtl2: /* SegCtl2 */ li t0, ((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ (6 << MIPS_SEGCFG_PA_SHIFT) | \ (1 << MIPS_SEGCFG_EU_SHIFT)) | \ (((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ (4 << MIPS_SEGCFG_PA_SHIFT) | \ (1 << MIPS_SEGCFG_EU_SHIFT)) << 16) it defines, that kernel logical addresses from the range 0x00000000 - 0x7fffffff are unmapped (no tlbs) and dictates, that in order to get a physical address for any logical addresses from 0x00000000 - 0x3fffffff range in kernel space, bits [31:29] of the logical address must be changed to 100, and (again in kernel space) for any logical addresses from 0x40000000 - 0x7fffffff range, bits [31:29] of the logical address must be changed to 110, right? What physical addresses will logical addresses 0x00000000 and 0x20000000 be translated in kernel space?.. logical 0x00000000 --> physical 0x80000000, and logical 0x20000000 --> .... 0x80000000 too? Since we must to change bits [31:29], we have to change bit 29 ('1') in logical address 0x200000000 to '0' (since PA for this range is 100). So, what physical addresses will all logical addresses which have '1' at 29 bit be translated, if we define PA as 100 and 110 in SegCtl2? It looks like there's no flat translation of logical addresses to physical addresses in kernel space, and this is obviously just not correct, there is something simple I've been overlooking. Thank you, Yuri ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-21 12:40 ` Yuri Frolov 0 siblings, 0 replies; 18+ messages in thread From: Yuri Frolov @ 2017-12-21 12:40 UTC (permalink / raw) Cc: linux-mips > I'm looking at arch/mips/include/asm//mach-malta/kernel-entry-init.h and there is a definition for > SegCtl2: > > /* SegCtl2 */ > li t0, ((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (6 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) | \ > (((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (4 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) << 16) > > it defines, that kernel logical addresses from the range 0x00000000 - > 0x7fffffff are unmapped (no tlbs) and dictates, that in order to get a > physical address for any logical addresses from 0x00000000 - > 0x3fffffff range in kernel space, bits [31:29] of the logical address > must be changed to 100, > and (again in kernel space) for any logical addresses from 0x40000000 > - 0x7fffffff range, bits [31:29] of the logical address must be > changed to 110, right? > > What physical addresses will logical addresses 0x00000000 and > 0x20000000 be translated in kernel space?.. logical 0x00000000 --> > physical 0x80000000, and logical 0x20000000 --> .... 0x80000000 too? > Since we must to change bits [31:29], we have to change bit 29 ('1') > in logical address 0x200000000 to '0' (since PA for this range is 100). > > So, what physical addresses will all logical addresses which have '1' > at 29 bit be translated, if we define PA as 100 and 110 in SegCtl2? It > looks like there's no flat translation of logical addresses to > physical addresses in kernel space, and this is obviously just not > correct, there is something simple I've been overlooking. > > Thank you, > Yuri The only way this scheme can work, is that 'these bits are used' phrase from Table 3.5 CFG (Segment Configuration) Field Description means not 'these bits substitute bits [31:29] in virtual address', but 'these bits are 'OR'-ed with bits [31:29] in virtual address. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-21 12:40 ` Yuri Frolov 0 siblings, 0 replies; 18+ messages in thread From: Yuri Frolov @ 2017-12-21 12:40 UTC (permalink / raw) Cc: linux-mips > I'm looking at arch/mips/include/asm//mach-malta/kernel-entry-init.h and there is a definition for > SegCtl2: > > /* SegCtl2 */ > li t0, ((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (6 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) | \ > (((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (4 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) << 16) > > it defines, that kernel logical addresses from the range 0x00000000 - > 0x7fffffff are unmapped (no tlbs) and dictates, that in order to get a > physical address for any logical addresses from 0x00000000 - > 0x3fffffff range in kernel space, bits [31:29] of the logical address > must be changed to 100, > and (again in kernel space) for any logical addresses from 0x40000000 > - 0x7fffffff range, bits [31:29] of the logical address must be > changed to 110, right? > > What physical addresses will logical addresses 0x00000000 and > 0x20000000 be translated in kernel space?.. logical 0x00000000 --> > physical 0x80000000, and logical 0x20000000 --> .... 0x80000000 too? > Since we must to change bits [31:29], we have to change bit 29 ('1') > in logical address 0x200000000 to '0' (since PA for this range is 100). > > So, what physical addresses will all logical addresses which have '1' > at 29 bit be translated, if we define PA as 100 and 110 in SegCtl2? It > looks like there's no flat translation of logical addresses to > physical addresses in kernel space, and this is obviously just not > correct, there is something simple I've been overlooking. > > Thank you, > Yuri The only way this scheme can work, is that 'these bits are used' phrase from Table 3.5 CFG (Segment Configuration) Field Description means not 'these bits substitute bits [31:29] in virtual address', but 'these bits are 'OR'-ed with bits [31:29] in virtual address. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-21 13:41 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-21 13:41 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton On Wed, Dec 20, 2017 at 06:11:04PM +0300, Yuri Frolov wrote: > Hi, James > > On 12/16/2017 02:28 AM, James Hogan wrote: > > On Fri, Dec 15, 2017 at 05:05:12PM +0300, Yuri Frolov wrote: > >> Hi, James > >> > >> On 12/14/2017 06:21 PM, James Hogan wrote: > >>> Hi Yuri, > >>> > >>> On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: > >>>> Hi, James. > >>>> > >>>> Do I understand correctly, that in case of CONFIG_EVA=y, any logical > >>>> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel > >>>> space is: > >>>> 1) Unmapped (no TLB translations) > >>>> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, > >>>> in fact a read from physical address 0x20000000? I mean, in legacy > >>>> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in > >>>> kernel space were being translated to the physical addresses from the > >>>> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or > >>>> something for EVA, the mapping is 1:1? > >>> That depends on the EVA configuration. The hardware is fairly flexible > >>> (which is both useful and painful - making supporting EVA from > >>> multiplatform kernels particularly awkward), but that is one possible > >>> configuration. > >>> > >>> E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) > >>> MUSK (TLB mapped for user & kernel) so that null dereferences from > >>> kernel code are safely caught, but that costs you 1GB of directly > >>> accessible physical address space from kernel mode. > >> So, do I understand correctly, that provided we have these TLB entries > >> in kernel, > >> > >> Index: 71 pgmask=16kb va=c0038000 asid=00 > >> [ri=0 xi=0 pa=200e0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=00000000 > >> c=0 d=0 v=0 g=1] > >> Index: 72 pgmask=16kb va=c0040000 asid=00 > >> [ri=0 xi=0 pa=200c0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=200c4000 > >> c=2 d=1 v=1 g=1] > >> > >> u32 l; > >> > >> l = readl((const volatile void *)(0x200c0000 + 0x20)) > > assuming you have segment5 (0x00000000..0x3FFFFFFF) set to MUSUK, with > > PA 0 (i.e. direct 1:1 mapping), it'd access PA 0x200c0020, but > > presumably the segment will be configured to be cached (CCA 3) or cached > > coherent (CCA 5). > > > >> and > >> l = *(u32 *)(0xc0040000+ 0x20) > > this will access physical addres 0x200c0020 uncached (CCA 2). > > > >> should return the same value? > > So that would depend I think on whether there is a dirty value in the > > cache. Possibly not. The cached mapping would see the dirty value. The > > uncached mapping may see a stale value straight from RAM. > > > > Cheers > > James > I'm looking at arch/mips/include/asm//mach-malta/kernel-entry-init.h and > there is a definition for SegCtl2: > > /* SegCtl2 */ > li t0, ((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (6 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) | \ > (((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (4 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) << 16) > > it defines, that kernel logical addresses from the range 0x00000000 - > 0x7fffffff are unmapped (no tlbs) and dictates, that in order to get a > physical address for any logical addresses from 0x00000000 - 0x3fffffff > range in kernel space, bits [31:29] of the logical address must be > changed to 100, > and (again in kernel space) for any logical addresses from 0x40000000 - > 0x7fffffff range, bits [31:29] of the logical address must be changed to > 110, right? yes, the Malta implementation is slightly ugly as it relies on a hardware physical memory alias of RAM starting at PA 0x80000000. > > What physical addresses will logical addresses 0x00000000 and 0x20000000 > be translated in kernel space?.. logical 0x00000000 --> physical > 0x80000000, and logical 0x20000000 --> .... 0x80000000 too? VA 0x20000000 -> PA 0xa0000000, since seg4 and seg5 are 1GB segments, so its only bits 30 and up that can be changed. I seem to remember the bit corresponding to bit 29 isn't even writable in the SegCtl2 register. Does that clarify things? Cheers James > Since we must to change bits [31:29], we have to change bit 29 ('1') in > logical address 0x200000000 to '0' (since PA for this range is 100). > > So, what physical addresses will all logical addresses which have '1' at > 29 bit be translated, if we define PA as 100 and 110 in SegCtl2? It > looks like there's no flat translation of logical addresses to physical > addresses in kernel space, and this is obviously just not correct, there > is something simple I've been overlooking. > > Thank you, > Yuri ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-21 13:41 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-21 13:41 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton On Wed, Dec 20, 2017 at 06:11:04PM +0300, Yuri Frolov wrote: > Hi, James > > On 12/16/2017 02:28 AM, James Hogan wrote: > > On Fri, Dec 15, 2017 at 05:05:12PM +0300, Yuri Frolov wrote: > >> Hi, James > >> > >> On 12/14/2017 06:21 PM, James Hogan wrote: > >>> Hi Yuri, > >>> > >>> On Thu, Dec 14, 2017 at 06:03:11PM +0300, Yuri Frolov wrote: > >>>> Hi, James. > >>>> > >>>> Do I understand correctly, that in case of CONFIG_EVA=y, any logical > >>>> address from 0x00000000 - 0xBFFFFFFF (3G) range accessed from the kernel > >>>> space is: > >>>> 1) Unmapped (no TLB translations) > >>>> 2) Is mapped 1:1 to physical addresses? E.g, readl from 0x20000000 is, > >>>> in fact a read from physical address 0x20000000? I mean, in legacy > >>>> memory mapping scheme, logical addresses 0x80000000 - 0xBFFFFFFF in > >>>> kernel space were being translated to the physical addresses from the > >>>> low 512Mb (phys 0x00000000 - 0x20000000), no such bits stripping or > >>>> something for EVA, the mapping is 1:1? > >>> That depends on the EVA configuration. The hardware is fairly flexible > >>> (which is both useful and painful - making supporting EVA from > >>> multiplatform kernels particularly awkward), but that is one possible > >>> configuration. > >>> > >>> E.g. ideally you probably want to keep seg5 (0x00000000..0x3FFFFFFF) > >>> MUSK (TLB mapped for user & kernel) so that null dereferences from > >>> kernel code are safely caught, but that costs you 1GB of directly > >>> accessible physical address space from kernel mode. > >> So, do I understand correctly, that provided we have these TLB entries > >> in kernel, > >> > >> Index: 71 pgmask=16kb va=c0038000 asid=00 > >> [ri=0 xi=0 pa=200e0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=00000000 > >> c=0 d=0 v=0 g=1] > >> Index: 72 pgmask=16kb va=c0040000 asid=00 > >> [ri=0 xi=0 pa=200c0000 c=2 d=1 v=1 g=1] [ri=0 xi=0 pa=200c4000 > >> c=2 d=1 v=1 g=1] > >> > >> u32 l; > >> > >> l = readl((const volatile void *)(0x200c0000 + 0x20)) > > assuming you have segment5 (0x00000000..0x3FFFFFFF) set to MUSUK, with > > PA 0 (i.e. direct 1:1 mapping), it'd access PA 0x200c0020, but > > presumably the segment will be configured to be cached (CCA 3) or cached > > coherent (CCA 5). > > > >> and > >> l = *(u32 *)(0xc0040000+ 0x20) > > this will access physical addres 0x200c0020 uncached (CCA 2). > > > >> should return the same value? > > So that would depend I think on whether there is a dirty value in the > > cache. Possibly not. The cached mapping would see the dirty value. The > > uncached mapping may see a stale value straight from RAM. > > > > Cheers > > James > I'm looking at arch/mips/include/asm//mach-malta/kernel-entry-init.h and > there is a definition for SegCtl2: > > /* SegCtl2 */ > li t0, ((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (6 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) | \ > (((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ > (4 << MIPS_SEGCFG_PA_SHIFT) | \ > (1 << MIPS_SEGCFG_EU_SHIFT)) << 16) > > it defines, that kernel logical addresses from the range 0x00000000 - > 0x7fffffff are unmapped (no tlbs) and dictates, that in order to get a > physical address for any logical addresses from 0x00000000 - 0x3fffffff > range in kernel space, bits [31:29] of the logical address must be > changed to 100, > and (again in kernel space) for any logical addresses from 0x40000000 - > 0x7fffffff range, bits [31:29] of the logical address must be changed to > 110, right? yes, the Malta implementation is slightly ugly as it relies on a hardware physical memory alias of RAM starting at PA 0x80000000. > > What physical addresses will logical addresses 0x00000000 and 0x20000000 > be translated in kernel space?.. logical 0x00000000 --> physical > 0x80000000, and logical 0x20000000 --> .... 0x80000000 too? VA 0x20000000 -> PA 0xa0000000, since seg4 and seg5 are 1GB segments, so its only bits 30 and up that can be changed. I seem to remember the bit corresponding to bit 29 isn't even writable in the SegCtl2 register. Does that clarify things? Cheers James > Since we must to change bits [31:29], we have to change bit 29 ('1') in > logical address 0x200000000 to '0' (since PA for this range is 100). > > So, what physical addresses will all logical addresses which have '1' at > 29 bit be translated, if we define PA as 100 and 110 in SegCtl2? It > looks like there's no flat translation of logical addresses to physical > addresses in kernel space, and this is obviously just not correct, there > is something simple I've been overlooking. > > Thank you, > Yuri ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region 2017-12-21 13:41 ` James Hogan (?) @ 2017-12-22 9:37 ` Yuri Frolov -1 siblings, 0 replies; 18+ messages in thread From: Yuri Frolov @ 2017-12-22 9:37 UTC (permalink / raw) To: James Hogan; +Cc: linux-mips, paul.burton >> I'm looking at arch/mips/include/asm//mach-malta/kernel-entry-init.h and >> there is a definition for SegCtl2: >> >> /* SegCtl2 */ >> li t0, ((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ >> (6 << MIPS_SEGCFG_PA_SHIFT) | \ >> (1 << MIPS_SEGCFG_EU_SHIFT)) | \ >> (((MIPS_SEGCFG_MUSUK << MIPS_SEGCFG_AM_SHIFT) | \ >> (4 << MIPS_SEGCFG_PA_SHIFT) | \ >> (1 << MIPS_SEGCFG_EU_SHIFT)) << 16) >> >> it defines, that kernel logical addresses from the range 0x00000000 - >> 0x7fffffff are unmapped (no tlbs) and dictates, that in order to get a >> physical address for any logical addresses from 0x00000000 - 0x3fffffff >> range in kernel space, bits [31:29] of the logical address must be >> changed to 100, >> and (again in kernel space) for any logical addresses from 0x40000000 - >> 0x7fffffff range, bits [31:29] of the logical address must be changed to >> 110, right? > yes, the Malta implementation is slightly ugly as it relies on a > hardware physical memory alias of RAM starting at PA 0x80000000. > >> What physical addresses will logical addresses 0x00000000 and 0x20000000 >> be translated in kernel space?.. logical 0x00000000 --> physical >> 0x80000000, and logical 0x20000000 --> .... 0x80000000 too? > VA 0x20000000 -> PA 0xa0000000, since seg4 and seg5 are 1GB segments, so > its only bits 30 and up that can be changed. Ah, I've got it now. > I seem to remember the bit > corresponding to bit 29 isn't even writable in the SegCtl2 register. I shoulda read the actual definition of SegCtl2, my bad. I presumed the definitions are the same for SegCtl0 - SegCtl2. > Does that clarify things? Yes, absolutely. Thank you very much! -- yuri ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region 2017-12-21 13:41 ` James Hogan (?) (?) @ 2017-12-22 11:18 ` Yuri Frolov 2017-12-22 11:49 ` James Hogan -1 siblings, 1 reply; 18+ messages in thread From: Yuri Frolov @ 2017-12-22 11:18 UTC (permalink / raw) To: James Hogan; +Cc: linux-mips, paul.burton Sorry, I forgot to ask, > yes, the Malta implementation is slightly ugly as it relies on a > hardware physical memory alias of RAM starting at PA 0x80000000. any good docs about this hardware aliasing? I'm not sure I understand it correctly, just want to understand things right. -- yuri ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-22 11:49 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-22 11:49 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton [-- Attachment #1: Type: text/plain, Size: 1161 bytes --] On Fri, Dec 22, 2017 at 02:18:26PM +0300, Yuri Frolov wrote: > Sorry, I forgot to ask, > > yes, the Malta implementation is slightly ugly as it relies on a > > hardware physical memory alias of RAM starting at PA 0x80000000. > any good docs about this hardware aliasing? I'm not sure I understand it > correctly, just want to understand things right. Basically it keeps kseg0 (seg3) and kseg1 (seg2) pointing at PA 0, which it runs kernel code from (i.e. same place it is loaded, without the EVA layout enabled yet), but then it does data accesses from seg4 and seg5 which point at PA 0x80000000. I think the intention is to allow the VA to PA offset to be the same for all cached segments. However because the caches don't know that different physical addresses refer to the same underlying RAM they aren't coherent with one another and the kernel has to be careful not to use both, and to flush the caches during boot. Its not an approach I would personally recommend, and if we get EVA support added to the generic platform I'd hope it would be a lot cleaner, perhaps using the since added kernel self-relocation. Cheers James [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [P5600 && EVA memory caching question] PCI region @ 2017-12-22 11:49 ` James Hogan 0 siblings, 0 replies; 18+ messages in thread From: James Hogan @ 2017-12-22 11:49 UTC (permalink / raw) To: Yuri Frolov; +Cc: linux-mips, paul.burton [-- Attachment #1: Type: text/plain, Size: 1161 bytes --] On Fri, Dec 22, 2017 at 02:18:26PM +0300, Yuri Frolov wrote: > Sorry, I forgot to ask, > > yes, the Malta implementation is slightly ugly as it relies on a > > hardware physical memory alias of RAM starting at PA 0x80000000. > any good docs about this hardware aliasing? I'm not sure I understand it > correctly, just want to understand things right. Basically it keeps kseg0 (seg3) and kseg1 (seg2) pointing at PA 0, which it runs kernel code from (i.e. same place it is loaded, without the EVA layout enabled yet), but then it does data accesses from seg4 and seg5 which point at PA 0x80000000. I think the intention is to allow the VA to PA offset to be the same for all cached segments. However because the caches don't know that different physical addresses refer to the same underlying RAM they aren't coherent with one another and the kernel has to be careful not to use both, and to flush the caches during boot. Its not an approach I would personally recommend, and if we get EVA support added to the generic platform I'd hope it would be a lot cleaner, perhaps using the since added kernel self-relocation. Cheers James [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2017-12-22 11:51 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-12-06 10:36 [P5600 && EVA memory caching question] PCI region Yuri Frolov 2017-12-06 11:46 ` James Hogan 2017-12-06 11:46 ` James Hogan 2017-12-14 15:03 ` Yuri Frolov 2017-12-14 15:21 ` James Hogan 2017-12-14 15:21 ` James Hogan 2017-12-15 14:05 ` Yuri Frolov 2017-12-15 23:28 ` James Hogan 2017-12-15 23:28 ` James Hogan 2017-12-20 15:11 ` Yuri Frolov 2017-12-21 12:40 ` Yuri Frolov 2017-12-21 12:40 ` Yuri Frolov 2017-12-21 13:41 ` James Hogan 2017-12-21 13:41 ` James Hogan 2017-12-22 9:37 ` Yuri Frolov 2017-12-22 11:18 ` Yuri Frolov 2017-12-22 11:49 ` James Hogan 2017-12-22 11:49 ` James Hogan
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.