* Resizeable PCI BAR support V3 @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Hi Bjorn and others on the lists, it's over a year that I initially came up with that, but now I finally have the time to finish it up. This set of patches allows device drivers to resize and most likely also relocate the PCI BAR of devices they manage to allow the CPU to access all of the device local memory at once. This is very useful for GFX device drivers where the default PCI BAR is only about 256MB in size for compatibility reasons, but the device easily have multiple gigabyte of local memory. Additional to the pure resize functionality this set also adds a quirk to enable a 64bit bar above 4GB on AMD Family 15h CPUs/APUs. Doing so is actually trivial and I plan to extend this to all recent AMD CPUs/APUs. Open questions: 1. Is this the right location to implement the 64bit BAR above 4GB for AMD CPUs? I guess that this needs a bit more cleanup. 2. Any idea how to better handle intermediate bridges? Reprogramming them isn't possible after drivers are loaded, but at least for GFX drivers we need to load them first to know how large we actually want our BAR to be (and to kick our vesafb/efifb). Thanks in advance, Christian. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Resizeable PCI BAR support V3 @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Hi Bjorn and others on the lists, it's over a year that I initially came up with that, but now I finally have the time to finish it up. This set of patches allows device drivers to resize and most likely also relocate the PCI BAR of devices they manage to allow the CPU to access all of the device local memory at once. This is very useful for GFX device drivers where the default PCI BAR is only about 256MB in size for compatibility reasons, but the device easily have multiple gigabyte of local memory. Additional to the pure resize functionality this set also adds a quirk to enable a 64bit bar above 4GB on AMD Family 15h CPUs/APUs. Doing so is actually trivial and I plan to extend this to all recent AMD CPUs/APUs. Open questions: 1. Is this the right location to implement the 64bit BAR above 4GB for AMD CPUs? I guess that this needs a bit more cleanup. 2. Any idea how to better handle intermediate bridges? Reprogramming them isn't possible after drivers are loaded, but at least for GFX drivers we need to load them first to know how large we actually want our BAR to be (and to kick our vesafb/efifb). Thanks in advance, Christian. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel From: Christian König <christian.koenig@amd.com> Just the defines and helper functions to read the possible sizes of a BAR and update it's size. See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf. This is useful for hardware with large local storage (mostly GFX) which only expose 256MB BARs initially to be compatible with 32bit systems. v2: provide read helper as well v3: improve function names, use unsigned values, add better comments. Signed-off-by: Christian König <christian.koenig@amd.com> --- drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 3 ++ include/uapi/linux/pci_regs.h | 7 +++ 3 files changed, 125 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index ba34907..c2d9f78 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2944,6 +2944,121 @@ bool pci_acs_path_enabled(struct pci_dev *start, } /** + * pci_rbar_get_possible_sizes - get possible sizes for BAR + * @dev: PCI device + * @bar: BAR to query + * + * Get the possible sizes of a resizeable BAR as bitmask defined in the spec + * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable. + */ +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar) +{ + unsigned pos, nbars; + u32 ctrl, cap; + unsigned i; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return 0; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx != bar) + continue; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap); + return (cap & PCI_REBAR_CTRL_SIZES_MASK) >> + PCI_REBAR_CTRL_SIZES_SHIFT; + } + + return 0; +} + +/** + * pci_rbar_get_current_size - get the current size of a BAR + * @dev: PCI device + * @bar: BAR to set size to + * + * Read the size of a BAR from the resizeable BAR config. + * Returns size if found or negativ error code. + */ +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar) +{ + unsigned pos, nbars; + u32 ctrl; + unsigned i; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return -ENOTSUPP; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx != bar) + continue; + + return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >> + PCI_REBAR_CTRL_BAR_SIZE_SHIFT; + } + + return -ENOENT; +} + +/** + * pci_rbar_set_size - set a new size for a BAR + * @dev: PCI device + * @bar: BAR to set size to + * @size: new size as defined in the spec (log2(size in bytes) - 20) + * + * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB). + * Returns true if resizing was successful, false otherwise. + */ +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size) +{ + unsigned pos, nbars; + u32 ctrl; + unsigned i; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return -ENOTSUPP; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx != bar) + continue; + + ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE_MASK; + ctrl |= size << PCI_REBAR_CTRL_BAR_SIZE_SHIFT; + pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl); + return 0; + } + + return -ENOENT; +} + +/** * pci_swizzle_interrupt_pin - swizzle INTx for device behind bridge * @dev: the PCI device * @pin: the INTx pin (1=INTA, 2=INTB, 3=INTC, 4=INTD) diff --git a/include/linux/pci.h b/include/linux/pci.h index a38772a..6954e50 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1946,6 +1946,9 @@ void pci_request_acs(void); bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags); bool pci_acs_path_enabled(struct pci_dev *start, struct pci_dev *end, u16 acs_flags); +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar); +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar); +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size); #define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */ #define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT) diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index e5a2e68..6de29d6 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -932,9 +932,16 @@ #define PCI_SATA_SIZEOF_LONG 16 /* Resizable BARs */ +#define PCI_REBAR_CAP 4 /* capability register */ +#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */ +#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */ #define PCI_REBAR_CTRL 8 /* control register */ +#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for bar index */ +#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for bar index */ #define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */ #define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */ +#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for bar size */ +#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for bar size */ /* Dynamic Power Allocation */ #define PCI_DPA_CAP 4 /* capability register */ -- 2.7.4 ^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas-DgEjT+Ai2ygdnm+yROfE0A, linux-pci-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, linux-kernel-u79uwXL29TY76Z2rM5mHXA From: Christian König <christian.koenig@amd.com> Just the defines and helper functions to read the possible sizes of a BAR and update it's size. See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf. This is useful for hardware with large local storage (mostly GFX) which only expose 256MB BARs initially to be compatible with 32bit systems. v2: provide read helper as well v3: improve function names, use unsigned values, add better comments. Signed-off-by: Christian König <christian.koenig@amd.com> --- drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 3 ++ include/uapi/linux/pci_regs.h | 7 +++ 3 files changed, 125 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index ba34907..c2d9f78 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -2944,6 +2944,121 @@ bool pci_acs_path_enabled(struct pci_dev *start, } /** + * pci_rbar_get_possible_sizes - get possible sizes for BAR + * @dev: PCI device + * @bar: BAR to query + * + * Get the possible sizes of a resizeable BAR as bitmask defined in the spec + * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable. + */ +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar) +{ + unsigned pos, nbars; + u32 ctrl, cap; + unsigned i; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return 0; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx != bar) + continue; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap); + return (cap & PCI_REBAR_CTRL_SIZES_MASK) >> + PCI_REBAR_CTRL_SIZES_SHIFT; + } + + return 0; +} + +/** + * pci_rbar_get_current_size - get the current size of a BAR + * @dev: PCI device + * @bar: BAR to set size to + * + * Read the size of a BAR from the resizeable BAR config. + * Returns size if found or negativ error code. + */ +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar) +{ + unsigned pos, nbars; + u32 ctrl; + unsigned i; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return -ENOTSUPP; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx != bar) + continue; + + return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >> + PCI_REBAR_CTRL_BAR_SIZE_SHIFT; + } + + return -ENOENT; +} + +/** + * pci_rbar_set_size - set a new size for a BAR + * @dev: PCI device + * @bar: BAR to set size to + * @size: new size as defined in the spec (log2(size in bytes) - 20) + * + * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB). + * Returns true if resizing was successful, false otherwise. + */ +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size) +{ + unsigned pos, nbars; + u32 ctrl; + unsigned i; + + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); + if (!pos) + return -ENOTSUPP; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; + + for (i = 0; i < nbars; ++i, pos += 8) { + int bar_idx; + + pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); + bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> + PCI_REBAR_CTRL_BAR_IDX_SHIFT; + if (bar_idx != bar) + continue; + + ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE_MASK; + ctrl |= size << PCI_REBAR_CTRL_BAR_SIZE_SHIFT; + pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl); + return 0; + } + + return -ENOENT; +} + +/** * pci_swizzle_interrupt_pin - swizzle INTx for device behind bridge * @dev: the PCI device * @pin: the INTx pin (1=INTA, 2=INTB, 3=INTC, 4=INTD) diff --git a/include/linux/pci.h b/include/linux/pci.h index a38772a..6954e50 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1946,6 +1946,9 @@ void pci_request_acs(void); bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags); bool pci_acs_path_enabled(struct pci_dev *start, struct pci_dev *end, u16 acs_flags); +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar); +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar); +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size); #define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */ #define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT) diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h index e5a2e68..6de29d6 100644 --- a/include/uapi/linux/pci_regs.h +++ b/include/uapi/linux/pci_regs.h @@ -932,9 +932,16 @@ #define PCI_SATA_SIZEOF_LONG 16 /* Resizable BARs */ +#define PCI_REBAR_CAP 4 /* capability register */ +#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */ +#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */ #define PCI_REBAR_CTRL 8 /* control register */ +#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for bar index */ +#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for bar index */ #define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */ #define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */ +#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for bar size */ +#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for bar size */ /* Dynamic Power Allocation */ #define PCI_DPA_CAP 4 /* capability register */ -- 2.7.4 _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply related [flat|nested] 72+ messages in thread
* Re: [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 2017-03-13 12:41 ` Christian König @ 2017-03-14 13:09 ` kbuild test robot -1 siblings, 0 replies; 72+ messages in thread From: kbuild test robot @ 2017-03-14 13:09 UTC (permalink / raw) To: Christian König Cc: kbuild-all, helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel [-- Attachment #1: Type: text/plain, Size: 4455 bytes --] Hi Christian, [auto build test WARNING on pci/next] [also build test WARNING on v4.11-rc2 next-20170310] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next reproduce: make htmldocs All warnings (new ones prefixed by >>): lib/crc32.c:148: warning: No description found for parameter 'tab)[256]' lib/crc32.c:148: warning: Excess function parameter 'tab' description in 'crc32_le_generic' lib/crc32.c:293: warning: No description found for parameter 'tab)[256]' lib/crc32.c:293: warning: Excess function parameter 'tab' description in 'crc32_be_generic' lib/crc32.c:1: warning: no structured comments found >> drivers/pci/pci.c:2951: warning: No description found for parameter 'pdev' >> drivers/pci/pci.c:2951: warning: Excess function parameter 'dev' description in 'pci_rbar_get_possible_sizes' drivers/pci/pci.c:2989: warning: No description found for parameter 'pdev' >> drivers/pci/pci.c:2989: warning: Excess function parameter 'dev' description in 'pci_rbar_get_current_size' drivers/pci/pci.c:3027: warning: No description found for parameter 'pdev' >> drivers/pci/pci.c:3027: warning: Excess function parameter 'dev' description in 'pci_rbar_set_size' vim +/pdev +2951 drivers/pci/pci.c 2945 * @bar: BAR to query 2946 * 2947 * Get the possible sizes of a resizeable BAR as bitmask defined in the spec 2948 * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable. 2949 */ 2950 u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar) > 2951 { 2952 unsigned pos, nbars; 2953 u32 ctrl, cap; 2954 unsigned i; 2955 2956 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); 2957 if (!pos) 2958 return 0; 2959 2960 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 2961 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; 2962 2963 for (i = 0; i < nbars; ++i, pos += 8) { 2964 int bar_idx; 2965 2966 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 2967 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> 2968 PCI_REBAR_CTRL_BAR_IDX_SHIFT; 2969 if (bar_idx != bar) 2970 continue; 2971 2972 pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap); 2973 return (cap & PCI_REBAR_CTRL_SIZES_MASK) >> 2974 PCI_REBAR_CTRL_SIZES_SHIFT; 2975 } 2976 2977 return 0; 2978 } 2979 2980 /** 2981 * pci_rbar_get_current_size - get the current size of a BAR 2982 * @dev: PCI device 2983 * @bar: BAR to set size to 2984 * 2985 * Read the size of a BAR from the resizeable BAR config. 2986 * Returns size if found or negativ error code. 2987 */ 2988 int pci_rbar_get_current_size(struct pci_dev *pdev, int bar) > 2989 { 2990 unsigned pos, nbars; 2991 u32 ctrl; 2992 unsigned i; 2993 2994 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); 2995 if (!pos) 2996 return -ENOTSUPP; 2997 2998 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 2999 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; 3000 3001 for (i = 0; i < nbars; ++i, pos += 8) { 3002 int bar_idx; 3003 3004 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 3005 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> 3006 PCI_REBAR_CTRL_BAR_IDX_SHIFT; 3007 if (bar_idx != bar) 3008 continue; 3009 3010 return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >> 3011 PCI_REBAR_CTRL_BAR_SIZE_SHIFT; 3012 } 3013 3014 return -ENOENT; 3015 } 3016 3017 /** 3018 * pci_rbar_set_size - set a new size for a BAR 3019 * @dev: PCI device 3020 * @bar: BAR to set size to 3021 * @size: new size as defined in the spec (log2(size in bytes) - 20) 3022 * 3023 * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB). 3024 * Returns true if resizing was successful, false otherwise. 3025 */ 3026 int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size) > 3027 { 3028 unsigned pos, nbars; 3029 u32 ctrl; 3030 unsigned i; --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 6576 bytes --] ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 @ 2017-03-14 13:09 ` kbuild test robot 0 siblings, 0 replies; 72+ messages in thread From: kbuild test robot @ 2017-03-14 13:09 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, linux-kernel, amd-gfx, platform-driver-x86, helgaas, kbuild-all [-- Attachment #1: Type: text/plain, Size: 4455 bytes --] Hi Christian, [auto build test WARNING on pci/next] [also build test WARNING on v4.11-rc2 next-20170310] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next reproduce: make htmldocs All warnings (new ones prefixed by >>): lib/crc32.c:148: warning: No description found for parameter 'tab)[256]' lib/crc32.c:148: warning: Excess function parameter 'tab' description in 'crc32_le_generic' lib/crc32.c:293: warning: No description found for parameter 'tab)[256]' lib/crc32.c:293: warning: Excess function parameter 'tab' description in 'crc32_be_generic' lib/crc32.c:1: warning: no structured comments found >> drivers/pci/pci.c:2951: warning: No description found for parameter 'pdev' >> drivers/pci/pci.c:2951: warning: Excess function parameter 'dev' description in 'pci_rbar_get_possible_sizes' drivers/pci/pci.c:2989: warning: No description found for parameter 'pdev' >> drivers/pci/pci.c:2989: warning: Excess function parameter 'dev' description in 'pci_rbar_get_current_size' drivers/pci/pci.c:3027: warning: No description found for parameter 'pdev' >> drivers/pci/pci.c:3027: warning: Excess function parameter 'dev' description in 'pci_rbar_set_size' vim +/pdev +2951 drivers/pci/pci.c 2945 * @bar: BAR to query 2946 * 2947 * Get the possible sizes of a resizeable BAR as bitmask defined in the spec 2948 * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable. 2949 */ 2950 u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar) > 2951 { 2952 unsigned pos, nbars; 2953 u32 ctrl, cap; 2954 unsigned i; 2955 2956 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); 2957 if (!pos) 2958 return 0; 2959 2960 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 2961 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; 2962 2963 for (i = 0; i < nbars; ++i, pos += 8) { 2964 int bar_idx; 2965 2966 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 2967 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> 2968 PCI_REBAR_CTRL_BAR_IDX_SHIFT; 2969 if (bar_idx != bar) 2970 continue; 2971 2972 pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap); 2973 return (cap & PCI_REBAR_CTRL_SIZES_MASK) >> 2974 PCI_REBAR_CTRL_SIZES_SHIFT; 2975 } 2976 2977 return 0; 2978 } 2979 2980 /** 2981 * pci_rbar_get_current_size - get the current size of a BAR 2982 * @dev: PCI device 2983 * @bar: BAR to set size to 2984 * 2985 * Read the size of a BAR from the resizeable BAR config. 2986 * Returns size if found or negativ error code. 2987 */ 2988 int pci_rbar_get_current_size(struct pci_dev *pdev, int bar) > 2989 { 2990 unsigned pos, nbars; 2991 u32 ctrl; 2992 unsigned i; 2993 2994 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR); 2995 if (!pos) 2996 return -ENOTSUPP; 2997 2998 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 2999 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT; 3000 3001 for (i = 0; i < nbars; ++i, pos += 8) { 3002 int bar_idx; 3003 3004 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl); 3005 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >> 3006 PCI_REBAR_CTRL_BAR_IDX_SHIFT; 3007 if (bar_idx != bar) 3008 continue; 3009 3010 return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >> 3011 PCI_REBAR_CTRL_BAR_SIZE_SHIFT; 3012 } 3013 3014 return -ENOENT; 3015 } 3016 3017 /** 3018 * pci_rbar_set_size - set a new size for a BAR 3019 * @dev: PCI device 3020 * @bar: BAR to set size to 3021 * @size: new size as defined in the spec (log2(size in bytes) - 20) 3022 * 3023 * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB). 3024 * Returns true if resizing was successful, false otherwise. 3025 */ 3026 int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size) > 3027 { 3028 unsigned pos, nbars; 3029 u32 ctrl; 3030 unsigned i; --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 6576 bytes --] [-- Attachment #3: Type: text/plain, Size: 160 bytes --] _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 2017-03-13 12:41 ` Christian König (?) (?) @ 2017-03-24 15:28 ` Bjorn Helgaas -1 siblings, 0 replies; 72+ messages in thread From: Bjorn Helgaas @ 2017-03-24 15:28 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 01:41:33PM +0100, Christian König wrote: > From: Christian König <christian.koenig@amd.com> > > Just the defines and helper functions to read the possible sizes of a BAR and > update it's size. s/it's/its/ > See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf. It's good to have the public ECN that anybody can read, but we should also have a reference to the full spec that incorporates it, e.g., PCIe r3.1, sec 7.22. > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -1946,6 +1946,9 @@ void pci_request_acs(void); > bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags); > bool pci_acs_path_enabled(struct pci_dev *start, > struct pci_dev *end, u16 acs_flags); > +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar); > +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar); > +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size); These should be declared in drivers/pci/pci.h unless they're needed outside drivers/pci. I hope they aren't needed outside, because they're not safe to use after the PCI core has claimed resources. > #define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */ > #define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT) > diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h > index e5a2e68..6de29d6 100644 > --- a/include/uapi/linux/pci_regs.h > +++ b/include/uapi/linux/pci_regs.h > @@ -932,9 +932,16 @@ > #define PCI_SATA_SIZEOF_LONG 16 > > /* Resizable BARs */ > +#define PCI_REBAR_CAP 4 /* capability register */ > +#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */ > +#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */ > #define PCI_REBAR_CTRL 8 /* control register */ > +#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for bar index */ > +#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for bar index */ > #define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */ > #define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */ > +#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for bar size */ > +#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for bar size */ s/bar/BAR/ several places above > /* Dynamic Power Allocation */ > #define PCI_DPA_CAP 4 /* capability register */ > -- > 2.7.4 > ^ permalink raw reply [flat|nested] 72+ messages in thread
* [PATCH 2/4] PCI: add functionality for resizing resources v2 2017-03-13 12:41 ` Christian König @ 2017-03-13 12:41 ` Christian König -1 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel From: Christian König <christian.koenig@amd.com> This allows device drivers to request resizing their BARs. The function only tries to reprogram the windows of the bridge directly above the requesting device and only the BAR of the same type (usually mem, 64bit, prefetchable). This is done to make sure not to disturb other drivers by changing the BARs of their devices. If reprogramming the bridge BAR fails the old status is restored and -ENOSPC returned to the calling device driver. v2: rebase on changes in rbar support Signed-off-by: Christian König <christian.koenig@amd.com> --- drivers/pci/setup-bus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/setup-res.c | 51 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 114 insertions(+) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index f30ca75..cfab2c7 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1923,6 +1923,67 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) } EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources); +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type) +{ + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | + IORESOURCE_PREFETCH | IORESOURCE_MEM_64; + + struct resource saved; + LIST_HEAD(add_list); + LIST_HEAD(fail_head); + struct pci_dev_resource *fail_res; + unsigned i; + int ret = 0; + + /* Release all children from the matching bridge resource */ + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { + struct resource *res = &bridge->resource[i]; + + if ((res->flags & type_mask) != (type & type_mask)) + continue; + + saved = *res; + if (res->parent) { + release_child_resources(res); + release_resource(res); + } + res->start = 0; + res->end = 0; + break; + } + + if (i == PCI_BRIDGE_RESOURCE_END) + return -ENOENT; + + __pci_bus_size_bridges(bridge->subordinate, &add_list); + __pci_bridge_assign_resources(bridge, &add_list, &fail_head); + BUG_ON(!list_empty(&add_list)); + + /* restore size and flags */ + list_for_each_entry(fail_res, &fail_head, list) { + struct resource *res = fail_res->res; + + res->start = fail_res->start; + res->end = fail_res->end; + res->flags = fail_res->flags; + } + + /* Revert to the old configuration */ + if (!list_empty(&fail_head)) { + struct resource *res = &bridge->resource[i]; + + res->start = saved.start; + res->end = saved.end; + res->flags = saved.flags; + + pci_claim_resource(bridge, i); + ret = -ENOSPC; + } + + free_list(&fail_head); + return ret; +} + void pci_assign_unassigned_bus_resources(struct pci_bus *bus) { struct pci_dev *dev; diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 9526e34..3bb1e29 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -363,6 +363,57 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz return 0; } +int pci_resize_resource(struct pci_dev *dev, int resno, int size) +{ + struct resource *res = dev->resource + resno; + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); + int old = pci_rbar_get_current_size(dev, resno); + u64 bytes = 1ULL << (size + 20); + int ret = 0; + + if (!sizes) + return -ENOTSUPP; + + if (!(sizes & (1 << size))) + return -EINVAL; + + if (old < 0) + return old; + + /* Make sure the resource isn't assigned before making it larger. */ + if (resource_size(res) < bytes && res->parent) { + release_resource(res); + res->end = resource_size(res) - 1; + res->start = 0; + if (resno < PCI_BRIDGE_RESOURCES) + pci_update_resource(dev, resno); + } + + ret = pci_rbar_set_size(dev, resno, size); + if (ret) + goto error_reassign; + + res->end = res->start + bytes - 1; + + ret = pci_reassign_bridge_resources(dev->bus->self, res->flags); + if (ret) + goto error_resize; + + pci_reenable_device(dev->bus->self); + return 0; + +error_resize: + pci_rbar_set_size(dev, resno, old); + res->end = res->start + (1ULL << (old + 20)) - 1; + +error_reassign: + pci_assign_unassigned_bus_resources(dev->bus); + pci_setup_bridge(dev->bus); + pci_reenable_device(dev->bus->self); + return ret; +} +EXPORT_SYMBOL(pci_resize_resource); + int pci_enable_resources(struct pci_dev *dev, int mask) { u16 cmd, old_cmd; diff --git a/include/linux/pci.h b/include/linux/pci.h index 6954e50..8eaebb4 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1055,6 +1055,7 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev); void pci_update_resource(struct pci_dev *dev, int resno); int __must_check pci_assign_resource(struct pci_dev *dev, int i); int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align); +int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size); int pci_select_bars(struct pci_dev *dev, unsigned long flags); bool pci_device_is_present(struct pci_dev *pdev); void pci_ignore_hotplug(struct pci_dev *dev); @@ -1135,6 +1136,7 @@ void pci_assign_unassigned_resources(void); void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge); void pci_assign_unassigned_bus_resources(struct pci_bus *bus); void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus); +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type); void pdev_enable_device(struct pci_dev *); int pci_enable_resources(struct pci_dev *, int mask); void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *), -- 2.7.4 ^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH 2/4] PCI: add functionality for resizing resources v2 @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel From: Christian König <christian.koenig@amd.com> This allows device drivers to request resizing their BARs. The function only tries to reprogram the windows of the bridge directly above the requesting device and only the BAR of the same type (usually mem, 64bit, prefetchable). This is done to make sure not to disturb other drivers by changing the BARs of their devices. If reprogramming the bridge BAR fails the old status is restored and -ENOSPC returned to the calling device driver. v2: rebase on changes in rbar support Signed-off-by: Christian König <christian.koenig@amd.com> --- drivers/pci/setup-bus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++ drivers/pci/setup-res.c | 51 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 114 insertions(+) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index f30ca75..cfab2c7 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1923,6 +1923,67 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) } EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources); +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type) +{ + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | + IORESOURCE_PREFETCH | IORESOURCE_MEM_64; + + struct resource saved; + LIST_HEAD(add_list); + LIST_HEAD(fail_head); + struct pci_dev_resource *fail_res; + unsigned i; + int ret = 0; + + /* Release all children from the matching bridge resource */ + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { + struct resource *res = &bridge->resource[i]; + + if ((res->flags & type_mask) != (type & type_mask)) + continue; + + saved = *res; + if (res->parent) { + release_child_resources(res); + release_resource(res); + } + res->start = 0; + res->end = 0; + break; + } + + if (i == PCI_BRIDGE_RESOURCE_END) + return -ENOENT; + + __pci_bus_size_bridges(bridge->subordinate, &add_list); + __pci_bridge_assign_resources(bridge, &add_list, &fail_head); + BUG_ON(!list_empty(&add_list)); + + /* restore size and flags */ + list_for_each_entry(fail_res, &fail_head, list) { + struct resource *res = fail_res->res; + + res->start = fail_res->start; + res->end = fail_res->end; + res->flags = fail_res->flags; + } + + /* Revert to the old configuration */ + if (!list_empty(&fail_head)) { + struct resource *res = &bridge->resource[i]; + + res->start = saved.start; + res->end = saved.end; + res->flags = saved.flags; + + pci_claim_resource(bridge, i); + ret = -ENOSPC; + } + + free_list(&fail_head); + return ret; +} + void pci_assign_unassigned_bus_resources(struct pci_bus *bus) { struct pci_dev *dev; diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 9526e34..3bb1e29 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -363,6 +363,57 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz return 0; } +int pci_resize_resource(struct pci_dev *dev, int resno, int size) +{ + struct resource *res = dev->resource + resno; + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); + int old = pci_rbar_get_current_size(dev, resno); + u64 bytes = 1ULL << (size + 20); + int ret = 0; + + if (!sizes) + return -ENOTSUPP; + + if (!(sizes & (1 << size))) + return -EINVAL; + + if (old < 0) + return old; + + /* Make sure the resource isn't assigned before making it larger. */ + if (resource_size(res) < bytes && res->parent) { + release_resource(res); + res->end = resource_size(res) - 1; + res->start = 0; + if (resno < PCI_BRIDGE_RESOURCES) + pci_update_resource(dev, resno); + } + + ret = pci_rbar_set_size(dev, resno, size); + if (ret) + goto error_reassign; + + res->end = res->start + bytes - 1; + + ret = pci_reassign_bridge_resources(dev->bus->self, res->flags); + if (ret) + goto error_resize; + + pci_reenable_device(dev->bus->self); + return 0; + +error_resize: + pci_rbar_set_size(dev, resno, old); + res->end = res->start + (1ULL << (old + 20)) - 1; + +error_reassign: + pci_assign_unassigned_bus_resources(dev->bus); + pci_setup_bridge(dev->bus); + pci_reenable_device(dev->bus->self); + return ret; +} +EXPORT_SYMBOL(pci_resize_resource); + int pci_enable_resources(struct pci_dev *dev, int mask) { u16 cmd, old_cmd; diff --git a/include/linux/pci.h b/include/linux/pci.h index 6954e50..8eaebb4 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1055,6 +1055,7 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev); void pci_update_resource(struct pci_dev *dev, int resno); int __must_check pci_assign_resource(struct pci_dev *dev, int i); int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align); +int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size); int pci_select_bars(struct pci_dev *dev, unsigned long flags); bool pci_device_is_present(struct pci_dev *pdev); void pci_ignore_hotplug(struct pci_dev *dev); @@ -1135,6 +1136,7 @@ void pci_assign_unassigned_resources(void); void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge); void pci_assign_unassigned_bus_resources(struct pci_bus *bus); void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus); +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type); void pdev_enable_device(struct pci_dev *); int pci_enable_resources(struct pci_dev *, int mask); void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *), -- 2.7.4 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply related [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 2017-03-13 12:41 ` Christian König (?) @ 2017-03-13 16:43 ` Andy Shevchenko -1 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:43 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 2:41 PM, Christian König <deathsimple@vodafone.de> wrote: > From: Christian König <christian.koenig@amd.com> > > This allows device drivers to request resizing their BARs. > > The function only tries to reprogram the windows of the bridge directly above > the requesting device and only the BAR of the same type (usually mem, 64bit, > prefetchable). This is done to make sure not to disturb other drivers by > changing the BARs of their devices. > > If reprogramming the bridge BAR fails the old status is restored and -ENOSPC > returned to the calling device driver. Some style comments. > v2: rebase on changes in rbar support This kind of comments usually goes after --- delimiter below. > Signed-off-by: Christian König <christian.koenig@amd.com> > --- ...here. > +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type) > +{ > + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | > + IORESOURCE_PREFETCH | IORESOURCE_MEM_64; > + > + struct resource saved; > + LIST_HEAD(add_list); > + LIST_HEAD(fail_head); > + struct pci_dev_resource *fail_res; Perhaps move before definition of 'saved'? > + unsigned i; > + int ret = 0; > + > + /* Release all children from the matching bridge resource */ > + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { > + struct resource *res = &bridge->resource[i]; > + > + if ((res->flags & type_mask) != (type & type_mask)) IIUC it would be if ((res->flags ^ type) & type_mask) (I consider 'diff' as XOR operation is more understandable, but it's up to you) > + continue; > + /* restore size and flags */ > + list_for_each_entry(fail_res, &fail_head, list) { > + struct resource *res = fail_res->res; > + > + res->start = fail_res->start; > + res->end = fail_res->end; > + res->flags = fail_res->flags; > + } > + > + /* Revert to the old configuration */ > + if (!list_empty(&fail_head)) { > + struct resource *res = &bridge->resource[i]; > + > + res->start = saved.start; > + res->end = saved.end; > + res->flags = saved.flags; Would *res = saved; work? > +int pci_resize_resource(struct pci_dev *dev, int resno, int size) > +{ > + struct resource *res = dev->resource + resno; > + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); > + int old = pci_rbar_get_current_size(dev, resno); > + u64 bytes = 1ULL << (size + 20); > + int ret = 0; > + I would put sizes = pci_rbar_get_possible_sizes(dev, resno); here > + if (!sizes) > + return -ENOTSUPP; > + > + if (!(sizes & (1 << size))) BIT(size) ? > + return -EINVAL; > + and old = pci_rbar_get_current_size(dev, resno); here > + if (old < 0) > + return old; > +error_resize: > + pci_rbar_set_size(dev, resno, old); > + res->end = res->start + (1ULL << (old + 20)) - 1; BIT_ULL(old + 20) ? -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 @ 2017-03-13 16:43 ` Andy Shevchenko 0 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:43 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 2:41 PM, Christian K=C3=B6nig <deathsimple@vodafone.de> wrote: > From: Christian K=C3=B6nig <christian.koenig@amd.com> > > This allows device drivers to request resizing their BARs. > > The function only tries to reprogram the windows of the bridge directly a= bove > the requesting device and only the BAR of the same type (usually mem, 64b= it, > prefetchable). This is done to make sure not to disturb other drivers by > changing the BARs of their devices. > > If reprogramming the bridge BAR fails the old status is restored and -ENO= SPC > returned to the calling device driver. Some style comments. > v2: rebase on changes in rbar support This kind of comments usually goes after --- delimiter below. > Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com> > --- ...here. > +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long = type) > +{ > + const unsigned long type_mask =3D IORESOURCE_IO | IORESOURCE_MEM = | > + IORESOURCE_PREFETCH | IORESOURCE_MEM_64; > + > + struct resource saved; > + LIST_HEAD(add_list); > + LIST_HEAD(fail_head); > + struct pci_dev_resource *fail_res; Perhaps move before definition of 'saved'? > + unsigned i; > + int ret =3D 0; > + > + /* Release all children from the matching bridge resource */ > + for (i =3D PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i= ) { > + struct resource *res =3D &bridge->resource[i]; > + > + if ((res->flags & type_mask) !=3D (type & type_mask)) IIUC it would be if ((res->flags ^ type) & type_mask) (I consider 'diff' as XOR operation is more understandable, but it's up to = you) > + continue; > + /* restore size and flags */ > + list_for_each_entry(fail_res, &fail_head, list) { > + struct resource *res =3D fail_res->res; > + > + res->start =3D fail_res->start; > + res->end =3D fail_res->end; > + res->flags =3D fail_res->flags; > + } > + > + /* Revert to the old configuration */ > + if (!list_empty(&fail_head)) { > + struct resource *res =3D &bridge->resource[i]; > + > + res->start =3D saved.start; > + res->end =3D saved.end; > + res->flags =3D saved.flags; Would *res =3D saved; work? > +int pci_resize_resource(struct pci_dev *dev, int resno, int size) > +{ > + struct resource *res =3D dev->resource + resno; > + u32 sizes =3D pci_rbar_get_possible_sizes(dev, resno); > + int old =3D pci_rbar_get_current_size(dev, resno); > + u64 bytes =3D 1ULL << (size + 20); > + int ret =3D 0; > + I would put sizes =3D pci_rbar_get_possible_sizes(dev, resno); here > + if (!sizes) > + return -ENOTSUPP; > + > + if (!(sizes & (1 << size))) BIT(size) ? > + return -EINVAL; > + and old =3D pci_rbar_get_current_size(dev, resno); here > + if (old < 0) > + return old; > +error_resize: > + pci_rbar_set_size(dev, resno, old); > + res->end =3D res->start + (1ULL << (old + 20)) - 1; BIT_ULL(old + 20) ? --=20 With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 @ 2017-03-13 16:43 ` Andy Shevchenko 0 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:43 UTC (permalink / raw) To: Christian König Cc: linux-pci, linux-kernel, amd-gfx, Platform Driver, helgaas, dri-devel On Mon, Mar 13, 2017 at 2:41 PM, Christian König <deathsimple@vodafone.de> wrote: > From: Christian König <christian.koenig@amd.com> > > This allows device drivers to request resizing their BARs. > > The function only tries to reprogram the windows of the bridge directly above > the requesting device and only the BAR of the same type (usually mem, 64bit, > prefetchable). This is done to make sure not to disturb other drivers by > changing the BARs of their devices. > > If reprogramming the bridge BAR fails the old status is restored and -ENOSPC > returned to the calling device driver. Some style comments. > v2: rebase on changes in rbar support This kind of comments usually goes after --- delimiter below. > Signed-off-by: Christian König <christian.koenig@amd.com> > --- ...here. > +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type) > +{ > + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | > + IORESOURCE_PREFETCH | IORESOURCE_MEM_64; > + > + struct resource saved; > + LIST_HEAD(add_list); > + LIST_HEAD(fail_head); > + struct pci_dev_resource *fail_res; Perhaps move before definition of 'saved'? > + unsigned i; > + int ret = 0; > + > + /* Release all children from the matching bridge resource */ > + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { > + struct resource *res = &bridge->resource[i]; > + > + if ((res->flags & type_mask) != (type & type_mask)) IIUC it would be if ((res->flags ^ type) & type_mask) (I consider 'diff' as XOR operation is more understandable, but it's up to you) > + continue; > + /* restore size and flags */ > + list_for_each_entry(fail_res, &fail_head, list) { > + struct resource *res = fail_res->res; > + > + res->start = fail_res->start; > + res->end = fail_res->end; > + res->flags = fail_res->flags; > + } > + > + /* Revert to the old configuration */ > + if (!list_empty(&fail_head)) { > + struct resource *res = &bridge->resource[i]; > + > + res->start = saved.start; > + res->end = saved.end; > + res->flags = saved.flags; Would *res = saved; work? > +int pci_resize_resource(struct pci_dev *dev, int resno, int size) > +{ > + struct resource *res = dev->resource + resno; > + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); > + int old = pci_rbar_get_current_size(dev, resno); > + u64 bytes = 1ULL << (size + 20); > + int ret = 0; > + I would put sizes = pci_rbar_get_possible_sizes(dev, resno); here > + if (!sizes) > + return -ENOTSUPP; > + > + if (!(sizes & (1 << size))) BIT(size) ? > + return -EINVAL; > + and old = pci_rbar_get_current_size(dev, resno); here > + if (old < 0) > + return old; > +error_resize: > + pci_rbar_set_size(dev, resno, old); > + res->end = res->start + (1ULL << (old + 20)) - 1; BIT_ULL(old + 20) ? -- With Best Regards, Andy Shevchenko _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 @ 2017-04-11 9:14 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-04-11 9:14 UTC (permalink / raw) To: Andy Shevchenko Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel Sorry for the delayed response, have been busy with other stuff recently. Am 13.03.2017 um 17:43 schrieb Andy Shevchenko: >> v2: rebase on changes in rbar support > This kind of comments usually goes after --- delimiter below. That would remove it from the commit message which I don't want. >> + unsigned i; >> + int ret = 0; >> + >> + /* Release all children from the matching bridge resource */ >> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { >> + struct resource *res = &bridge->resource[i]; >> + > >> + if ((res->flags & type_mask) != (type & type_mask)) > IIUC it would be > if ((res->flags ^ type) & type_mask) > > (I consider 'diff' as XOR operation is more understandable, but it's up to you) I think like it is is easier to read. >> + res->start = saved.start; >> + res->end = saved.end; >> + res->flags = saved.flags; > Would > *res = saved; > work? No, res also contains a bunch of pointers into the tree which we should not override. >> +int pci_resize_resource(struct pci_dev *dev, int resno, int size) >> +{ >> + struct resource *res = dev->resource + resno; >> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); >> + int old = pci_rbar_get_current_size(dev, resno); >> + u64 bytes = 1ULL << (size + 20); >> + int ret = 0; >> + > I would put > sizes = pci_rbar_get_possible_sizes(dev, resno); > here Good idea, done. >> + if (!sizes) >> + return -ENOTSUPP; >> + >> + if (!(sizes & (1 << size))) > BIT(size) ? Done. > and > old = pci_rbar_get_current_size(dev, resno); > here Good idea as well. >> +error_resize: >> + pci_rbar_set_size(dev, resno, old); >> + res->end = res->start + (1ULL << (old + 20)) - 1; > BIT_ULL(old + 20) ? Nope, that is actually a size in bytes. Not a bitfield. So while BIT_ULL yields the right result it would be harder to understand. Christian. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 @ 2017-04-11 9:14 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-04-11 9:14 UTC (permalink / raw) To: Andy Shevchenko Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Platform Driver, helgaas-DgEjT+Ai2ygdnm+yROfE0A, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Sorry for the delayed response, have been busy with other stuff recently. Am 13.03.2017 um 17:43 schrieb Andy Shevchenko: >> v2: rebase on changes in rbar support > This kind of comments usually goes after --- delimiter below. That would remove it from the commit message which I don't want. >> + unsigned i; >> + int ret = 0; >> + >> + /* Release all children from the matching bridge resource */ >> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { >> + struct resource *res = &bridge->resource[i]; >> + > >> + if ((res->flags & type_mask) != (type & type_mask)) > IIUC it would be > if ((res->flags ^ type) & type_mask) > > (I consider 'diff' as XOR operation is more understandable, but it's up to you) I think like it is is easier to read. >> + res->start = saved.start; >> + res->end = saved.end; >> + res->flags = saved.flags; > Would > *res = saved; > work? No, res also contains a bunch of pointers into the tree which we should not override. >> +int pci_resize_resource(struct pci_dev *dev, int resno, int size) >> +{ >> + struct resource *res = dev->resource + resno; >> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); >> + int old = pci_rbar_get_current_size(dev, resno); >> + u64 bytes = 1ULL << (size + 20); >> + int ret = 0; >> + > I would put > sizes = pci_rbar_get_possible_sizes(dev, resno); > here Good idea, done. >> + if (!sizes) >> + return -ENOTSUPP; >> + >> + if (!(sizes & (1 << size))) > BIT(size) ? Done. > and > old = pci_rbar_get_current_size(dev, resno); > here Good idea as well. >> +error_resize: >> + pci_rbar_set_size(dev, resno, old); >> + res->end = res->start + (1ULL << (old + 20)) - 1; > BIT_ULL(old + 20) ? Nope, that is actually a size in bytes. Not a bitfield. So while BIT_ULL yields the right result it would be harder to understand. Christian. _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 2017-03-13 12:41 ` Christian König (?) (?) @ 2017-03-14 9:01 ` kbuild test robot -1 siblings, 0 replies; 72+ messages in thread From: kbuild test robot @ 2017-03-14 9:01 UTC (permalink / raw) To: Christian König Cc: kbuild-all, helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel [-- Attachment #1: Type: text/plain, Size: 2131 bytes --] Hi Christian, [auto build test WARNING on pci/next] [also build test WARNING on v4.11-rc2 next-20170310] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next config: i386-randconfig-x077-201711 (attached as .config) compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901 reproduce: # save the attached .config to linux build tree make ARCH=i386 All warnings (new ones prefixed by >>): drivers/pci/setup-res.c: In function 'pci_resize_resource': >> drivers/pci/setup-res.c:422:2: warning: ignoring return value of 'pci_reenable_device', declared with attribute warn_unused_result [-Wunused-result] pci_reenable_device(dev->bus->self); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ drivers/pci/setup-res.c:432:2: warning: ignoring return value of 'pci_reenable_device', declared with attribute warn_unused_result [-Wunused-result] pci_reenable_device(dev->bus->self); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ vim +/pci_reenable_device +422 drivers/pci/setup-res.c 406 res->end = resource_size(res) - 1; 407 res->start = 0; 408 if (resno < PCI_BRIDGE_RESOURCES) 409 pci_update_resource(dev, resno); 410 } 411 412 ret = pci_rbar_set_size(dev, resno, size); 413 if (ret) 414 goto error_reassign; 415 416 res->end = res->start + bytes - 1; 417 418 ret = pci_reassign_bridge_resources(dev->bus->self, res->flags); 419 if (ret) 420 goto error_resize; 421 > 422 pci_reenable_device(dev->bus->self); 423 return 0; 424 425 error_resize: 426 pci_rbar_set_size(dev, resno, old); 427 res->end = res->start + (1ULL << (old + 20)) - 1; 428 429 error_reassign: 430 pci_assign_unassigned_bus_resources(dev->bus); --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 24132 bytes --] ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 2017-03-13 12:41 ` Christian König ` (2 preceding siblings ...) (?) @ 2017-03-24 21:34 ` Bjorn Helgaas 2017-04-11 15:37 ` Christian König -1 siblings, 1 reply; 72+ messages in thread From: Bjorn Helgaas @ 2017-03-24 21:34 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 01:41:34PM +0100, Christian König wrote: > From: Christian König <christian.koenig@amd.com> > > This allows device drivers to request resizing their BARs. > > The function only tries to reprogram the windows of the bridge directly above > the requesting device and only the BAR of the same type (usually mem, 64bit, > prefetchable). This is done to make sure not to disturb other drivers by > changing the BARs of their devices. > > If reprogramming the bridge BAR fails the old status is restored and -ENOSPC > returned to the calling device driver. > > v2: rebase on changes in rbar support > > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > drivers/pci/setup-bus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++ > drivers/pci/setup-res.c | 51 +++++++++++++++++++++++++++++++++++++++++ > include/linux/pci.h | 2 ++ > 3 files changed, 114 insertions(+) > > diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c > index f30ca75..cfab2c7 100644 > --- a/drivers/pci/setup-bus.c > +++ b/drivers/pci/setup-bus.c > @@ -1923,6 +1923,67 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) > } > EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources); > > +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type) > +{ > + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM | > + IORESOURCE_PREFETCH | IORESOURCE_MEM_64; > + > + struct resource saved; > + LIST_HEAD(add_list); > + LIST_HEAD(fail_head); > + struct pci_dev_resource *fail_res; > + unsigned i; > + int ret = 0; > + > + /* Release all children from the matching bridge resource */ > + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) { Nit: use post-increment unless you need pre-increment. > + struct resource *res = &bridge->resource[i]; > + > + if ((res->flags & type_mask) != (type & type_mask)) > + continue; > + > + saved = *res; > + if (res->parent) { > + release_child_resources(res); Doesn't this recursively release *all* child resources? There could be BARs from several devices, or even windows of downstream bridges, inside this window. The drivers of those other devices aren't expecting things to change here. > + release_resource(res); > + } > + res->start = 0; > + res->end = 0; > + break; > + } > + > + if (i == PCI_BRIDGE_RESOURCE_END) > + return -ENOENT; > + > + __pci_bus_size_bridges(bridge->subordinate, &add_list); > + __pci_bridge_assign_resources(bridge, &add_list, &fail_head); > + BUG_ON(!list_empty(&add_list)); > + > + /* restore size and flags */ > + list_for_each_entry(fail_res, &fail_head, list) { > + struct resource *res = fail_res->res; > + > + res->start = fail_res->start; > + res->end = fail_res->end; > + res->flags = fail_res->flags; > + } > + > + /* Revert to the old configuration */ > + if (!list_empty(&fail_head)) { > + struct resource *res = &bridge->resource[i]; > + > + res->start = saved.start; > + res->end = saved.end; > + res->flags = saved.flags; > + > + pci_claim_resource(bridge, i); > + ret = -ENOSPC; > + } > + > + free_list(&fail_head); > + return ret; > +} > + > void pci_assign_unassigned_bus_resources(struct pci_bus *bus) > { > struct pci_dev *dev; > diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c > index 9526e34..3bb1e29 100644 > --- a/drivers/pci/setup-res.c > +++ b/drivers/pci/setup-res.c > @@ -363,6 +363,57 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz > return 0; > } > > +int pci_resize_resource(struct pci_dev *dev, int resno, int size) > +{ > + struct resource *res = dev->resource + resno; > + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); > + int old = pci_rbar_get_current_size(dev, resno); > + u64 bytes = 1ULL << (size + 20); > + int ret = 0; I think we should fail the request if the device is enabled, i.e., if the PCI_COMMAND_MEMORY bit is set. We can't safely change the BAR while memory decoding is enabled. I know there's code in pci_std_update_resource() that turns off PCI_COMMAND_MEMORY, but I think that's a mistake: I think it should fail when asked to update an enabled BAR the same way pci_iov_update_resource() does. I'm not sure why you call pci_reenable_device() below, but I think I would rather have the driver do something like this: pci_disable_device(dev); pci_resize_resource(dev, 0, size); pci_enable_device(dev); That way it's very clear to the driver that it must re-read all BARs after resizing this one. > + if (!sizes) > + return -ENOTSUPP; > + > + if (!(sizes & (1 << size))) > + return -EINVAL; > + > + if (old < 0) > + return old; > + > + /* Make sure the resource isn't assigned before making it larger. */ > + if (resource_size(res) < bytes && res->parent) { > + release_resource(res); > + res->end = resource_size(res) - 1; > + res->start = 0; > + if (resno < PCI_BRIDGE_RESOURCES) > + pci_update_resource(dev, resno); Why do we need to write this zero to the BAR? If PCI_COMMAND_MEMORY is set, I think this is dangerous, and if it's not set, I think it's unnecessary. I think we should set the IORESOURCE_UNSET bit to indicate that the resource does not have an address assigned to it. It should get cleared later anyway, but having IORESOURCE_UNSET will make any debug messages emitted while reassigning resources make more sense. > + } > + > + ret = pci_rbar_set_size(dev, resno, size); > + if (ret) > + goto error_reassign; > + > + res->end = res->start + bytes - 1; > + > + ret = pci_reassign_bridge_resources(dev->bus->self, res->flags); > + if (ret) > + goto error_resize; > + > + pci_reenable_device(dev->bus->self); > + return 0; > + > +error_resize: > + pci_rbar_set_size(dev, resno, old); > + res->end = res->start + (1ULL << (old + 20)) - 1; > + > +error_reassign: > + pci_assign_unassigned_bus_resources(dev->bus); > + pci_setup_bridge(dev->bus); Could this bridge-related recovery code go inside pci_reassign_bridge_resources()? > + pci_reenable_device(dev->bus->self); > + return ret; > +} > +EXPORT_SYMBOL(pci_resize_resource); > + > int pci_enable_resources(struct pci_dev *dev, int mask) > { > u16 cmd, old_cmd; > diff --git a/include/linux/pci.h b/include/linux/pci.h > index 6954e50..8eaebb4 100644 > --- a/include/linux/pci.h > +++ b/include/linux/pci.h > @@ -1055,6 +1055,7 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev); > void pci_update_resource(struct pci_dev *dev, int resno); > int __must_check pci_assign_resource(struct pci_dev *dev, int i); > int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align); > +int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size); > int pci_select_bars(struct pci_dev *dev, unsigned long flags); > bool pci_device_is_present(struct pci_dev *pdev); > void pci_ignore_hotplug(struct pci_dev *dev); > @@ -1135,6 +1136,7 @@ void pci_assign_unassigned_resources(void); > void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge); > void pci_assign_unassigned_bus_resources(struct pci_bus *bus); > void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus); > +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type); > void pdev_enable_device(struct pci_dev *); > int pci_enable_resources(struct pci_dev *, int mask); > void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *), > -- > 2.7.4 > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 2017-03-24 21:34 ` Bjorn Helgaas @ 2017-04-11 15:37 ` Christian König 2017-04-12 16:37 ` Bjorn Helgaas 0 siblings, 1 reply; 72+ messages in thread From: Christian König @ 2017-04-11 15:37 UTC (permalink / raw) To: Bjorn Helgaas Cc: linux-pci, linux-kernel, amd-gfx, dri-devel, platform-driver-x86 Hi Bjorn, first of all sorry for the delay, had been busy with other stuff in the last few weeks. Am 24.03.2017 um 22:34 schrieb Bjorn Helgaas: >> + release_child_resources(res); > Doesn't this recursively release *all* child resources? There could > be BARs from several devices, or even windows of downstream bridges, > inside this window. The drivers of those other devices aren't > expecting things to change here. Yes, the original idea was that the driver calling this knows that the BARs will be changed for the bridge it belongs to. But you're right. I've changed it in the way that our device driver will release all resource first and then call the function to resize its BAR. The function will return an error in the next version of the patch if the bridge BAR which needs to be moved for this is still used by child resources. >> +int pci_resize_resource(struct pci_dev *dev, int resno, int size) >> +{ >> + struct resource *res = dev->resource + resno; >> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno); >> + int old = pci_rbar_get_current_size(dev, resno); >> + u64 bytes = 1ULL << (size + 20); >> + int ret = 0; > I think we should fail the request if the device is enabled, i.e., if > the PCI_COMMAND_MEMORY bit is set. We can't safely change the BAR > while memory decoding is enabled. > > I know there's code in pci_std_update_resource() that turns off > PCI_COMMAND_MEMORY, but I think that's a mistake: I think it should > fail when asked to update an enabled BAR the same way > pci_iov_update_resource() does. > > I'm not sure why you call pci_reenable_device() below, but I think I > would rather have the driver do something like this: > > pci_disable_device(dev); > pci_resize_resource(dev, 0, size); > pci_enable_device(dev); > > That way it's very clear to the driver that it must re-read all BARs > after resizing this one. I've tried it, but this actually doesn't seem to work. At least on the board I've tried pci_disable_device() doesn't clear the PCI_COMMAND_MEMORY bit, it just clears the master bit. Additional to that the power management reference counting pci_disable_device() and pci_enable_device() doesn't look like what I want for this functionality. How about the driver needs to disabled memory decoding itself before trying to resize anything? >> + if (!sizes) >> + return -ENOTSUPP; >> + >> + if (!(sizes & (1 << size))) >> + return -EINVAL; >> + >> + if (old < 0) >> + return old; >> + >> + /* Make sure the resource isn't assigned before making it larger. */ >> + if (resource_size(res) < bytes && res->parent) { >> + release_resource(res); >> + res->end = resource_size(res) - 1; >> + res->start = 0; >> + if (resno < PCI_BRIDGE_RESOURCES) >> + pci_update_resource(dev, resno); > Why do we need to write this zero to the BAR? If PCI_COMMAND_MEMORY > is set, I think this is dangerous, and if it's not set, I think it's > unnecessary. > > I think we should set the IORESOURCE_UNSET bit to indicate that the > resource does not have an address assigned to it. It should get > cleared later anyway, but having IORESOURCE_UNSET will make any debug > messages emitted while reassigning resources make more sense. Makes sense, changed. >> + return 0; >> + >> +error_resize: >> + pci_rbar_set_size(dev, resno, old); >> + res->end = res->start + (1ULL << (old + 20)) - 1; >> + >> +error_reassign: >> + pci_assign_unassigned_bus_resources(dev->bus); >> + pci_setup_bridge(dev->bus); > Could this bridge-related recovery code go inside > pci_reassign_bridge_resources()? No, we need to restore the original size of the resource before trying the recovery code when something goes wrong. I will address all other comments in the next version of the patch as well. Regards, Christian. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2 2017-04-11 15:37 ` Christian König @ 2017-04-12 16:37 ` Bjorn Helgaas 0 siblings, 0 replies; 72+ messages in thread From: Bjorn Helgaas @ 2017-04-12 16:37 UTC (permalink / raw) To: Christian König Cc: linux-pci, linux-kernel, amd-gfx, dri-devel, platform-driver-x86 On Tue, Apr 11, 2017 at 05:37:04PM +0200, Christian König wrote: > >>+int pci_resize_resource(struct pci_dev *dev, int resno, int size) > >>+{ > >>+ struct resource *res = dev->resource + resno; > >>+ u32 sizes = pci_rbar_get_possible_sizes(dev, resno); > >>+ int old = pci_rbar_get_current_size(dev, resno); > >>+ u64 bytes = 1ULL << (size + 20); > >>+ int ret = 0; > >I think we should fail the request if the device is enabled, i.e., if > >the PCI_COMMAND_MEMORY bit is set. We can't safely change the BAR > >while memory decoding is enabled. > > > >I know there's code in pci_std_update_resource() that turns off > >PCI_COMMAND_MEMORY, but I think that's a mistake: I think it should > >fail when asked to update an enabled BAR the same way > >pci_iov_update_resource() does. > > > >I'm not sure why you call pci_reenable_device() below, but I think I > >would rather have the driver do something like this: > > > > pci_disable_device(dev); > > pci_resize_resource(dev, 0, size); > > pci_enable_device(dev); > > > >That way it's very clear to the driver that it must re-read all BARs > >after resizing this one. > > I've tried it, but this actually doesn't seem to work. > > At least on the board I've tried pci_disable_device() doesn't clear > the PCI_COMMAND_MEMORY bit, it just clears the master bit. Yeah, you're right. We generally turn *on* PCI_COMMAND_MEMORY in the pci_enable_device() path, but the pci_disable_device() path doesn't turn it off. > Additional to that the power management reference counting > pci_disable_device() and pci_enable_device() doesn't look like what > I want for this functionality. > > How about the driver needs to disabled memory decoding itself before > trying to resize anything? There are a few places that do that, so maybe that's the best option. Bjorn ^ permalink raw reply [flat|nested] 72+ messages in thread
* [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel From: Christian König <christian.koenig@amd.com> Most BIOS don't enable this because of compatibility reasons. Manually enable a 64bit BAR of 64GB size so that we have enough room for PCI devices. Signed-off-by: Christian König <christian.koenig@amd.com> --- arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index 6d52b94..bff5242 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar); DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar); DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar); DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar); + +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) +{ + const uint64_t size = 64ULL * 1024 * 1024 * 1024; + uint32_t base, limit, high; + struct resource *res; + unsigned i; + int r; + + for (i = 0; i < 8; ++i) { + + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); + + /* Is this slot free? */ + if ((base & 0x3) == 0x0) + break; + + base >>= 8; + base |= high << 24; + + /* Abort if a slot already configures a 64bit BAR. */ + if (base > 0x10000) + return; + + } + + if (i == 8) + return; + + res = kzalloc(sizeof(*res), GFP_KERNEL); + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | + IORESOURCE_WINDOW; + res->name = dev->bus->name; + r = allocate_resource(&iomem_resource, res, size, 0x100000000, + 0xfd00000000, size, NULL, NULL); + if (r) { + kfree(res); + return; + } + + base = ((res->start >> 8) & 0xffffff00) | 0x3; + limit = ((res->end + 1) >> 8) & 0xffffff00; + high = ((res->start >> 40) & 0xff) | + ((((res->end + 1) >> 40) & 0xff) << 16); + + pci_write_config_dword(dev, 0x180 + i * 0x4, high); + pci_write_config_dword(dev, 0x84 + i * 0x8, limit); + pci_write_config_dword(dev, 0x80 + i * 0x8, base); + + pci_bus_add_resource(dev->bus, res, 0); +} +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); -- 2.7.4 ^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas-DgEjT+Ai2ygdnm+yROfE0A, linux-pci-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, linux-kernel-u79uwXL29TY76Z2rM5mHXA From: Christian König <christian.koenig@amd.com> Most BIOS don't enable this because of compatibility reasons. Manually enable a 64bit BAR of 64GB size so that we have enough room for PCI devices. Signed-off-by: Christian König <christian.koenig@amd.com> --- arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c index 6d52b94..bff5242 100644 --- a/arch/x86/pci/fixup.c +++ b/arch/x86/pci/fixup.c @@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar); DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar); DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar); DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar); + +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) +{ + const uint64_t size = 64ULL * 1024 * 1024 * 1024; + uint32_t base, limit, high; + struct resource *res; + unsigned i; + int r; + + for (i = 0; i < 8; ++i) { + + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); + + /* Is this slot free? */ + if ((base & 0x3) == 0x0) + break; + + base >>= 8; + base |= high << 24; + + /* Abort if a slot already configures a 64bit BAR. */ + if (base > 0x10000) + return; + + } + + if (i == 8) + return; + + res = kzalloc(sizeof(*res), GFP_KERNEL); + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | + IORESOURCE_WINDOW; + res->name = dev->bus->name; + r = allocate_resource(&iomem_resource, res, size, 0x100000000, + 0xfd00000000, size, NULL, NULL); + if (r) { + kfree(res); + return; + } + + base = ((res->start >> 8) & 0xffffff00) | 0x3; + limit = ((res->end + 1) >> 8) & 0xffffff00; + high = ((res->start >> 40) & 0xff) | + ((((res->end + 1) >> 40) & 0xff) << 16); + + pci_write_config_dword(dev, 0x180 + i * 0x4, high); + pci_write_config_dword(dev, 0x84 + i * 0x8, limit); + pci_write_config_dword(dev, 0x80 + i * 0x8, base); + + pci_bus_add_resource(dev->bus, res, 0); +} +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); -- 2.7.4 _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply related [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors 2017-03-13 12:41 ` Christian König (?) @ 2017-03-13 16:49 ` Andy Shevchenko -1 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:49 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 2:41 PM, Christian König <deathsimple@vodafone.de> wrote: > Most BIOS don't enable this because of compatibility reasons. > > Manually enable a 64bit BAR of 64GB size so that we have > enough room for PCI devices. > +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > +{ > + const uint64_t size = 64ULL * 1024 * 1024 * 1024; Perhaps extend <linux/sizes.h> and use SZ_64G here? It would be nice to do, since some of the drivers already are using sizes like 4GB and alike. > + uint32_t base, limit, high; > + struct resource *res; > + unsigned i; > + int r; > + > + for (i = 0; i < 8; ++i) { > + Redundant empty line. > + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); > + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); > + > + /* Is this slot free? */ > + if ((base & 0x3) == 0x0) > + break; > + > + base >>= 8; > + base |= high << 24; > + > + /* Abort if a slot already configures a 64bit BAR. */ > + if (base > 0x10000) > + return; > + Ditto. > + } > + Ditto. > + if (i == 8) > + return; > + > + res = kzalloc(sizeof(*res), GFP_KERNEL); > + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | > + IORESOURCE_WINDOW; > + res->name = dev->bus->name; > + r = allocate_resource(&iomem_resource, res, size, 0x100000000, > + 0xfd00000000, size, NULL, NULL); > + if (r) { > + kfree(res); > + return; > + } > + > + base = ((res->start >> 8) & 0xffffff00) | 0x3; > + limit = ((res->end + 1) >> 8) & 0xffffff00; > + high = ((res->start >> 40) & 0xff) | > + ((((res->end + 1) >> 40) & 0xff) << 16); Perhaps some of constants can be replaced by defines (I think some of them are already defined in ioport.h or somewhere else). -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-03-13 16:49 ` Andy Shevchenko 0 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:49 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 2:41 PM, Christian K=C3=B6nig <deathsimple@vodafone.de> wrote: > Most BIOS don't enable this because of compatibility reasons. > > Manually enable a 64bit BAR of 64GB size so that we have > enough room for PCI devices. > +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > +{ > + const uint64_t size =3D 64ULL * 1024 * 1024 * 1024; Perhaps extend <linux/sizes.h> and use SZ_64G here? It would be nice to do, since some of the drivers already are using sizes like 4GB and alike. > + uint32_t base, limit, high; > + struct resource *res; > + unsigned i; > + int r; > + > + for (i =3D 0; i < 8; ++i) { > + Redundant empty line. > + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); > + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); > + > + /* Is this slot free? */ > + if ((base & 0x3) =3D=3D 0x0) > + break; > + > + base >>=3D 8; > + base |=3D high << 24; > + > + /* Abort if a slot already configures a 64bit BAR. */ > + if (base > 0x10000) > + return; > + Ditto. > + } > + Ditto. > + if (i =3D=3D 8) > + return; > + > + res =3D kzalloc(sizeof(*res), GFP_KERNEL); > + res->flags =3D IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_= MEM_64 | > + IORESOURCE_WINDOW; > + res->name =3D dev->bus->name; > + r =3D allocate_resource(&iomem_resource, res, size, 0x100000000, > + 0xfd00000000, size, NULL, NULL); > + if (r) { > + kfree(res); > + return; > + } > + > + base =3D ((res->start >> 8) & 0xffffff00) | 0x3; > + limit =3D ((res->end + 1) >> 8) & 0xffffff00; > + high =3D ((res->start >> 40) & 0xff) | > + ((((res->end + 1) >> 40) & 0xff) << 16); Perhaps some of constants can be replaced by defines (I think some of them are already defined in ioport.h or somewhere else). --=20 With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-03-13 16:49 ` Andy Shevchenko 0 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:49 UTC (permalink / raw) To: Christian König Cc: linux-pci, linux-kernel, amd-gfx, Platform Driver, helgaas, dri-devel On Mon, Mar 13, 2017 at 2:41 PM, Christian König <deathsimple@vodafone.de> wrote: > Most BIOS don't enable this because of compatibility reasons. > > Manually enable a 64bit BAR of 64GB size so that we have > enough room for PCI devices. > +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > +{ > + const uint64_t size = 64ULL * 1024 * 1024 * 1024; Perhaps extend <linux/sizes.h> and use SZ_64G here? It would be nice to do, since some of the drivers already are using sizes like 4GB and alike. > + uint32_t base, limit, high; > + struct resource *res; > + unsigned i; > + int r; > + > + for (i = 0; i < 8; ++i) { > + Redundant empty line. > + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); > + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); > + > + /* Is this slot free? */ > + if ((base & 0x3) == 0x0) > + break; > + > + base >>= 8; > + base |= high << 24; > + > + /* Abort if a slot already configures a 64bit BAR. */ > + if (base > 0x10000) > + return; > + Ditto. > + } > + Ditto. > + if (i == 8) > + return; > + > + res = kzalloc(sizeof(*res), GFP_KERNEL); > + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | > + IORESOURCE_WINDOW; > + res->name = dev->bus->name; > + r = allocate_resource(&iomem_resource, res, size, 0x100000000, > + 0xfd00000000, size, NULL, NULL); > + if (r) { > + kfree(res); > + return; > + } > + > + base = ((res->start >> 8) & 0xffffff00) | 0x3; > + limit = ((res->end + 1) >> 8) & 0xffffff00; > + high = ((res->start >> 40) & 0xff) | > + ((((res->end + 1) >> 40) & 0xff) << 16); Perhaps some of constants can be replaced by defines (I think some of them are already defined in ioport.h or somewhere else). -- With Best Regards, Andy Shevchenko _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-04-11 9:21 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-04-11 9:21 UTC (permalink / raw) To: Andy Shevchenko Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel Am 13.03.2017 um 17:49 schrieb Andy Shevchenko: > On Mon, Mar 13, 2017 at 2:41 PM, Christian König > <deathsimple@vodafone.de> wrote: > >> Most BIOS don't enable this because of compatibility reasons. >> >> Manually enable a 64bit BAR of 64GB size so that we have >> enough room for PCI devices. >> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) >> +{ >> + const uint64_t size = 64ULL * 1024 * 1024 * 1024; > Perhaps extend <linux/sizes.h> and use SZ_64G here? > > It would be nice to do, since some of the drivers already are using > sizes like 4GB and alike. Actually using 64GB here was just for testing and to get some initial feedback. I think we want to use all the remaining address space for PCIe, but for this we would need a new function in the resource management I think. Going to take a deeper look when I'm sure we actually want this. >> + if (i == 8) >> + return; >> + >> + res = kzalloc(sizeof(*res), GFP_KERNEL); >> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | >> + IORESOURCE_WINDOW; >> + res->name = dev->bus->name; >> + r = allocate_resource(&iomem_resource, res, size, 0x100000000, >> + 0xfd00000000, size, NULL, NULL); >> + if (r) { >> + kfree(res); >> + return; >> + } >> + >> + base = ((res->start >> 8) & 0xffffff00) | 0x3; >> + limit = ((res->end + 1) >> 8) & 0xffffff00; >> + high = ((res->start >> 40) & 0xff) | >> + ((((res->end + 1) >> 40) & 0xff) << 16); > Perhaps some of constants can be replaced by defines (I think some of > them are already defined in ioport.h or somewhere else). Yeah, good idea. But that stuff is purely AMD CPU specific, so won't belong into ioport.h or similar common code. Does anybody have any idea where I could put this? Regards, Christian. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-04-11 9:21 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-04-11 9:21 UTC (permalink / raw) To: Andy Shevchenko Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Platform Driver, helgaas-DgEjT+Ai2ygdnm+yROfE0A, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Am 13.03.2017 um 17:49 schrieb Andy Shevchenko: > On Mon, Mar 13, 2017 at 2:41 PM, Christian König > <deathsimple@vodafone.de> wrote: > >> Most BIOS don't enable this because of compatibility reasons. >> >> Manually enable a 64bit BAR of 64GB size so that we have >> enough room for PCI devices. >> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) >> +{ >> + const uint64_t size = 64ULL * 1024 * 1024 * 1024; > Perhaps extend <linux/sizes.h> and use SZ_64G here? > > It would be nice to do, since some of the drivers already are using > sizes like 4GB and alike. Actually using 64GB here was just for testing and to get some initial feedback. I think we want to use all the remaining address space for PCIe, but for this we would need a new function in the resource management I think. Going to take a deeper look when I'm sure we actually want this. >> + if (i == 8) >> + return; >> + >> + res = kzalloc(sizeof(*res), GFP_KERNEL); >> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | >> + IORESOURCE_WINDOW; >> + res->name = dev->bus->name; >> + r = allocate_resource(&iomem_resource, res, size, 0x100000000, >> + 0xfd00000000, size, NULL, NULL); >> + if (r) { >> + kfree(res); >> + return; >> + } >> + >> + base = ((res->start >> 8) & 0xffffff00) | 0x3; >> + limit = ((res->end + 1) >> 8) & 0xffffff00; >> + high = ((res->start >> 40) & 0xff) | >> + ((((res->end + 1) >> 40) & 0xff) << 16); > Perhaps some of constants can be replaced by defines (I think some of > them are already defined in ioport.h or somewhere else). Yeah, good idea. But that stuff is purely AMD CPU specific, so won't belong into ioport.h or similar common code. Does anybody have any idea where I could put this? Regards, Christian. _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors 2017-03-13 12:41 ` Christian König @ 2017-03-14 9:25 ` kbuild test robot -1 siblings, 0 replies; 72+ messages in thread From: kbuild test robot @ 2017-03-14 9:25 UTC (permalink / raw) To: Christian König Cc: kbuild-all, helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel [-- Attachment #1: Type: text/plain, Size: 2346 bytes --] Hi Christian, [auto build test WARNING on pci/next] [also build test WARNING on v4.11-rc2 next-20170310] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next config: i386-randconfig-s0-201711 (attached as .config) compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901 reproduce: # save the attached .config to linux build tree make ARCH=i386 All warnings (new ones prefixed by >>): arch/x86/pci/fixup.c: In function 'pci_amd_enable_64bit_bar': >> arch/x86/pci/fixup.c:608:52: warning: large integer implicitly truncated to unsigned type [-Woverflow] r = allocate_resource(&iomem_resource, res, size, 0x100000000, ^~~~~~~~~~~ arch/x86/pci/fixup.c:609:10: warning: large integer implicitly truncated to unsigned type [-Woverflow] 0xfd00000000, size, NULL, NULL); ^~~~~~~~~~~~ >> arch/x86/pci/fixup.c:617:22: warning: right shift count >= width of type [-Wshift-count-overflow] high = ((res->start >> 40) & 0xff) | ^~ arch/x86/pci/fixup.c:618:21: warning: right shift count >= width of type [-Wshift-count-overflow] ((((res->end + 1) >> 40) & 0xff) << 16); ^~ vim +608 arch/x86/pci/fixup.c 602 return; 603 604 res = kzalloc(sizeof(*res), GFP_KERNEL); 605 res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | 606 IORESOURCE_WINDOW; 607 res->name = dev->bus->name; > 608 r = allocate_resource(&iomem_resource, res, size, 0x100000000, 609 0xfd00000000, size, NULL, NULL); 610 if (r) { 611 kfree(res); 612 return; 613 } 614 615 base = ((res->start >> 8) & 0xffffff00) | 0x3; 616 limit = ((res->end + 1) >> 8) & 0xffffff00; > 617 high = ((res->start >> 40) & 0xff) | 618 ((((res->end + 1) >> 40) & 0xff) << 16); 619 620 pci_write_config_dword(dev, 0x180 + i * 0x4, high); --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 27265 bytes --] ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-03-14 9:25 ` kbuild test robot 0 siblings, 0 replies; 72+ messages in thread From: kbuild test robot @ 2017-03-14 9:25 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, linux-kernel, amd-gfx, platform-driver-x86, helgaas, kbuild-all [-- Attachment #1: Type: text/plain, Size: 2346 bytes --] Hi Christian, [auto build test WARNING on pci/next] [also build test WARNING on v4.11-rc2 next-20170310] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334 base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next config: i386-randconfig-s0-201711 (attached as .config) compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901 reproduce: # save the attached .config to linux build tree make ARCH=i386 All warnings (new ones prefixed by >>): arch/x86/pci/fixup.c: In function 'pci_amd_enable_64bit_bar': >> arch/x86/pci/fixup.c:608:52: warning: large integer implicitly truncated to unsigned type [-Woverflow] r = allocate_resource(&iomem_resource, res, size, 0x100000000, ^~~~~~~~~~~ arch/x86/pci/fixup.c:609:10: warning: large integer implicitly truncated to unsigned type [-Woverflow] 0xfd00000000, size, NULL, NULL); ^~~~~~~~~~~~ >> arch/x86/pci/fixup.c:617:22: warning: right shift count >= width of type [-Wshift-count-overflow] high = ((res->start >> 40) & 0xff) | ^~ arch/x86/pci/fixup.c:618:21: warning: right shift count >= width of type [-Wshift-count-overflow] ((((res->end + 1) >> 40) & 0xff) << 16); ^~ vim +608 arch/x86/pci/fixup.c 602 return; 603 604 res = kzalloc(sizeof(*res), GFP_KERNEL); 605 res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | 606 IORESOURCE_WINDOW; 607 res->name = dev->bus->name; > 608 r = allocate_resource(&iomem_resource, res, size, 0x100000000, 609 0xfd00000000, size, NULL, NULL); 610 if (r) { 611 kfree(res); 612 return; 613 } 614 615 base = ((res->start >> 8) & 0xffffff00) | 0x3; 616 limit = ((res->end + 1) >> 8) & 0xffffff00; > 617 high = ((res->start >> 40) & 0xff) | 618 ((((res->end + 1) >> 40) & 0xff) << 16); 619 620 pci_write_config_dword(dev, 0x180 + i * 0x4, high); --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 27265 bytes --] [-- Attachment #3: Type: text/plain, Size: 160 bytes --] _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors 2017-03-13 12:41 ` Christian König @ 2017-03-24 15:47 ` Bjorn Helgaas -1 siblings, 0 replies; 72+ messages in thread From: Bjorn Helgaas @ 2017-03-24 15:47 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote: > From: Christian König <christian.koenig@amd.com> > > Most BIOS don't enable this because of compatibility reasons. Can you give any more details here? Without more hints, it's hard to know whether any of the compatibility reasons might apply to Linux as well. > Manually enable a 64bit BAR of 64GB size so that we have > enough room for PCI devices. >From the context, I'm guessing this is enabling a new 64GB window through the PCI host bridge? That might be documented as a "BAR", but it's not anything the Linux PCI core would recognize as a BAR. I think the specs would envision this being done via an ACPI _SRS method on the PNP0A03 host bridge device. That would be a more generic path that would work on any host bridge. Did you explore that possibility? I would prefer to avoid adding device-specific code if that's possible. > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 53 insertions(+) > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > index 6d52b94..bff5242 100644 > --- a/arch/x86/pci/fixup.c > +++ b/arch/x86/pci/fixup.c > @@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar); > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar); > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar); > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar); > + > +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > +{ > + const uint64_t size = 64ULL * 1024 * 1024 * 1024; > + uint32_t base, limit, high; > + struct resource *res; > + unsigned i; > + int r; > + > + for (i = 0; i < 8; ++i) { > + > + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); > + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); > + > + /* Is this slot free? */ > + if ((base & 0x3) == 0x0) > + break; > + > + base >>= 8; > + base |= high << 24; > + > + /* Abort if a slot already configures a 64bit BAR. */ > + if (base > 0x10000) > + return; > + > + } > + > + if (i == 8) > + return; > + > + res = kzalloc(sizeof(*res), GFP_KERNEL); > + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | > + IORESOURCE_WINDOW; > + res->name = dev->bus->name; > + r = allocate_resource(&iomem_resource, res, size, 0x100000000, > + 0xfd00000000, size, NULL, NULL); > + if (r) { > + kfree(res); > + return; > + } > + > + base = ((res->start >> 8) & 0xffffff00) | 0x3; > + limit = ((res->end + 1) >> 8) & 0xffffff00; > + high = ((res->start >> 40) & 0xff) | > + ((((res->end + 1) >> 40) & 0xff) << 16); > + > + pci_write_config_dword(dev, 0x180 + i * 0x4, high); > + pci_write_config_dword(dev, 0x84 + i * 0x8, limit); > + pci_write_config_dword(dev, 0x80 + i * 0x8, base); > + > + pci_bus_add_resource(dev->bus, res, 0); We would need some sort of printk here to explain how this new window magically appeared. > +} > +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); > -- > 2.7.4 > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-03-24 15:47 ` Bjorn Helgaas 0 siblings, 0 replies; 72+ messages in thread From: Bjorn Helgaas @ 2017-03-24 15:47 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote: > From: Christian König <christian.koenig@amd.com> > > Most BIOS don't enable this because of compatibility reasons. Can you give any more details here? Without more hints, it's hard to know whether any of the compatibility reasons might apply to Linux as well. > Manually enable a 64bit BAR of 64GB size so that we have > enough room for PCI devices. From the context, I'm guessing this is enabling a new 64GB window through the PCI host bridge? That might be documented as a "BAR", but it's not anything the Linux PCI core would recognize as a BAR. I think the specs would envision this being done via an ACPI _SRS method on the PNP0A03 host bridge device. That would be a more generic path that would work on any host bridge. Did you explore that possibility? I would prefer to avoid adding device-specific code if that's possible. > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 53 insertions(+) > > diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c > index 6d52b94..bff5242 100644 > --- a/arch/x86/pci/fixup.c > +++ b/arch/x86/pci/fixup.c > @@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar); > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar); > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar); > DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar); > + > +static void pci_amd_enable_64bit_bar(struct pci_dev *dev) > +{ > + const uint64_t size = 64ULL * 1024 * 1024 * 1024; > + uint32_t base, limit, high; > + struct resource *res; > + unsigned i; > + int r; > + > + for (i = 0; i < 8; ++i) { > + > + pci_read_config_dword(dev, 0x80 + i * 0x8, &base); > + pci_read_config_dword(dev, 0x180 + i * 0x4, &high); > + > + /* Is this slot free? */ > + if ((base & 0x3) == 0x0) > + break; > + > + base >>= 8; > + base |= high << 24; > + > + /* Abort if a slot already configures a 64bit BAR. */ > + if (base > 0x10000) > + return; > + > + } > + > + if (i == 8) > + return; > + > + res = kzalloc(sizeof(*res), GFP_KERNEL); > + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 | > + IORESOURCE_WINDOW; > + res->name = dev->bus->name; > + r = allocate_resource(&iomem_resource, res, size, 0x100000000, > + 0xfd00000000, size, NULL, NULL); > + if (r) { > + kfree(res); > + return; > + } > + > + base = ((res->start >> 8) & 0xffffff00) | 0x3; > + limit = ((res->end + 1) >> 8) & 0xffffff00; > + high = ((res->start >> 40) & 0xff) | > + ((((res->end + 1) >> 40) & 0xff) << 16); > + > + pci_write_config_dword(dev, 0x180 + i * 0x4, high); > + pci_write_config_dword(dev, 0x84 + i * 0x8, limit); > + pci_write_config_dword(dev, 0x80 + i * 0x8, base); > + > + pci_bus_add_resource(dev->bus, res, 0); We would need some sort of printk here to explain how this new window magically appeared. > +} > +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar); > -- > 2.7.4 > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-04-11 15:48 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-04-11 15:48 UTC (permalink / raw) To: Bjorn Helgaas Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Am 24.03.2017 um 16:47 schrieb Bjorn Helgaas: > On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote: >> From: Christian König <christian.koenig@amd.com> >> >> Most BIOS don't enable this because of compatibility reasons. > Can you give any more details here? Without more hints, it's hard to > know whether any of the compatibility reasons might apply to Linux as > well. Unfortunately not, I could try to ask a few more people at AMD if they know the background. I was told that there are a few boards which offers that as a BIOS option, but so far I haven't found any (and I have quite a few here). My best guess is that older windows versions have a problem with that. >> Manually enable a 64bit BAR of 64GB size so that we have >> enough room for PCI devices. > From the context, I'm guessing this is enabling a new 64GB window > through the PCI host bridge? Yes, exactly. Sorry for the confusion. > That might be documented as a "BAR", but > it's not anything the Linux PCI core would recognize as a BAR. At least the AMD NB documentation calls this the host BARs. But I'm perfectly fine with any terminology. How about calling it host bridge window instead? > I think the specs would envision this being done via an ACPI _SRS > method on the PNP0A03 host bridge device. That would be a more > generic path that would work on any host bridge. Did you explore that > possibility? I would prefer to avoid adding device-specific code if > that's possible. I've checked quite a few boards, but none of them actually implements it this way. M$ is working on a new ACPI table to enable this vendor neutral, but I guess that will still take a while. I want to support this for all AMD CPU released in the past 5 years or so, so we are going to deal with a bunch of older boards as well. >> + pci_bus_add_resource(dev->bus, res, 0); > We would need some sort of printk here to explain how this new window > magically appeared. Good point, consider this done. But is this actually the right place of doing it? Or would you prefer something to be added to the probing code? I think those fixups are applied a bit later, aren't they? Best regards, Christian. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2017-04-11 15:48 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-04-11 15:48 UTC (permalink / raw) To: Bjorn Helgaas Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA Am 24.03.2017 um 16:47 schrieb Bjorn Helgaas: > On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote: >> From: Christian König <christian.koenig@amd.com> >> >> Most BIOS don't enable this because of compatibility reasons. > Can you give any more details here? Without more hints, it's hard to > know whether any of the compatibility reasons might apply to Linux as > well. Unfortunately not, I could try to ask a few more people at AMD if they know the background. I was told that there are a few boards which offers that as a BIOS option, but so far I haven't found any (and I have quite a few here). My best guess is that older windows versions have a problem with that. >> Manually enable a 64bit BAR of 64GB size so that we have >> enough room for PCI devices. > From the context, I'm guessing this is enabling a new 64GB window > through the PCI host bridge? Yes, exactly. Sorry for the confusion. > That might be documented as a "BAR", but > it's not anything the Linux PCI core would recognize as a BAR. At least the AMD NB documentation calls this the host BARs. But I'm perfectly fine with any terminology. How about calling it host bridge window instead? > I think the specs would envision this being done via an ACPI _SRS > method on the PNP0A03 host bridge device. That would be a more > generic path that would work on any host bridge. Did you explore that > possibility? I would prefer to avoid adding device-specific code if > that's possible. I've checked quite a few boards, but none of them actually implements it this way. M$ is working on a new ACPI table to enable this vendor neutral, but I guess that will still take a while. I want to support this for all AMD CPU released in the past 5 years or so, so we are going to deal with a bunch of older boards as well. >> + pci_bus_add_resource(dev->bus, res, 0); > We would need some sort of printk here to explain how this new window > magically appeared. Good point, consider this done. But is this actually the right place of doing it? Or would you prefer something to be added to the probing code? I think those fixups are applied a bit later, aren't they? Best regards, Christian. _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors 2017-04-11 15:48 ` Christian König (?) @ 2017-04-12 16:55 ` Bjorn Helgaas 2017-04-25 13:01 ` Christian König -1 siblings, 1 reply; 72+ messages in thread From: Bjorn Helgaas @ 2017-04-12 16:55 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Tue, Apr 11, 2017 at 05:48:25PM +0200, Christian König wrote: > Am 24.03.2017 um 16:47 schrieb Bjorn Helgaas: > >On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote: > >>From: Christian König <christian.koenig@amd.com> > >> > >>Most BIOS don't enable this because of compatibility reasons. > >Can you give any more details here? Without more hints, it's hard to > >know whether any of the compatibility reasons might apply to Linux as > >well. > > Unfortunately not, I could try to ask a few more people at AMD if > they know the background. > > I was told that there are a few boards which offers that as a BIOS > option, but so far I haven't found any (and I have quite a few > here). > > My best guess is that older windows versions have a problem with that. > > >>Manually enable a 64bit BAR of 64GB size so that we have > >>enough room for PCI devices. > > From the context, I'm guessing this is enabling a new 64GB window > >through the PCI host bridge? > > Yes, exactly. Sorry for the confusion. > > >That might be documented as a "BAR", but > >it's not anything the Linux PCI core would recognize as a BAR. > > At least the AMD NB documentation calls this the host BARs. But I'm > perfectly fine with any terminology. > > How about calling it host bridge window instead? That works for me. > >I think the specs would envision this being done via an ACPI _SRS > >method on the PNP0A03 host bridge device. That would be a more > >generic path that would work on any host bridge. Did you explore that > >possibility? I would prefer to avoid adding device-specific code if > >that's possible. > > I've checked quite a few boards, but none of them actually > implements it this way. > > M$ is working on a new ACPI table to enable this vendor neutral, but > I guess that will still take a while. > > I want to support this for all AMD CPU released in the past 5 years > or so, so we are going to deal with a bunch of older boards as well. I've never seen _SRS for host bridges either. I'm curious about what sort of new table will be proposed. It seems like the existing ACPI resource framework could manage it, but I certainly don't know all the issues. > >>+ pci_bus_add_resource(dev->bus, res, 0); > >We would need some sort of printk here to explain how this new window > >magically appeared. > > Good point, consider this done. > > But is this actually the right place of doing it? Or would you > prefer something to be added to the probing code? > > I think those fixups are applied a bit later, aren't they? Logically, this should be done before we enumerate the PCI devices below the host bridge, so a PCI device fixup is not the ideal place for it, but it might be the most practical. I could imagine some sort of quirk like the ones in drivers/pnp/quirks.c that could add the window to the host bridge _CRS and program the bridge to open it. But the PCI host bridges aren't handled through the path that applies those fixups, and it would be messy to identify your bridges (you currently use PCI vendor/device IDs, which are only available after enumerating the device). So this doesn't seem like a viable strategy. Bjorn ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors 2017-04-12 16:55 ` Bjorn Helgaas @ 2017-04-25 13:01 ` Christian König 2017-05-17 21:36 ` Bjorn Helgaas 0 siblings, 1 reply; 72+ messages in thread From: Christian König @ 2017-04-25 13:01 UTC (permalink / raw) To: Bjorn Helgaas Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Am 12.04.2017 um 18:55 schrieb Bjorn Helgaas: > [SNIP] >>> I think the specs would envision this being done via an ACPI _SRS >>> method on the PNP0A03 host bridge device. That would be a more >>> generic path that would work on any host bridge. Did you explore that >>> possibility? I would prefer to avoid adding device-specific code if >>> that's possible. >> I've checked quite a few boards, but none of them actually >> implements it this way. >> >> M$ is working on a new ACPI table to enable this vendor neutral, but >> I guess that will still take a while. >> >> I want to support this for all AMD CPU released in the past 5 years >> or so, so we are going to deal with a bunch of older boards as well. > I've never seen _SRS for host bridges either. I'm curious about what > sort of new table will be proposed. It seems like the existing ACPI > resource framework could manage it, but I certainly don't know all the > issues. No idea either since I'm not involved into that. My job is to get it working on the existing hw generations and that alone is enough work :) My best guess is that MS is going to either make _SRS on the host bridge or a pre-configured 64bit window mandatory for the BIOS. >>>> + pci_bus_add_resource(dev->bus, res, 0); >>> We would need some sort of printk here to explain how this new window >>> magically appeared. >> Good point, consider this done. >> >> But is this actually the right place of doing it? Or would you >> prefer something to be added to the probing code? >> >> I think those fixups are applied a bit later, aren't they? > Logically, this should be done before we enumerate the PCI devices > below the host bridge, so a PCI device fixup is not the ideal place > for it, but it might be the most practical. Since the modification must be done on a device connected to the root bus I run into quite a chicken and egg problem if I try to do it before the enumeration. > I could imagine some sort of quirk like the ones in > drivers/pnp/quirks.c that could add the window to the host bridge _CRS > and program the bridge to open it. But the PCI host bridges aren't > handled through the path that applies those fixups, and it would be > messy to identify your bridges (you currently use PCI vendor/device > IDs, which are only available after enumerating the device). So this > doesn't seem like a viable strategy. I've tried that, but gave up rather quickly. Looks like the current approach indeed work find even with "pci=realloc", so I'm going to stick with that. Regards, Christian. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors 2017-04-25 13:01 ` Christian König @ 2017-05-17 21:36 ` Bjorn Helgaas 0 siblings, 0 replies; 72+ messages in thread From: Bjorn Helgaas @ 2017-05-17 21:36 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Tue, Apr 25, 2017 at 03:01:35PM +0200, Christian König wrote: > Am 12.04.2017 um 18:55 schrieb Bjorn Helgaas: > >[SNIP] > >>>I think the specs would envision this being done via an ACPI _SRS > >>>method on the PNP0A03 host bridge device. That would be a more > >>>generic path that would work on any host bridge. Did you explore that > >>>possibility? I would prefer to avoid adding device-specific code if > >>>that's possible. > >>I've checked quite a few boards, but none of them actually > >>implements it this way. > >> > >>M$ is working on a new ACPI table to enable this vendor neutral, but > >>I guess that will still take a while. > >> > >>I want to support this for all AMD CPU released in the past 5 years > >>or so, so we are going to deal with a bunch of older boards as well. > >I've never seen _SRS for host bridges either. I'm curious about what > >sort of new table will be proposed. It seems like the existing ACPI > >resource framework could manage it, but I certainly don't know all the > >issues. > > No idea either since I'm not involved into that. My job is to get it > working on the existing hw generations and that alone is enough work > :) > > My best guess is that MS is going to either make _SRS on the host > bridge or a pre-configured 64bit window mandatory for the BIOS. While researching something else, I noticed that the PCI Firmware Spec, rev 3.2, does indeed call out _PRS and _SRS as the mechanism for the OS to configure host bridge windows. See sections 4.1.3, 4.3.2.1, and 4.6.5. Sec 4.6.5 also includes an implementation note that might be a clue about the "compatibility issues" that prevent the BIOS from enabling the window in the first place. I'd like to incorporate some of this info into these changes, probably in a code comment and changelog, so we can encourage a more generic approach in the future, even if we can't use it in all existing cases. Bjorn ^ permalink raw reply [flat|nested] 72+ messages in thread
* [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-13 12:41 ` Christian König @ 2017-03-13 12:41 ` Christian König -1 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel From: Christian König <christian.koenig@amd.com> Try to resize BAR0 to let CPU access all of VRAM. Signed-off-by: Christian König <christian.koenig@amd.com> --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- 4 files changed, 40 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 3b81ded..905ded9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, struct ttm_mem_reg *mem); void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); +void amdgpu_resize_bar0(struct amdgpu_device *adev); void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); int amdgpu_ttm_init(struct amdgpu_device *adev); void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 118f4e6..92955fe 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); } +/** + * amdgpu_resize_bar0 - try to resize BAR0 + * + * @adev: amdgpu_device pointer + * + * Try to resize BAR0 to make all VRAM CPU accessible. + */ +void amdgpu_resize_bar0(struct amdgpu_device *adev) +{ + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; + int r; + + r = pci_resize_resource(adev->pdev, 0, size); + + if (r == -ENOTSUPP) { + /* The hardware don't support the extension. */ + return; + + } else if (r == -ENOSPC) { + DRM_INFO("Not enoigh PCI address space for a large BAR."); + } else if (r) { + DRM_ERROR("Problem resizing BAR0 (%d).", r); + } + + /* Reinit the doorbell mapping, it is most likely moved as well */ + amdgpu_doorbell_fini(adev); + BUG_ON(amdgpu_doorbell_init(adev)); +} + /* * GPU helpers function. */ diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index dc9b6d6..36a7aa5 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) break; } adev->mc.vram_width = numchan * chansize; - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + if (!(adev->flags & AMD_IS_APU)) + amdgpu_resize_bar0(adev); + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); + #ifdef CONFIG_X86_64 if (adev->flags & AMD_IS_APU) { adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index c087b00..7761ad3 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) break; } adev->mc.vram_width = numchan * chansize; - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + if (!(adev->flags & AMD_IS_APU)) + amdgpu_resize_bar0(adev); + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); + #ifdef CONFIG_X86_64 if (adev->flags & AMD_IS_APU) { adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; -- 2.7.4 ^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-13 12:41 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw) To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel From: Christian König <christian.koenig@amd.com> Try to resize BAR0 to let CPU access all of VRAM. Signed-off-by: Christian König <christian.koenig@amd.com> --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- 4 files changed, 40 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 3b81ded..905ded9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, struct ttm_mem_reg *mem); void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); +void amdgpu_resize_bar0(struct amdgpu_device *adev); void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); int amdgpu_ttm_init(struct amdgpu_device *adev); void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 118f4e6..92955fe 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); } +/** + * amdgpu_resize_bar0 - try to resize BAR0 + * + * @adev: amdgpu_device pointer + * + * Try to resize BAR0 to make all VRAM CPU accessible. + */ +void amdgpu_resize_bar0(struct amdgpu_device *adev) +{ + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; + int r; + + r = pci_resize_resource(adev->pdev, 0, size); + + if (r == -ENOTSUPP) { + /* The hardware don't support the extension. */ + return; + + } else if (r == -ENOSPC) { + DRM_INFO("Not enoigh PCI address space for a large BAR."); + } else if (r) { + DRM_ERROR("Problem resizing BAR0 (%d).", r); + } + + /* Reinit the doorbell mapping, it is most likely moved as well */ + amdgpu_doorbell_fini(adev); + BUG_ON(amdgpu_doorbell_init(adev)); +} + /* * GPU helpers function. */ diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index dc9b6d6..36a7aa5 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) break; } adev->mc.vram_width = numchan * chansize; - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + if (!(adev->flags & AMD_IS_APU)) + amdgpu_resize_bar0(adev); + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); + #ifdef CONFIG_X86_64 if (adev->flags & AMD_IS_APU) { adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index c087b00..7761ad3 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) break; } adev->mc.vram_width = numchan * chansize; - /* Could aper size report 0 ? */ - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); /* size in MB on si */ adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; + if (!(adev->flags & AMD_IS_APU)) + amdgpu_resize_bar0(adev); + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); + #ifdef CONFIG_X86_64 if (adev->flags & AMD_IS_APU) { adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; -- 2.7.4 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply related [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-13 12:41 ` Christian König @ 2017-03-13 16:51 ` Andy Shevchenko -1 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:51 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 2:41 PM, Christian König <deathsimple@vodafone.de> wrote: > From: Christian König <christian.koenig@amd.com> > > Try to resize BAR0 to let CPU access all of VRAM. > +void amdgpu_resize_bar0(struct amdgpu_device *adev) > +{ > + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > + int r; > + > + r = pci_resize_resource(adev->pdev, 0, size); > + Redundant. > + if (r == -ENOTSUPP) { > + /* The hardware don't support the extension. */ > + return; > + Ditto. > + } else if (r == -ENOSPC) { Useless use of else. And thus of curly braces. > + DRM_INFO("Not enoigh PCI address space for a large BAR."); > + } else if (r) { > + DRM_ERROR("Problem resizing BAR0 (%d).", r); > + } > + > + /* Reinit the doorbell mapping, it is most likely moved as well */ > + amdgpu_doorbell_fini(adev); > + BUG_ON(amdgpu_doorbell_init(adev)); Comment why it's used here. -- With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-13 16:51 ` Andy Shevchenko 0 siblings, 0 replies; 72+ messages in thread From: Andy Shevchenko @ 2017-03-13 16:51 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 2:41 PM, Christian K=C3=B6nig <deathsimple@vodafone.de> wrote: > From: Christian K=C3=B6nig <christian.koenig@amd.com> > > Try to resize BAR0 to let CPU access all of VRAM. > +void amdgpu_resize_bar0(struct amdgpu_device *adev) > +{ > + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20= ; > + int r; > + > + r =3D pci_resize_resource(adev->pdev, 0, size); > + Redundant. > + if (r =3D=3D -ENOTSUPP) { > + /* The hardware don't support the extension. */ > + return; > + Ditto. > + } else if (r =3D=3D -ENOSPC) { Useless use of else. And thus of curly braces. > + DRM_INFO("Not enoigh PCI address space for a large BAR.")= ; > + } else if (r) { > + DRM_ERROR("Problem resizing BAR0 (%d).", r); > + } > + > + /* Reinit the doorbell mapping, it is most likely moved as well *= / > + amdgpu_doorbell_fini(adev); > + BUG_ON(amdgpu_doorbell_init(adev)); Comment why it's used here. --=20 With Best Regards, Andy Shevchenko ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-13 12:41 ` Christian König (?) @ 2017-03-15 7:23 ` Ayyappa Ch -1 siblings, 0 replies; 72+ messages in thread From: Ayyappa Ch @ 2017-03-15 7:23 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Is it possible on Carrizo asics? Or only supports on newer asics? On Mon, Mar 13, 2017 at 6:11 PM, Christian König <deathsimple@vodafone.de> wrote: > From: Christian König <christian.koenig@amd.com> > > Try to resize BAR0 to let CPU access all of VRAM. > > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ > drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > 4 files changed, 40 insertions(+), 6 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > index 3b81ded..905ded9 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, > struct ttm_mem_reg *mem); > void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); > void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); > +void amdgpu_resize_bar0(struct amdgpu_device *adev); > void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); > int amdgpu_ttm_init(struct amdgpu_device *adev); > void amdgpu_ttm_fini(struct amdgpu_device *adev); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > index 118f4e6..92955fe 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) > mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > } > > +/** > + * amdgpu_resize_bar0 - try to resize BAR0 > + * > + * @adev: amdgpu_device pointer > + * > + * Try to resize BAR0 to make all VRAM CPU accessible. > + */ > +void amdgpu_resize_bar0(struct amdgpu_device *adev) > +{ > + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > + int r; > + > + r = pci_resize_resource(adev->pdev, 0, size); > + > + if (r == -ENOTSUPP) { > + /* The hardware don't support the extension. */ > + return; > + > + } else if (r == -ENOSPC) { > + DRM_INFO("Not enoigh PCI address space for a large BAR."); > + } else if (r) { > + DRM_ERROR("Problem resizing BAR0 (%d).", r); > + } > + > + /* Reinit the doorbell mapping, it is most likely moved as well */ > + amdgpu_doorbell_fini(adev); > + BUG_ON(amdgpu_doorbell_init(adev)); > +} > + > /* > * GPU helpers function. > */ > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > index dc9b6d6..36a7aa5 100644 > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) > break; > } > adev->mc.vram_width = numchan * chansize; > - /* Could aper size report 0 ? */ > - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > /* size in MB on si */ > adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > > + if (!(adev->flags & AMD_IS_APU)) > + amdgpu_resize_bar0(adev); > + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > + > #ifdef CONFIG_X86_64 > if (adev->flags & AMD_IS_APU) { > adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > index c087b00..7761ad3 100644 > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) > break; > } > adev->mc.vram_width = numchan * chansize; > - /* Could aper size report 0 ? */ > - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > /* size in MB on si */ > adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > > + if (!(adev->flags & AMD_IS_APU)) > + amdgpu_resize_bar0(adev); > + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > + > #ifdef CONFIG_X86_64 > if (adev->flags & AMD_IS_APU) { > adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > -- > 2.7.4 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 7:23 ` Ayyappa Ch 0 siblings, 0 replies; 72+ messages in thread From: Ayyappa Ch @ 2017-03-15 7:23 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Is it possible on Carrizo asics? Or only supports on newer asics? On Mon, Mar 13, 2017 at 6:11 PM, Christian K=C3=B6nig <deathsimple@vodafone.de> wrote: > From: Christian K=C3=B6nig <christian.koenig@amd.com> > > Try to resize BAR0 to let CPU access all of VRAM. > > Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 ++++++++++++++++++++++++= +++++ > drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > 4 files changed, 40 insertions(+), 6 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/am= dgpu/amdgpu.h > index 3b81ded..905ded9 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_devi= ce *adev, struct ttm_tt *ttm, > struct ttm_mem_reg *mem); > void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *= mc, u64 base); > void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *m= c); > +void amdgpu_resize_bar0(struct amdgpu_device *adev); > void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 siz= e); > int amdgpu_ttm_init(struct amdgpu_device *adev); > void amdgpu_ttm_fini(struct amdgpu_device *adev); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_device.c > index 118f4e6..92955fe 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev,= struct amdgpu_mc *mc) > mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > } > > +/** > + * amdgpu_resize_bar0 - try to resize BAR0 > + * > + * @adev: amdgpu_device pointer > + * > + * Try to resize BAR0 to make all VRAM CPU accessible. > + */ > +void amdgpu_resize_bar0(struct amdgpu_device *adev) > +{ > + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20= ; > + int r; > + > + r =3D pci_resize_resource(adev->pdev, 0, size); > + > + if (r =3D=3D -ENOTSUPP) { > + /* The hardware don't support the extension. */ > + return; > + > + } else if (r =3D=3D -ENOSPC) { > + DRM_INFO("Not enoigh PCI address space for a large BAR.")= ; > + } else if (r) { > + DRM_ERROR("Problem resizing BAR0 (%d).", r); > + } > + > + /* Reinit the doorbell mapping, it is most likely moved as well *= / > + amdgpu_doorbell_fini(adev); > + BUG_ON(amdgpu_doorbell_init(adev)); > +} > + > /* > * GPU helpers function. > */ > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/= amdgpu/gmc_v7_0.c > index dc9b6d6..36a7aa5 100644 > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *a= dev) > break; > } > adev->mc.vram_width =3D numchan * chansize; > - /* Could aper size report 0 ? */ > - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); > - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); > /* size in MB on si */ > adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 10= 24ULL; > adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * = 1024ULL; > > + if (!(adev->flags & AMD_IS_APU)) > + amdgpu_resize_bar0(adev); > + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); > + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); > + > #ifdef CONFIG_X86_64 > if (adev->flags & AMD_IS_APU) { > adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)) <= < 22; > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/= amdgpu/gmc_v8_0.c > index c087b00..7761ad3 100644 > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *a= dev) > break; > } > adev->mc.vram_width =3D numchan * chansize; > - /* Could aper size report 0 ? */ > - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); > - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); > /* size in MB on si */ > adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 10= 24ULL; > adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * = 1024ULL; > > + if (!(adev->flags & AMD_IS_APU)) > + amdgpu_resize_bar0(adev); > + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); > + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); > + > #ifdef CONFIG_X86_64 > if (adev->flags & AMD_IS_APU) { > adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)) <= < 22; > -- > 2.7.4 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 7:23 ` Ayyappa Ch 0 siblings, 0 replies; 72+ messages in thread From: Ayyappa Ch @ 2017-03-15 7:23 UTC (permalink / raw) To: Christian König Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel Is it possible on Carrizo asics? Or only supports on newer asics? On Mon, Mar 13, 2017 at 6:11 PM, Christian König <deathsimple@vodafone.de> wrote: > From: Christian König <christian.koenig@amd.com> > > Try to resize BAR0 to let CPU access all of VRAM. > > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ > drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > 4 files changed, 40 insertions(+), 6 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > index 3b81ded..905ded9 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, > struct ttm_mem_reg *mem); > void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); > void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); > +void amdgpu_resize_bar0(struct amdgpu_device *adev); > void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); > int amdgpu_ttm_init(struct amdgpu_device *adev); > void amdgpu_ttm_fini(struct amdgpu_device *adev); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > index 118f4e6..92955fe 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) > mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > } > > +/** > + * amdgpu_resize_bar0 - try to resize BAR0 > + * > + * @adev: amdgpu_device pointer > + * > + * Try to resize BAR0 to make all VRAM CPU accessible. > + */ > +void amdgpu_resize_bar0(struct amdgpu_device *adev) > +{ > + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > + int r; > + > + r = pci_resize_resource(adev->pdev, 0, size); > + > + if (r == -ENOTSUPP) { > + /* The hardware don't support the extension. */ > + return; > + > + } else if (r == -ENOSPC) { > + DRM_INFO("Not enoigh PCI address space for a large BAR."); > + } else if (r) { > + DRM_ERROR("Problem resizing BAR0 (%d).", r); > + } > + > + /* Reinit the doorbell mapping, it is most likely moved as well */ > + amdgpu_doorbell_fini(adev); > + BUG_ON(amdgpu_doorbell_init(adev)); > +} > + > /* > * GPU helpers function. > */ > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > index dc9b6d6..36a7aa5 100644 > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) > break; > } > adev->mc.vram_width = numchan * chansize; > - /* Could aper size report 0 ? */ > - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > /* size in MB on si */ > adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > > + if (!(adev->flags & AMD_IS_APU)) > + amdgpu_resize_bar0(adev); > + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > + > #ifdef CONFIG_X86_64 > if (adev->flags & AMD_IS_APU) { > adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > index c087b00..7761ad3 100644 > --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) > break; > } > adev->mc.vram_width = numchan * chansize; > - /* Could aper size report 0 ? */ > - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > /* size in MB on si */ > adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; > > + if (!(adev->flags & AMD_IS_APU)) > + amdgpu_resize_bar0(adev); > + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > + > #ifdef CONFIG_X86_64 > if (adev->flags & AMD_IS_APU) { > adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > -- > 2.7.4 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-15 7:23 ` Ayyappa Ch @ 2017-03-15 7:37 ` Christian König -1 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 7:37 UTC (permalink / raw) To: Ayyappa Ch Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel Carizzo is an APU and resizing BARs isn't needed nor supported there. The CPU can access the full stolen VRAM directly on that hardware. As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. Christian. Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > Is it possible on Carrizo asics? Or only supports on newer asics? > > On Mon, Mar 13, 2017 at 6:11 PM, Christian König > <deathsimple@vodafone.de> wrote: >> From: Christian König <christian.koenig@amd.com> >> >> Try to resize BAR0 to let CPU access all of VRAM. >> >> Signed-off-by: Christian König <christian.koenig@amd.com> >> --- >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> 4 files changed, 40 insertions(+), 6 deletions(-) >> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> index 3b81ded..905ded9 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >> struct ttm_mem_reg *mem); >> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); >> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >> int amdgpu_ttm_init(struct amdgpu_device *adev); >> void amdgpu_ttm_fini(struct amdgpu_device *adev); >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> index 118f4e6..92955fe 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >> } >> >> +/** >> + * amdgpu_resize_bar0 - try to resize BAR0 >> + * >> + * @adev: amdgpu_device pointer >> + * >> + * Try to resize BAR0 to make all VRAM CPU accessible. >> + */ >> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >> +{ >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >> + int r; >> + >> + r = pci_resize_resource(adev->pdev, 0, size); >> + >> + if (r == -ENOTSUPP) { >> + /* The hardware don't support the extension. */ >> + return; >> + >> + } else if (r == -ENOSPC) { >> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >> + } else if (r) { >> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> + } >> + >> + /* Reinit the doorbell mapping, it is most likely moved as well */ >> + amdgpu_doorbell_fini(adev); >> + BUG_ON(amdgpu_doorbell_init(adev)); >> +} >> + >> /* >> * GPU helpers function. >> */ >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> index dc9b6d6..36a7aa5 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> index c087b00..7761ad3 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> -- >> 2.7.4 >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 7:37 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 7:37 UTC (permalink / raw) To: Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel Carizzo is an APU and resizing BARs isn't needed nor supported there. The CPU can access the full stolen VRAM directly on that hardware. As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. Christian. Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > Is it possible on Carrizo asics? Or only supports on newer asics? > > On Mon, Mar 13, 2017 at 6:11 PM, Christian König > <deathsimple@vodafone.de> wrote: >> From: Christian König <christian.koenig@amd.com> >> >> Try to resize BAR0 to let CPU access all of VRAM. >> >> Signed-off-by: Christian König <christian.koenig@amd.com> >> --- >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> 4 files changed, 40 insertions(+), 6 deletions(-) >> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> index 3b81ded..905ded9 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >> struct ttm_mem_reg *mem); >> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); >> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >> int amdgpu_ttm_init(struct amdgpu_device *adev); >> void amdgpu_ttm_fini(struct amdgpu_device *adev); >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> index 118f4e6..92955fe 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >> } >> >> +/** >> + * amdgpu_resize_bar0 - try to resize BAR0 >> + * >> + * @adev: amdgpu_device pointer >> + * >> + * Try to resize BAR0 to make all VRAM CPU accessible. >> + */ >> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >> +{ >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >> + int r; >> + >> + r = pci_resize_resource(adev->pdev, 0, size); >> + >> + if (r == -ENOTSUPP) { >> + /* The hardware don't support the extension. */ >> + return; >> + >> + } else if (r == -ENOSPC) { >> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >> + } else if (r) { >> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> + } >> + >> + /* Reinit the doorbell mapping, it is most likely moved as well */ >> + amdgpu_doorbell_fini(adev); >> + BUG_ON(amdgpu_doorbell_init(adev)); >> +} >> + >> /* >> * GPU helpers function. >> */ >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> index dc9b6d6..36a7aa5 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> index c087b00..7761ad3 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> -- >> 2.7.4 >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 8:25 ` Zhou, David(ChunMing) 0 siblings, 0 replies; 72+ messages in thread From: Zhou, David(ChunMing) @ 2017-03-15 8:25 UTC (permalink / raw) To: Christian König, Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel Does that means we don't need invisible vram later? David -----Original Message----- From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig Sent: Wednesday, March 15, 2017 3:38 PM To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access Carizzo is an APU and resizing BARs isn't needed nor supported there. The CPU can access the full stolen VRAM directly on that hardware. As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. Christian. Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > Is it possible on Carrizo asics? Or only supports on newer asics? > > On Mon, Mar 13, 2017 at 6:11 PM, Christian König > <deathsimple@vodafone.de> wrote: >> From: Christian König <christian.koenig@amd.com> >> >> Try to resize BAR0 to let CPU access all of VRAM. >> >> Signed-off-by: Christian König <christian.koenig@amd.com> >> --- >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> 4 files changed, 40 insertions(+), 6 deletions(-) >> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> index 3b81ded..905ded9 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >> struct ttm_mem_reg *mem); >> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >> amdgpu_mc *mc); >> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >> int amdgpu_ttm_init(struct amdgpu_device *adev); >> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> index 118f4e6..92955fe 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >> } >> >> +/** >> + * amdgpu_resize_bar0 - try to resize BAR0 >> + * >> + * @adev: amdgpu_device pointer >> + * >> + * Try to resize BAR0 to make all VRAM CPU accessible. >> + */ >> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >> + int r; >> + >> + r = pci_resize_resource(adev->pdev, 0, size); >> + >> + if (r == -ENOTSUPP) { >> + /* The hardware don't support the extension. */ >> + return; >> + >> + } else if (r == -ENOSPC) { >> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >> + } else if (r) { >> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> + } >> + >> + /* Reinit the doorbell mapping, it is most likely moved as well */ >> + amdgpu_doorbell_fini(adev); >> + BUG_ON(amdgpu_doorbell_init(adev)); >> +} >> + >> /* >> * GPU helpers function. >> */ >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> index dc9b6d6..36a7aa5 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >> * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = >> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> index c087b00..7761ad3 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >> * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = >> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> -- >> 2.7.4 >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 8:25 ` Zhou, David(ChunMing) 0 siblings, 0 replies; 72+ messages in thread From: Zhou, David(ChunMing) @ 2017-03-15 8:25 UTC (permalink / raw) To: Christian König, Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel RG9lcyB0aGF0IG1lYW5zIHdlIGRvbid0IG5lZWQgaW52aXNpYmxlIHZyYW0gbGF0ZXI/DQoNCkRh dmlkDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBkcmktZGV2ZWwgW21haWx0 bzpkcmktZGV2ZWwtYm91bmNlc0BsaXN0cy5mcmVlZGVza3RvcC5vcmddIE9uIEJlaGFsZiBPZiBD aHJpc3RpYW4gSz9uaWcNClNlbnQ6IFdlZG5lc2RheSwgTWFyY2ggMTUsIDIwMTcgMzozOCBQTQ0K VG86IEF5eWFwcGEgQ2ggPGF5eWFwcGEuY2gubGludXhAZ21haWwuY29tPg0KQ2M6IGxpbnV4LXBj aUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7IGFtZC1nZnhA bGlzdHMuZnJlZWRlc2t0b3Aub3JnOyBwbGF0Zm9ybS1kcml2ZXIteDg2QHZnZXIua2VybmVsLm9y ZzsgaGVsZ2Fhc0BrZXJuZWwub3JnOyBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQpT dWJqZWN0OiBSZTogW1BBVENIIDQvNF0gZHJtL2FtZGdwdTogcmVzaXplIFZSQU0gQkFSIGZvciBD UFUgYWNjZXNzDQoNCkNhcml6em8gaXMgYW4gQVBVIGFuZCByZXNpemluZyBCQVJzIGlzbid0IG5l ZWRlZCBub3Igc3VwcG9ydGVkIHRoZXJlLiANClRoZSBDUFUgY2FuIGFjY2VzcyB0aGUgZnVsbCBz dG9sZW4gVlJBTSBkaXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KDQpBcyBmYXIgYXMgSSBrbm93 IEFTSUNzIHdpdGggc3VwcG9ydCBmb3IgdGhpcyBhcmUgVG9uZ2EsIEZpamkgYW5kIGFsbCBQb2xh cmlzIHZhcmlhbnRzLg0KDQpDaHJpc3RpYW4uDQoNCkFtIDE1LjAzLjIwMTcgdW0gMDg6MjMgc2No cmllYiBBeXlhcHBhIENoOg0KPiBJcyBpdCBwb3NzaWJsZSBvbiBDYXJyaXpvIGFzaWNzPyBPciBv bmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNzPw0KPg0KPiBPbiBNb24sIE1hciAxMywgMjAxNyBh dCA2OjExIFBNLCBDaHJpc3RpYW4gS8O2bmlnIA0KPiA8ZGVhdGhzaW1wbGVAdm9kYWZvbmUuZGU+ IHdyb3RlOg0KPj4gRnJvbTogQ2hyaXN0aWFuIEvDtm5pZyA8Y2hyaXN0aWFuLmtvZW5pZ0BhbWQu Y29tPg0KPj4NCj4+IFRyeSB0byByZXNpemUgQkFSMCB0byBsZXQgQ1BVIGFjY2VzcyBhbGwgb2Yg VlJBTS4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4u a29lbmlnQGFtZC5jb20+DQo+PiAtLS0NCj4+ICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv YW1kZ3B1LmggICAgICAgIHwgIDEgKw0KPj4gICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9h bWRncHVfZGV2aWNlLmMgfCAyOSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPj4gICBk cml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjdfMC5jICAgICAgfCAgOCArKysrKy0tLQ0K Pj4gICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jICAgICAgfCAgOCArKysr Ky0tLQ0KPj4gICA0IGZpbGVzIGNoYW5nZWQsIDQwIGluc2VydGlvbnMoKyksIDYgZGVsZXRpb25z KC0pDQo+Pg0KPj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdw dS5oIA0KPj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPj4gaW5kZXgg M2I4MWRlZC4uOTA1ZGVkOSAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k Z3B1L2FtZGdwdS5oDQo+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUu aA0KPj4gQEAgLTE3MTksNiArMTcxOSw3IEBAIHVpbnQ2NF90IGFtZGdwdV90dG1fdHRfcHRlX2Zs YWdzKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LCBzdHJ1Y3QgdHRtX3R0ICp0dG0sDQo+PiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHR0bV9tZW1fcmVnICptZW0p Ow0KPj4gICB2b2lkIGFtZGdwdV92cmFtX2xvY2F0aW9uKHN0cnVjdCBhbWRncHVfZGV2aWNlICph ZGV2LCBzdHJ1Y3QgYW1kZ3B1X21jICptYywgdTY0IGJhc2UpOw0KPj4gICB2b2lkIGFtZGdwdV9n dHRfbG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdCANCj4+IGFtZGdw dV9tYyAqbWMpOw0KPj4gK3ZvaWQgYW1kZ3B1X3Jlc2l6ZV9iYXIwKHN0cnVjdCBhbWRncHVfZGV2 aWNlICphZGV2KTsNCj4+ICAgdm9pZCBhbWRncHVfdHRtX3NldF9hY3RpdmVfdnJhbV9zaXplKHN0 cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LCB1NjQgc2l6ZSk7DQo+PiAgIGludCBhbWRncHVfdHRt X2luaXQoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYpOw0KPj4gICB2b2lkIGFtZGdwdV90dG1f ZmluaShzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldik7IGRpZmYgLS1naXQgDQo+PiBhL2RyaXZl cnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9kZXZpY2UuYyANCj4+IGIvZHJpdmVycy9ncHUv ZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+PiBpbmRleCAxMThmNGU2Li45Mjk1NWZl IDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2Rldmlj ZS5jDQo+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMN Cj4+IEBAIC02OTIsNiArNjkyLDM1IEBAIHZvaWQgYW1kZ3B1X2d0dF9sb2NhdGlvbihzdHJ1Y3Qg YW1kZ3B1X2RldmljZSAqYWRldiwgc3RydWN0IGFtZGdwdV9tYyAqbWMpDQo+PiAgICAgICAgICAg ICAgICAgICAgICAgICAgbWMtPmd0dF9zaXplID4+IDIwLCBtYy0+Z3R0X3N0YXJ0LCBtYy0+Z3R0 X2VuZCk7DQo+PiAgIH0NCj4+DQo+PiArLyoqDQo+PiArICogYW1kZ3B1X3Jlc2l6ZV9iYXIwIC0g dHJ5IHRvIHJlc2l6ZSBCQVIwDQo+PiArICoNCj4+ICsgKiBAYWRldjogYW1kZ3B1X2RldmljZSBw b2ludGVyDQo+PiArICoNCj4+ICsgKiBUcnkgdG8gcmVzaXplIEJBUjAgdG8gbWFrZSBhbGwgVlJB TSBDUFUgYWNjZXNzaWJsZS4NCj4+ICsgKi8NCj4+ICt2b2lkIGFtZGdwdV9yZXNpemVfYmFyMChz dHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldikgew0KPj4gKyAgICAgICB1MzIgc2l6ZSA9IG1heChp bG9nMihhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSAtIDEpICsgMSwgMjApIC0gMjA7DQo+PiArICAg ICAgIGludCByOw0KPj4gKw0KPj4gKyAgICAgICByID0gcGNpX3Jlc2l6ZV9yZXNvdXJjZShhZGV2 LT5wZGV2LCAwLCBzaXplKTsNCj4+ICsNCj4+ICsgICAgICAgaWYgKHIgPT0gLUVOT1RTVVBQKSB7 DQo+PiArICAgICAgICAgICAgICAgLyogVGhlIGhhcmR3YXJlIGRvbid0IHN1cHBvcnQgdGhlIGV4 dGVuc2lvbi4gKi8NCj4+ICsgICAgICAgICAgICAgICByZXR1cm47DQo+PiArDQo+PiArICAgICAg IH0gZWxzZSBpZiAociA9PSAtRU5PU1BDKSB7DQo+PiArICAgICAgICAgICAgICAgRFJNX0lORk8o Ik5vdCBlbm9pZ2ggUENJIGFkZHJlc3Mgc3BhY2UgZm9yIGEgbGFyZ2UgQkFSLiIpOw0KPj4gKyAg ICAgICB9IGVsc2UgaWYgKHIpIHsNCj4+ICsgICAgICAgICAgICAgICBEUk1fRVJST1IoIlByb2Js ZW0gcmVzaXppbmcgQkFSMCAoJWQpLiIsIHIpOw0KPj4gKyAgICAgICB9DQo+PiArDQo+PiArICAg ICAgIC8qIFJlaW5pdCB0aGUgZG9vcmJlbGwgbWFwcGluZywgaXQgaXMgbW9zdCBsaWtlbHkgbW92 ZWQgYXMgd2VsbCAqLw0KPj4gKyAgICAgICBhbWRncHVfZG9vcmJlbGxfZmluaShhZGV2KTsNCj4+ ICsgICAgICAgQlVHX09OKGFtZGdwdV9kb29yYmVsbF9pbml0KGFkZXYpKTsNCj4+ICt9DQo+PiAr DQo+PiAgIC8qDQo+PiAgICAqIEdQVSBoZWxwZXJzIGZ1bmN0aW9uLg0KPj4gICAgKi8NCj4+IGRp ZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjdfMC5jIA0KPj4gYi9k cml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjdfMC5jDQo+PiBpbmRleCBkYzliNmQ2Li4z NmE3YWE1IDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3 XzAuYw0KPj4gKysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0KPj4g QEAgLTM2NywxMyArMzY3LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y3XzBfbWNfaW5pdChzdHJ1Y3Qg YW1kZ3B1X2RldmljZSAqYWRldikNCj4+ICAgICAgICAgICAgICAgICAgYnJlYWs7DQo+PiAgICAg ICAgICB9DQo+PiAgICAgICAgICBhZGV2LT5tYy52cmFtX3dpZHRoID0gbnVtY2hhbiAqIGNoYW5z aXplOw0KPj4gLSAgICAgICAvKiBDb3VsZCBhcGVyIHNpemUgcmVwb3J0IDAgPyAqLw0KPj4gLSAg ICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwg MCk7DQo+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRl di0+cGRldiwgMCk7DQo+PiAgICAgICAgICAvKiBzaXplIGluIE1CIG9uIHNpICovDQo+PiAgICAg ICAgICBhZGV2LT5tYy5tY192cmFtX3NpemUgPSBSUkVHMzIobW1DT05GSUdfTUVNU0laRSkgKiAx MDI0VUxMICogMTAyNFVMTDsNCj4+ICAgICAgICAgIGFkZXYtPm1jLnJlYWxfdnJhbV9zaXplID0g UlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICogMTAyNFVMTCANCj4+ICogMTAyNFVMTDsNCj4+DQo+ PiArICAgICAgIGlmICghKGFkZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkpDQo+PiArICAgICAgICAg ICAgICAgYW1kZ3B1X3Jlc2l6ZV9iYXIwKGFkZXYpOw0KPj4gKyAgICAgICBhZGV2LT5tYy5hcGVy X2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCk7DQo+PiArICAgICAgIGFk ZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+PiAr DQo+PiAgICNpZmRlZiBDT05GSUdfWDg2XzY0DQo+PiAgICAgICAgICBpZiAoYWRldi0+ZmxhZ3Mg JiBBTURfSVNfQVBVKSB7DQo+PiAgICAgICAgICAgICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9 IA0KPj4gKCh1NjQpUlJFRzMyKG1tTUNfVk1fRkJfT0ZGU0VUKSkgPDwgMjI7IGRpZmYgLS1naXQg DQo+PiBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMgDQo+PiBiL2RyaXZl cnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMNCj4+IGluZGV4IGMwODdiMDAuLjc3NjFh ZDMgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5j DQo+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+PiBAQCAt NDU5LDEzICs0NTksMTUgQEAgc3RhdGljIGludCBnbWNfdjhfMF9tY19pbml0KHN0cnVjdCBhbWRn cHVfZGV2aWNlICphZGV2KQ0KPj4gICAgICAgICAgICAgICAgICBicmVhazsNCj4+ICAgICAgICAg IH0NCj4+ICAgICAgICAgIGFkZXYtPm1jLnZyYW1fd2lkdGggPSBudW1jaGFuICogY2hhbnNpemU7 DQo+PiAtICAgICAgIC8qIENvdWxkIGFwZXIgc2l6ZSByZXBvcnQgMCA/ICovDQo+PiAtICAgICAg IGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsN Cj4+IC0gICAgICAgYWRldi0+bWMuYXBlcl9zaXplID0gcGNpX3Jlc291cmNlX2xlbihhZGV2LT5w ZGV2LCAwKTsNCj4+ICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24gc2kgKi8NCj4+ICAgICAgICAg IGFkZXYtPm1jLm1jX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqIDEwMjRV TEwgKiAxMDI0VUxMOw0KPj4gICAgICAgICAgYWRldi0+bWMucmVhbF92cmFtX3NpemUgPSBSUkVH MzIobW1DT05GSUdfTUVNU0laRSkgKiAxMDI0VUxMIA0KPj4gKiAxMDI0VUxMOw0KPj4NCj4+ICsg ICAgICAgaWYgKCEoYWRldi0+ZmxhZ3MgJiBBTURfSVNfQVBVKSkNCj4+ICsgICAgICAgICAgICAg ICBhbWRncHVfcmVzaXplX2JhcjAoYWRldik7DQo+PiArICAgICAgIGFkZXYtPm1jLmFwZXJfYmFz ZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsNCj4+ICsgICAgICAgYWRldi0+ bWMuYXBlcl9zaXplID0gcGNpX3Jlc291cmNlX2xlbihhZGV2LT5wZGV2LCAwKTsNCj4+ICsNCj4+ ICAgI2lmZGVmIENPTkZJR19YODZfNjQNCj4+ICAgICAgICAgIGlmIChhZGV2LT5mbGFncyAmIEFN RF9JU19BUFUpIHsNCj4+ICAgICAgICAgICAgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gDQo+ PiAoKHU2NClSUkVHMzIobW1NQ19WTV9GQl9PRkZTRVQpKSA8PCAyMjsNCj4+IC0tDQo+PiAyLjcu NA0KPj4NCj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f DQo+PiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+PiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0 b3Aub3JnDQo+PiBodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZv L2RyaS1kZXZlbA0KDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fDQpkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQpkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0 b3Aub3JnDQpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Ry aS1kZXZlbA0K ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 8:25 ` Zhou, David(ChunMing) 0 siblings, 0 replies; 72+ messages in thread From: Zhou, David(ChunMing) @ 2017-03-15 8:25 UTC (permalink / raw) To: Christian König, Ayyappa Ch Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, helgaas-DgEjT+Ai2ygdnm+yROfE0A, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Does that means we don't need invisible vram later? David -----Original Message----- From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig Sent: Wednesday, March 15, 2017 3:38 PM To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access Carizzo is an APU and resizing BARs isn't needed nor supported there. The CPU can access the full stolen VRAM directly on that hardware. As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. Christian. Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > Is it possible on Carrizo asics? Or only supports on newer asics? > > On Mon, Mar 13, 2017 at 6:11 PM, Christian König > <deathsimple@vodafone.de> wrote: >> From: Christian König <christian.koenig@amd.com> >> >> Try to resize BAR0 to let CPU access all of VRAM. >> >> Signed-off-by: Christian König <christian.koenig@amd.com> >> --- >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> 4 files changed, 40 insertions(+), 6 deletions(-) >> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> index 3b81ded..905ded9 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >> struct ttm_mem_reg *mem); >> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >> amdgpu_mc *mc); >> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >> int amdgpu_ttm_init(struct amdgpu_device *adev); >> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> index 118f4e6..92955fe 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >> } >> >> +/** >> + * amdgpu_resize_bar0 - try to resize BAR0 >> + * >> + * @adev: amdgpu_device pointer >> + * >> + * Try to resize BAR0 to make all VRAM CPU accessible. >> + */ >> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >> + int r; >> + >> + r = pci_resize_resource(adev->pdev, 0, size); >> + >> + if (r == -ENOTSUPP) { >> + /* The hardware don't support the extension. */ >> + return; >> + >> + } else if (r == -ENOSPC) { >> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >> + } else if (r) { >> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> + } >> + >> + /* Reinit the doorbell mapping, it is most likely moved as well */ >> + amdgpu_doorbell_fini(adev); >> + BUG_ON(amdgpu_doorbell_init(adev)); >> +} >> + >> /* >> * GPU helpers function. >> */ >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> index dc9b6d6..36a7aa5 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >> * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = >> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> index c087b00..7761ad3 100644 >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >> break; >> } >> adev->mc.vram_width = numchan * chansize; >> - /* Could aper size report 0 ? */ >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> /* size in MB on si */ >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >> * 1024ULL; >> >> + if (!(adev->flags & AMD_IS_APU)) >> + amdgpu_resize_bar0(adev); >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> + >> #ifdef CONFIG_X86_64 >> if (adev->flags & AMD_IS_APU) { >> adev->mc.aper_base = >> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> -- >> 2.7.4 >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-15 8:25 ` Zhou, David(ChunMing) (?) @ 2017-03-15 9:29 ` Christian König -1 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 9:29 UTC (permalink / raw) To: Zhou, David(ChunMing), Ayyappa Ch Cc: linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx Yes, exactly that. Christian. Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > Does that means we don't need invisible vram later? > > David > > -----Original Message----- > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig > Sent: Wednesday, March 15, 2017 3:38 PM > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > The CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. > > Christian. > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> <deathsimple@vodafone.de> wrote: >>> From: Christian König <christian.koenig@amd.com> >>> >>> Try to resize BAR0 to let CPU access all of VRAM. >>> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>> 4 files changed, 40 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> index 3b81ded..905ded9 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >>> struct ttm_mem_reg *mem); >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >>> amdgpu_mc *mc); >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> index 118f4e6..92955fe 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>> } >>> >>> +/** >>> + * amdgpu_resize_bar0 - try to resize BAR0 >>> + * >>> + * @adev: amdgpu_device pointer >>> + * >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>> + */ >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>> + int r; >>> + >>> + r = pci_resize_resource(adev->pdev, 0, size); >>> + >>> + if (r == -ENOTSUPP) { >>> + /* The hardware don't support the extension. */ >>> + return; >>> + >>> + } else if (r == -ENOSPC) { >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >>> + } else if (r) { >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>> + } >>> + >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >>> + amdgpu_doorbell_fini(adev); >>> + BUG_ON(amdgpu_doorbell_init(adev)); >>> +} >>> + >>> /* >>> * GPU helpers function. >>> */ >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> index dc9b6d6..36a7aa5 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >>> * 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> index c087b00..7761ad3 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >>> * 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >>> -- >>> 2.7.4 >>> >>> _______________________________________________ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 9:29 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 9:29 UTC (permalink / raw) To: Zhou, David(ChunMing), Ayyappa Ch Cc: linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx Yes, exactly that. Christian. Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > Does that means we don't need invisible vram later? > > David > > -----Original Message----- > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig > Sent: Wednesday, March 15, 2017 3:38 PM > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > The CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. > > Christian. > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> <deathsimple@vodafone.de> wrote: >>> From: Christian König <christian.koenig@amd.com> >>> >>> Try to resize BAR0 to let CPU access all of VRAM. >>> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>> 4 files changed, 40 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> index 3b81ded..905ded9 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >>> struct ttm_mem_reg *mem); >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >>> amdgpu_mc *mc); >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> index 118f4e6..92955fe 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>> } >>> >>> +/** >>> + * amdgpu_resize_bar0 - try to resize BAR0 >>> + * >>> + * @adev: amdgpu_device pointer >>> + * >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>> + */ >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>> + int r; >>> + >>> + r = pci_resize_resource(adev->pdev, 0, size); >>> + >>> + if (r == -ENOTSUPP) { >>> + /* The hardware don't support the extension. */ >>> + return; >>> + >>> + } else if (r == -ENOSPC) { >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >>> + } else if (r) { >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>> + } >>> + >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >>> + amdgpu_doorbell_fini(adev); >>> + BUG_ON(amdgpu_doorbell_init(adev)); >>> +} >>> + >>> /* >>> * GPU helpers function. >>> */ >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> index dc9b6d6..36a7aa5 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >>> * 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> index c087b00..7761ad3 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >>> * 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >>> -- >>> 2.7.4 >>> >>> _______________________________________________ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 9:29 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 9:29 UTC (permalink / raw) To: Zhou, David(ChunMing), Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel Yes, exactly that. Christian. Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > Does that means we don't need invisible vram later? > > David > > -----Original Message----- > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig > Sent: Wednesday, March 15, 2017 3:38 PM > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > The CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. > > Christian. > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> <deathsimple@vodafone.de> wrote: >>> From: Christian König <christian.koenig@amd.com> >>> >>> Try to resize BAR0 to let CPU access all of VRAM. >>> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>> 4 files changed, 40 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> index 3b81ded..905ded9 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, >>> struct ttm_mem_reg *mem); >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >>> amdgpu_mc *mc); >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> index 118f4e6..92955fe 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>> } >>> >>> +/** >>> + * amdgpu_resize_bar0 - try to resize BAR0 >>> + * >>> + * @adev: amdgpu_device pointer >>> + * >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>> + */ >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>> + int r; >>> + >>> + r = pci_resize_resource(adev->pdev, 0, size); >>> + >>> + if (r == -ENOTSUPP) { >>> + /* The hardware don't support the extension. */ >>> + return; >>> + >>> + } else if (r == -ENOSPC) { >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >>> + } else if (r) { >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>> + } >>> + >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >>> + amdgpu_doorbell_fini(adev); >>> + BUG_ON(amdgpu_doorbell_init(adev)); >>> +} >>> + >>> /* >>> * GPU helpers function. >>> */ >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> index dc9b6d6..36a7aa5 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >>> * 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> index c087b00..7761ad3 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL >>> * 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >>> -- >>> 2.7.4 >>> >>> _______________________________________________ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:19 ` Zhang, Jerry 0 siblings, 0 replies; 72+ messages in thread From: Zhang, Jerry @ 2017-03-16 2:19 UTC (permalink / raw) To: Christian König, Zhou, David(ChunMing), Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel > -----Original Message----- > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of > Christian K?nig > Sent: Wednesday, March 15, 2017 17:29 > To: Zhou, David(ChunMing); Ayyappa Ch > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Yes, exactly that. (I'm not familiar with PCI too much.) Is there any restrict for PCI device? I'm concerning if any PCI couldn't support it on some motherboard. > > Christian. > > Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > > Does that means we don't need invisible vram later? > > > > David > > > > -----Original Message----- > > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > > Behalf Of Christian K?nig > > Sent: Wednesday, March 15, 2017 3:38 PM > > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; > > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > > helgaas@kernel.org; dri-devel@lists.freedesktop.org > > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > > The CPU can access the full stolen VRAM directly on that hardware. > > > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. > > > > Christian. > > > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > >> Is it possible on Carrizo asics? Or only supports on newer asics? > >> > >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König > >> <deathsimple@vodafone.de> wrote: > >>> From: Christian König <christian.koenig@amd.com> > >>> > >>> Try to resize BAR0 to let CPU access all of VRAM. > >>> > >>> Signed-off-by: Christian König <christian.koenig@amd.com> > >>> --- > >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > +++++++++++++++++++++++++++++ > >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >>> 4 files changed, 40 insertions(+), 6 deletions(-) > >>> > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> index 3b81ded..905ded9 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > amdgpu_device *adev, struct ttm_tt *ttm, > >>> struct ttm_mem_reg *mem); > >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct > amdgpu_mc *mc, u64 base); > >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > >>> amdgpu_mc *mc); > >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, > u64 size); > >>> int amdgpu_ttm_init(struct amdgpu_device *adev); > >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git > >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> index 118f4e6..92955fe 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device > *adev, struct amdgpu_mc *mc) > >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >>> } > >>> > >>> +/** > >>> + * amdgpu_resize_bar0 - try to resize BAR0 > >>> + * > >>> + * @adev: amdgpu_device pointer > >>> + * > >>> + * Try to resize BAR0 to make all VRAM CPU accessible. > >>> + */ > >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { > >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >>> + int r; > >>> + > >>> + r = pci_resize_resource(adev->pdev, 0, size); > >>> + > >>> + if (r == -ENOTSUPP) { > >>> + /* The hardware don't support the extension. */ > >>> + return; > >>> + > >>> + } else if (r == -ENOSPC) { > >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >>> + } else if (r) { > >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >>> + } > >>> + > >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >>> + amdgpu_doorbell_fini(adev); > >>> + BUG_ON(amdgpu_doorbell_init(adev)); > >>> +} > >>> + > >>> /* > >>> * GPU helpers function. > >>> */ > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> index dc9b6d6..36a7aa5 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device > *adev) > >>> break; > >>> } > >>> adev->mc.vram_width = numchan * chansize; > >>> - /* Could aper size report 0 ? */ > >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> /* size in MB on si */ > >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * > 1024ULL; > >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >>> 1024ULL > >>> * 1024ULL; > >>> > >>> + if (!(adev->flags & AMD_IS_APU)) > >>> + amdgpu_resize_bar0(adev); > >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> + > >>> #ifdef CONFIG_X86_64 > >>> if (adev->flags & AMD_IS_APU) { > >>> adev->mc.aper_base = > >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git > >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> index c087b00..7761ad3 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device > *adev) > >>> break; > >>> } > >>> adev->mc.vram_width = numchan * chansize; > >>> - /* Could aper size report 0 ? */ > >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> /* size in MB on si */ > >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * > 1024ULL; > >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >>> 1024ULL > >>> * 1024ULL; > >>> > >>> + if (!(adev->flags & AMD_IS_APU)) > >>> + amdgpu_resize_bar0(adev); > >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> + > >>> #ifdef CONFIG_X86_64 > >>> if (adev->flags & AMD_IS_APU) { > >>> adev->mc.aper_base = > >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > >>> -- > >>> 2.7.4 > >>> > >>> _______________________________________________ > >>> dri-devel mailing list > >>> dri-devel@lists.freedesktop.org > >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > > _______________________________________________ > > dri-devel mailing list > > dri-devel@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:19 ` Zhang, Jerry 0 siblings, 0 replies; 72+ messages in thread From: Zhang, Jerry @ 2017-03-16 2:19 UTC (permalink / raw) To: Christian König, Zhou, David(ChunMing), Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBkcmktZGV2ZWwgW21haWx0bzpk cmktZGV2ZWwtYm91bmNlc0BsaXN0cy5mcmVlZGVza3RvcC5vcmddIE9uIEJlaGFsZiBPZg0KPiBD aHJpc3RpYW4gSz9uaWcNCj4gU2VudDogV2VkbmVzZGF5LCBNYXJjaCAxNSwgMjAxNyAxNzoyOQ0K PiBUbzogWmhvdSwgRGF2aWQoQ2h1bk1pbmcpOyBBeXlhcHBhIENoDQo+IENjOiBsaW51eC1wY2lA dmdlci5rZXJuZWwub3JnOyBsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnOyBhbWQtDQo+IGdm eEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7IHBsYXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwu b3JnOw0KPiBoZWxnYWFzQGtlcm5lbC5vcmc7IGRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5v cmcNCj4gU3ViamVjdDogUmU6IFtQQVRDSCA0LzRdIGRybS9hbWRncHU6IHJlc2l6ZSBWUkFNIEJB UiBmb3IgQ1BVIGFjY2Vzcw0KPiANCj4gWWVzLCBleGFjdGx5IHRoYXQuDQoNCihJJ20gbm90IGZh bWlsaWFyIHdpdGggUENJIHRvbyBtdWNoLikNCklzIHRoZXJlIGFueSByZXN0cmljdCBmb3IgUENJ IGRldmljZT8NCkknbSBjb25jZXJuaW5nIGlmIGFueSBQQ0kgY291bGRuJ3Qgc3VwcG9ydCBpdCBv biBzb21lIG1vdGhlcmJvYXJkLg0KDQo+IA0KPiBDaHJpc3RpYW4uDQo+IA0KPiBBbSAxNS4wMy4y MDE3IHVtIDA5OjI1IHNjaHJpZWIgWmhvdSwgRGF2aWQoQ2h1bk1pbmcpOg0KPiA+IERvZXMgdGhh dCBtZWFucyB3ZSBkb24ndCBuZWVkIGludmlzaWJsZSB2cmFtIGxhdGVyPw0KPiA+DQo+ID4gRGF2 aWQNCj4gPg0KPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gRnJvbTogZHJpLWRl dmVsIFttYWlsdG86ZHJpLWRldmVsLWJvdW5jZXNAbGlzdHMuZnJlZWRlc2t0b3Aub3JnXSBPbg0K PiA+IEJlaGFsZiBPZiBDaHJpc3RpYW4gSz9uaWcNCj4gPiBTZW50OiBXZWRuZXNkYXksIE1hcmNo IDE1LCAyMDE3IDM6MzggUE0NCj4gPiBUbzogQXl5YXBwYSBDaCA8YXl5YXBwYS5jaC5saW51eEBn bWFpbC5jb20+DQo+ID4gQ2M6IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5l bEB2Z2VyLmtlcm5lbC5vcmc7DQo+ID4gYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7IHBs YXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwub3JnOw0KPiA+IGhlbGdhYXNAa2VybmVsLm9y ZzsgZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+IFN1YmplY3Q6IFJlOiBbUEFU Q0ggNC80XSBkcm0vYW1kZ3B1OiByZXNpemUgVlJBTSBCQVIgZm9yIENQVSBhY2Nlc3MNCj4gPg0K PiA+IENhcml6em8gaXMgYW4gQVBVIGFuZCByZXNpemluZyBCQVJzIGlzbid0IG5lZWRlZCBub3Ig c3VwcG9ydGVkIHRoZXJlLg0KPiA+IFRoZSBDUFUgY2FuIGFjY2VzcyB0aGUgZnVsbCBzdG9sZW4g VlJBTSBkaXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KPiA+DQo+ID4gQXMgZmFyIGFzIEkga25v dyBBU0lDcyB3aXRoIHN1cHBvcnQgZm9yIHRoaXMgYXJlIFRvbmdhLCBGaWppIGFuZCBhbGwgUG9s YXJpcyB2YXJpYW50cy4NCj4gPg0KPiA+IENocmlzdGlhbi4NCj4gPg0KPiA+IEFtIDE1LjAzLjIw MTcgdW0gMDg6MjMgc2NocmllYiBBeXlhcHBhIENoOg0KPiA+PiBJcyBpdCBwb3NzaWJsZSBvbiBD YXJyaXpvIGFzaWNzPyBPciBvbmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNzPw0KPiA+Pg0KPiA+ PiBPbiBNb24sIE1hciAxMywgMjAxNyBhdCA2OjExIFBNLCBDaHJpc3RpYW4gS8O2bmlnDQo+ID4+ IDxkZWF0aHNpbXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6DQo+ID4+PiBGcm9tOiBDaHJpc3RpYW4g S8O2bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+DQo+ID4+Pg0KPiA+Pj4gVHJ5IHRvIHJl c2l6ZSBCQVIwIHRvIGxldCBDUFUgYWNjZXNzIGFsbCBvZiBWUkFNLg0KPiA+Pj4NCj4gPj4+IFNp Z25lZC1vZmYtYnk6IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4N Cj4gPj4+IC0tLQ0KPiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1Lmgg ICAgICAgIHwgIDEgKw0KPiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1 X2RldmljZS5jIHwgMjkNCj4gKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4gPj4+ICAg IGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMgICAgICB8ICA4ICsrKysrLS0t DQo+ID4+PiAgICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jICAgICAgfCAg OCArKysrKy0tLQ0KPiA+Pj4gICAgNCBmaWxlcyBjaGFuZ2VkLCA0MCBpbnNlcnRpb25zKCspLCA2 IGRlbGV0aW9ucygtKQ0KPiA+Pj4NCj4gPj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0v YW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9h bWRncHUuaA0KPiA+Pj4gaW5kZXggM2I4MWRlZC4uOTA1ZGVkOSAxMDA2NDQNCj4gPj4+IC0tLSBh L2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+PiArKysgYi9kcml2ZXJz L2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+Pj4gQEAgLTE3MTksNiArMTcxOSw3IEBA IHVpbnQ2NF90IGFtZGdwdV90dG1fdHRfcHRlX2ZsYWdzKHN0cnVjdA0KPiBhbWRncHVfZGV2aWNl ICphZGV2LCBzdHJ1Y3QgdHRtX3R0ICp0dG0sDQo+ID4+PiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIHN0cnVjdCB0dG1fbWVtX3JlZyAqbWVtKTsNCj4gPj4+ICAgIHZvaWQgYW1k Z3B1X3ZyYW1fbG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiBh bWRncHVfbWMgKm1jLCB1NjQgYmFzZSk7DQo+ID4+PiAgICB2b2lkIGFtZGdwdV9ndHRfbG9jYXRp b24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiA+Pj4gYW1kZ3B1X21jICpt Yyk7DQo+ID4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFtZGdwdV9kZXZpY2Ug KmFkZXYpOw0KPiA+Pj4gICAgdm9pZCBhbWRncHVfdHRtX3NldF9hY3RpdmVfdnJhbV9zaXplKHN0 cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LA0KPiB1NjQgc2l6ZSk7DQo+ID4+PiAgICBpbnQgYW1k Z3B1X3R0bV9pbml0KHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KTsNCj4gPj4+ICAgIHZvaWQg YW1kZ3B1X3R0bV9maW5pKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KTsgZGlmZiAtLWdpdA0K PiA+Pj4gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMNCj4gPj4+ IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+ID4+PiBpbmRl eCAxMThmNGU2Li45Mjk1NWZlIDEwMDY0NA0KPiA+Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2Ft ZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+ID4+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1k L2FtZGdwdS9hbWRncHVfZGV2aWNlLmMNCj4gPj4+IEBAIC02OTIsNiArNjkyLDM1IEBAIHZvaWQg YW1kZ3B1X2d0dF9sb2NhdGlvbihzdHJ1Y3QgYW1kZ3B1X2RldmljZQ0KPiAqYWRldiwgc3RydWN0 IGFtZGdwdV9tYyAqbWMpDQo+ID4+PiAgICAgICAgICAgICAgICAgICAgICAgICAgIG1jLT5ndHRf c2l6ZSA+PiAyMCwgbWMtPmd0dF9zdGFydCwgbWMtPmd0dF9lbmQpOw0KPiA+Pj4gICAgfQ0KPiA+ Pj4NCj4gPj4+ICsvKioNCj4gPj4+ICsgKiBhbWRncHVfcmVzaXplX2JhcjAgLSB0cnkgdG8gcmVz aXplIEJBUjANCj4gPj4+ICsgKg0KPiA+Pj4gKyAqIEBhZGV2OiBhbWRncHVfZGV2aWNlIHBvaW50 ZXINCj4gPj4+ICsgKg0KPiA+Pj4gKyAqIFRyeSB0byByZXNpemUgQkFSMCB0byBtYWtlIGFsbCBW UkFNIENQVSBhY2Nlc3NpYmxlLg0KPiA+Pj4gKyAqLw0KPiA+Pj4gK3ZvaWQgYW1kZ3B1X3Jlc2l6 ZV9iYXIwKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KSB7DQo+ID4+PiArICAgICAgIHUzMiBz aXplID0gbWF4KGlsb2cyKGFkZXYtPm1jLnJlYWxfdnJhbV9zaXplIC0gMSkgKyAxLCAyMCkgLSAy MDsNCj4gPj4+ICsgICAgICAgaW50IHI7DQo+ID4+PiArDQo+ID4+PiArICAgICAgIHIgPSBwY2lf cmVzaXplX3Jlc291cmNlKGFkZXYtPnBkZXYsIDAsIHNpemUpOw0KPiA+Pj4gKw0KPiA+Pj4gKyAg ICAgICBpZiAociA9PSAtRU5PVFNVUFApIHsNCj4gPj4+ICsgICAgICAgICAgICAgICAvKiBUaGUg aGFyZHdhcmUgZG9uJ3Qgc3VwcG9ydCB0aGUgZXh0ZW5zaW9uLiAqLw0KPiA+Pj4gKyAgICAgICAg ICAgICAgIHJldHVybjsNCj4gPj4+ICsNCj4gPj4+ICsgICAgICAgfSBlbHNlIGlmIChyID09IC1F Tk9TUEMpIHsNCj4gPj4+ICsgICAgICAgICAgICAgICBEUk1fSU5GTygiTm90IGVub2lnaCBQQ0kg YWRkcmVzcyBzcGFjZSBmb3IgYSBsYXJnZSBCQVIuIik7DQo+ID4+PiArICAgICAgIH0gZWxzZSBp ZiAocikgew0KPiA+Pj4gKyAgICAgICAgICAgICAgIERSTV9FUlJPUigiUHJvYmxlbSByZXNpemlu ZyBCQVIwICglZCkuIiwgcik7DQo+ID4+PiArICAgICAgIH0NCj4gPj4+ICsNCj4gPj4+ICsgICAg ICAgLyogUmVpbml0IHRoZSBkb29yYmVsbCBtYXBwaW5nLCBpdCBpcyBtb3N0IGxpa2VseSBtb3Zl ZCBhcyB3ZWxsICovDQo+ID4+PiArICAgICAgIGFtZGdwdV9kb29yYmVsbF9maW5pKGFkZXYpOw0K PiA+Pj4gKyAgICAgICBCVUdfT04oYW1kZ3B1X2Rvb3JiZWxsX2luaXQoYWRldikpOw0KPiA+Pj4g K30NCj4gPj4+ICsNCj4gPj4+ICAgIC8qDQo+ID4+PiAgICAgKiBHUFUgaGVscGVycyBmdW5jdGlv bi4NCj4gPj4+ICAgICAqLw0KPiA+Pj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQv YW1kZ3B1L2dtY192N18wLmMNCj4gPj4+IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21j X3Y3XzAuYw0KPiA+Pj4gaW5kZXggZGM5YjZkNi4uMzZhN2FhNSAxMDA2NDQNCj4gPj4+IC0tLSBh L2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMNCj4gPj4+ICsrKyBiL2RyaXZl cnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMNCj4gPj4+IEBAIC0zNjcsMTMgKzM2Nywx NSBAQCBzdGF0aWMgaW50IGdtY192N18wX21jX2luaXQoc3RydWN0IGFtZGdwdV9kZXZpY2UNCj4g KmFkZXYpDQo+ID4+PiAgICAgICAgICAgICAgICAgICBicmVhazsNCj4gPj4+ICAgICAgICAgICB9 DQo+ID4+PiAgICAgICAgICAgYWRldi0+bWMudnJhbV93aWR0aCA9IG51bWNoYW4gKiBjaGFuc2l6 ZTsNCj4gPj4+IC0gICAgICAgLyogQ291bGQgYXBlciBzaXplIHJlcG9ydCAwID8gKi8NCj4gPj4+ IC0gICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jlc291cmNlX3N0YXJ0KGFkZXYtPnBk ZXYsIDApOw0KPiA+Pj4gLSAgICAgICBhZGV2LT5tYy5hcGVyX3NpemUgPSBwY2lfcmVzb3VyY2Vf bGVuKGFkZXYtPnBkZXYsIDApOw0KPiA+Pj4gICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24gc2kg Ki8NCj4gPj4+ICAgICAgICAgICBhZGV2LT5tYy5tY192cmFtX3NpemUgPSBSUkVHMzIobW1DT05G SUdfTUVNU0laRSkgKiAxMDI0VUxMICoNCj4gMTAyNFVMTDsNCj4gPj4+ICAgICAgICAgICBhZGV2 LT5tYy5yZWFsX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqDQo+ID4+PiAx MDI0VUxMDQo+ID4+PiAqIDEwMjRVTEw7DQo+ID4+Pg0KPiA+Pj4gKyAgICAgICBpZiAoIShhZGV2 LT5mbGFncyAmIEFNRF9JU19BUFUpKQ0KPiA+Pj4gKyAgICAgICAgICAgICAgIGFtZGdwdV9yZXNp emVfYmFyMChhZGV2KTsNCj4gPj4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jl c291cmNlX3N0YXJ0KGFkZXYtPnBkZXYsIDApOw0KPiA+Pj4gKyAgICAgICBhZGV2LT5tYy5hcGVy X3NpemUgPSBwY2lfcmVzb3VyY2VfbGVuKGFkZXYtPnBkZXYsIDApOw0KPiA+Pj4gKw0KPiA+Pj4g ICAgI2lmZGVmIENPTkZJR19YODZfNjQNCj4gPj4+ICAgICAgICAgICBpZiAoYWRldi0+ZmxhZ3Mg JiBBTURfSVNfQVBVKSB7DQo+ID4+PiAgICAgICAgICAgICAgICAgICBhZGV2LT5tYy5hcGVyX2Jh c2UgPQ0KPiA+Pj4gKCh1NjQpUlJFRzMyKG1tTUNfVk1fRkJfT0ZGU0VUKSkgPDwgMjI7IGRpZmYg LS1naXQNCj4gPj4+IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYw0KPiA+ Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+ID4+PiBpbmRleCBj MDg3YjAwLi43NzYxYWQzIDEwMDY0NA0KPiA+Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9h bWRncHUvZ21jX3Y4XzAuYw0KPiA+Pj4gKysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv Z21jX3Y4XzAuYw0KPiA+Pj4gQEAgLTQ1OSwxMyArNDU5LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y4 XzBfbWNfaW5pdChzdHJ1Y3QgYW1kZ3B1X2RldmljZQ0KPiAqYWRldikNCj4gPj4+ICAgICAgICAg ICAgICAgICAgIGJyZWFrOw0KPiA+Pj4gICAgICAgICAgIH0NCj4gPj4+ICAgICAgICAgICBhZGV2 LT5tYy52cmFtX3dpZHRoID0gbnVtY2hhbiAqIGNoYW5zaXplOw0KPiA+Pj4gLSAgICAgICAvKiBD b3VsZCBhcGVyIHNpemUgcmVwb3J0IDAgPyAqLw0KPiA+Pj4gLSAgICAgICBhZGV2LT5tYy5hcGVy X2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCk7DQo+ID4+PiAtICAgICAg IGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ ID4+PiAgICAgICAgICAgLyogc2l6ZSBpbiBNQiBvbiBzaSAqLw0KPiA+Pj4gICAgICAgICAgIGFk ZXYtPm1jLm1jX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqIDEwMjRVTEwg Kg0KPiAxMDI0VUxMOw0KPiA+Pj4gICAgICAgICAgIGFkZXYtPm1jLnJlYWxfdnJhbV9zaXplID0g UlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gPj4+IDEwMjRVTEwNCj4gPj4+ICogMTAyNFVM TDsNCj4gPj4+DQo+ID4+PiArICAgICAgIGlmICghKGFkZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkp DQo+ID4+PiArICAgICAgICAgICAgICAgYW1kZ3B1X3Jlc2l6ZV9iYXIwKGFkZXYpOw0KPiA+Pj4g KyAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRl diwgMCk7DQo+ID4+PiArICAgICAgIGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9s ZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+PiArDQo+ID4+PiAgICAjaWZkZWYgQ09ORklHX1g4Nl82 NA0KPiA+Pj4gICAgICAgICAgIGlmIChhZGV2LT5mbGFncyAmIEFNRF9JU19BUFUpIHsNCj4gPj4+ ICAgICAgICAgICAgICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9DQo+ID4+PiAoKHU2NClSUkVH MzIobW1NQ19WTV9GQl9PRkZTRVQpKSA8PCAyMjsNCj4gPj4+IC0tDQo+ID4+PiAyLjcuNA0KPiA+ Pj4NCj4gPj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f DQo+ID4+PiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+ID4+PiBkcmktZGV2ZWxAbGlzdHMuZnJl ZWRlc2t0b3Aub3JnDQo+ID4+PiBodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFu L2xpc3RpbmZvL2RyaS1kZXZlbA0KPiA+DQo+ID4gX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX18NCj4gPiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+ID4gZHJp LWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+IGh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0 b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsDQo+ID4gX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPiBhbWQtZ2Z4IG1haWxpbmcgbGlzdA0K PiA+IGFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQo+ID4gaHR0cHM6Ly9saXN0cy5mcmVl ZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4DQo+IA0KPiANCj4gX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gZHJpLWRldmVsIG1haWxp bmcgbGlzdA0KPiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQo+IGh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsDQo= ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:19 ` Zhang, Jerry 0 siblings, 0 replies; 72+ messages in thread From: Zhang, Jerry @ 2017-03-16 2:19 UTC (permalink / raw) To: Christian König, Zhou, David(ChunMing), Ayyappa Ch Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, helgaas-DgEjT+Ai2ygdnm+yROfE0A, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW > -----Original Message----- > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of > Christian K?nig > Sent: Wednesday, March 15, 2017 17:29 > To: Zhou, David(ChunMing); Ayyappa Ch > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Yes, exactly that. (I'm not familiar with PCI too much.) Is there any restrict for PCI device? I'm concerning if any PCI couldn't support it on some motherboard. > > Christian. > > Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > > Does that means we don't need invisible vram later? > > > > David > > > > -----Original Message----- > > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > > Behalf Of Christian K?nig > > Sent: Wednesday, March 15, 2017 3:38 PM > > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; > > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > > helgaas@kernel.org; dri-devel@lists.freedesktop.org > > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > > The CPU can access the full stolen VRAM directly on that hardware. > > > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. > > > > Christian. > > > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > >> Is it possible on Carrizo asics? Or only supports on newer asics? > >> > >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König > >> <deathsimple@vodafone.de> wrote: > >>> From: Christian König <christian.koenig@amd.com> > >>> > >>> Try to resize BAR0 to let CPU access all of VRAM. > >>> > >>> Signed-off-by: Christian König <christian.koenig@amd.com> > >>> --- > >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > +++++++++++++++++++++++++++++ > >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >>> 4 files changed, 40 insertions(+), 6 deletions(-) > >>> > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> index 3b81ded..905ded9 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > amdgpu_device *adev, struct ttm_tt *ttm, > >>> struct ttm_mem_reg *mem); > >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct > amdgpu_mc *mc, u64 base); > >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > >>> amdgpu_mc *mc); > >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, > u64 size); > >>> int amdgpu_ttm_init(struct amdgpu_device *adev); > >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git > >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> index 118f4e6..92955fe 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device > *adev, struct amdgpu_mc *mc) > >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >>> } > >>> > >>> +/** > >>> + * amdgpu_resize_bar0 - try to resize BAR0 > >>> + * > >>> + * @adev: amdgpu_device pointer > >>> + * > >>> + * Try to resize BAR0 to make all VRAM CPU accessible. > >>> + */ > >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { > >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >>> + int r; > >>> + > >>> + r = pci_resize_resource(adev->pdev, 0, size); > >>> + > >>> + if (r == -ENOTSUPP) { > >>> + /* The hardware don't support the extension. */ > >>> + return; > >>> + > >>> + } else if (r == -ENOSPC) { > >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >>> + } else if (r) { > >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >>> + } > >>> + > >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >>> + amdgpu_doorbell_fini(adev); > >>> + BUG_ON(amdgpu_doorbell_init(adev)); > >>> +} > >>> + > >>> /* > >>> * GPU helpers function. > >>> */ > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> index dc9b6d6..36a7aa5 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device > *adev) > >>> break; > >>> } > >>> adev->mc.vram_width = numchan * chansize; > >>> - /* Could aper size report 0 ? */ > >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> /* size in MB on si */ > >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * > 1024ULL; > >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >>> 1024ULL > >>> * 1024ULL; > >>> > >>> + if (!(adev->flags & AMD_IS_APU)) > >>> + amdgpu_resize_bar0(adev); > >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> + > >>> #ifdef CONFIG_X86_64 > >>> if (adev->flags & AMD_IS_APU) { > >>> adev->mc.aper_base = > >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git > >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> index c087b00..7761ad3 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device > *adev) > >>> break; > >>> } > >>> adev->mc.vram_width = numchan * chansize; > >>> - /* Could aper size report 0 ? */ > >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> /* size in MB on si */ > >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * > 1024ULL; > >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >>> 1024ULL > >>> * 1024ULL; > >>> > >>> + if (!(adev->flags & AMD_IS_APU)) > >>> + amdgpu_resize_bar0(adev); > >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >>> + > >>> #ifdef CONFIG_X86_64 > >>> if (adev->flags & AMD_IS_APU) { > >>> adev->mc.aper_base = > >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > >>> -- > >>> 2.7.4 > >>> > >>> _______________________________________________ > >>> dri-devel mailing list > >>> dri-devel@lists.freedesktop.org > >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > > _______________________________________________ > > dri-devel mailing list > > dri-devel@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:25 ` Alex Deucher 0 siblings, 0 replies; 72+ messages in thread From: Alex Deucher @ 2017-03-16 2:25 UTC (permalink / raw) To: Zhang, Jerry Cc: Christian König, Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: >> -----Original Message----- >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of >> Christian K?nig >> Sent: Wednesday, March 15, 2017 17:29 >> To: Zhou, David(ChunMing); Ayyappa Ch >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> helgaas@kernel.org; dri-devel@lists.freedesktop.org >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> >> Yes, exactly that. > > (I'm not familiar with PCI too much.) > Is there any restrict for PCI device? > I'm concerning if any PCI couldn't support it on some motherboard. It depends on the PCI root bridge. This patch set only implements support for AMD root bridges. Intel and other vendors would need similar code. Alex > >> >> Christian. >> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): >> > Does that means we don't need invisible vram later? >> > >> > David >> > >> > -----Original Message----- >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >> > Behalf Of Christian K?nig >> > Sent: Wednesday, March 15, 2017 3:38 PM >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there. >> > The CPU can access the full stolen VRAM directly on that hardware. >> > >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. >> > >> > Christian. >> > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> >> <deathsimple@vodafone.de> wrote: >> >>> From: Christian König <christian.koenig@amd.com> >> >>> >> >>> Try to resize BAR0 to let CPU access all of VRAM. >> >>> >> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >> >>> --- >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >> +++++++++++++++++++++++++++++ >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> >>> 4 files changed, 40 insertions(+), 6 deletions(-) >> >>> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> index 3b81ded..905ded9 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >> amdgpu_device *adev, struct ttm_tt *ttm, >> >>> struct ttm_mem_reg *mem); >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct >> amdgpu_mc *mc, u64 base); >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >> >>> amdgpu_mc *mc); >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, >> u64 size); >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> index 118f4e6..92955fe 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device >> *adev, struct amdgpu_mc *mc) >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >> >>> } >> >>> >> >>> +/** >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 >> >>> + * >> >>> + * @adev: amdgpu_device pointer >> >>> + * >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >> >>> + */ >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >> >>> + int r; >> >>> + >> >>> + r = pci_resize_resource(adev->pdev, 0, size); >> >>> + >> >>> + if (r == -ENOTSUPP) { >> >>> + /* The hardware don't support the extension. */ >> >>> + return; >> >>> + >> >>> + } else if (r == -ENOSPC) { >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >> >>> + } else if (r) { >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> >>> + } >> >>> + >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >> >>> + amdgpu_doorbell_fini(adev); >> >>> + BUG_ON(amdgpu_doorbell_init(adev)); >> >>> +} >> >>> + >> >>> /* >> >>> * GPU helpers function. >> >>> */ >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> index dc9b6d6..36a7aa5 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >> *adev) >> >>> break; >> >>> } >> >>> adev->mc.vram_width = numchan * chansize; >> >>> - /* Could aper size report 0 ? */ >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> /* size in MB on si */ >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >> 1024ULL; >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >> >>> 1024ULL >> >>> * 1024ULL; >> >>> >> >>> + if (!(adev->flags & AMD_IS_APU)) >> >>> + amdgpu_resize_bar0(adev); >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> + >> >>> #ifdef CONFIG_X86_64 >> >>> if (adev->flags & AMD_IS_APU) { >> >>> adev->mc.aper_base = >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> index c087b00..7761ad3 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >> *adev) >> >>> break; >> >>> } >> >>> adev->mc.vram_width = numchan * chansize; >> >>> - /* Could aper size report 0 ? */ >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> /* size in MB on si */ >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >> 1024ULL; >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >> >>> 1024ULL >> >>> * 1024ULL; >> >>> >> >>> + if (!(adev->flags & AMD_IS_APU)) >> >>> + amdgpu_resize_bar0(adev); >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> + >> >>> #ifdef CONFIG_X86_64 >> >>> if (adev->flags & AMD_IS_APU) { >> >>> adev->mc.aper_base = >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> >>> -- >> >>> 2.7.4 >> >>> >> >>> _______________________________________________ >> >>> dri-devel mailing list >> >>> dri-devel@lists.freedesktop.org >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >> > >> > _______________________________________________ >> > dri-devel mailing list >> > dri-devel@lists.freedesktop.org >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel >> > _______________________________________________ >> > amd-gfx mailing list >> > amd-gfx@lists.freedesktop.org >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx >> >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:25 ` Alex Deucher 0 siblings, 0 replies; 72+ messages in thread From: Alex Deucher @ 2017-03-16 2:25 UTC (permalink / raw) To: Zhang, Jerry Cc: Christian König, Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: >> -----Original Message----- >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Beha= lf Of >> Christian K?nig >> Sent: Wednesday, March 15, 2017 17:29 >> To: Zhou, David(ChunMing); Ayyappa Ch >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> helgaas@kernel.org; dri-devel@lists.freedesktop.org >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> >> Yes, exactly that. > > (I'm not familiar with PCI too much.) > Is there any restrict for PCI device? > I'm concerning if any PCI couldn't support it on some motherboard. It depends on the PCI root bridge. This patch set only implements support for AMD root bridges. Intel and other vendors would need similar code. Alex > >> >> Christian. >> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): >> > Does that means we don't need invisible vram later? >> > >> > David >> > >> > -----Original Message----- >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >> > Behalf Of Christian K?nig >> > Sent: Wednesday, March 15, 2017 3:38 PM >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there. >> > The CPU can access the full stolen VRAM directly on that hardware. >> > >> > As far as I know ASICs with support for this are Tonga, Fiji and all P= olaris variants. >> > >> > Christian. >> > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian K=C3=B6nig >> >> <deathsimple@vodafone.de> wrote: >> >>> From: Christian K=C3=B6nig <christian.koenig@amd.com> >> >>> >> >>> Try to resize BAR0 to let CPU access all of VRAM. >> >>> >> >>> Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com> >> >>> --- >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >> +++++++++++++++++++++++++++++ >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> >>> 4 files changed, 40 insertions(+), 6 deletions(-) >> >>> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> index 3b81ded..905ded9 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >> amdgpu_device *adev, struct ttm_tt *ttm, >> >>> struct ttm_mem_reg *mem); >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct >> amdgpu_mc *mc, u64 base); >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >> >>> amdgpu_mc *mc); >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, >> u64 size); >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> index 118f4e6..92955fe 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device >> *adev, struct amdgpu_mc *mc) >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt= _end); >> >>> } >> >>> >> >>> +/** >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 >> >>> + * >> >>> + * @adev: amdgpu_device pointer >> >>> + * >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >> >>> + */ >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >> >>> + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20)= - 20; >> >>> + int r; >> >>> + >> >>> + r =3D pci_resize_resource(adev->pdev, 0, size); >> >>> + >> >>> + if (r =3D=3D -ENOTSUPP) { >> >>> + /* The hardware don't support the extension. */ >> >>> + return; >> >>> + >> >>> + } else if (r =3D=3D -ENOSPC) { >> >>> + DRM_INFO("Not enoigh PCI address space for a large B= AR."); >> >>> + } else if (r) { >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> >>> + } >> >>> + >> >>> + /* Reinit the doorbell mapping, it is most likely moved as w= ell */ >> >>> + amdgpu_doorbell_fini(adev); >> >>> + BUG_ON(amdgpu_doorbell_init(adev)); >> >>> +} >> >>> + >> >>> /* >> >>> * GPU helpers function. >> >>> */ >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> index dc9b6d6..36a7aa5 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_devi= ce >> *adev) >> >>> break; >> >>> } >> >>> adev->mc.vram_width =3D numchan * chansize; >> >>> - /* Could aper size report 0 ? */ >> >>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >> >>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >> >>> /* size in MB on si */ >> >>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024U= LL * >> 1024ULL; >> >>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * >> >>> 1024ULL >> >>> * 1024ULL; >> >>> >> >>> + if (!(adev->flags & AMD_IS_APU)) >> >>> + amdgpu_resize_bar0(adev); >> >>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >> >>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >> >>> + >> >>> #ifdef CONFIG_X86_64 >> >>> if (adev->flags & AMD_IS_APU) { >> >>> adev->mc.aper_base =3D >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> index c087b00..7761ad3 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_devi= ce >> *adev) >> >>> break; >> >>> } >> >>> adev->mc.vram_width =3D numchan * chansize; >> >>> - /* Could aper size report 0 ? */ >> >>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >> >>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >> >>> /* size in MB on si */ >> >>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024U= LL * >> 1024ULL; >> >>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * >> >>> 1024ULL >> >>> * 1024ULL; >> >>> >> >>> + if (!(adev->flags & AMD_IS_APU)) >> >>> + amdgpu_resize_bar0(adev); >> >>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >> >>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >> >>> + >> >>> #ifdef CONFIG_X86_64 >> >>> if (adev->flags & AMD_IS_APU) { >> >>> adev->mc.aper_base =3D >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> >>> -- >> >>> 2.7.4 >> >>> >> >>> _______________________________________________ >> >>> dri-devel mailing list >> >>> dri-devel@lists.freedesktop.org >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >> > >> > _______________________________________________ >> > dri-devel mailing list >> > dri-devel@lists.freedesktop.org >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel >> > _______________________________________________ >> > amd-gfx mailing list >> > amd-gfx@lists.freedesktop.org >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx >> >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:25 ` Alex Deucher 0 siblings, 0 replies; 72+ messages in thread From: Alex Deucher @ 2017-03-16 2:25 UTC (permalink / raw) To: Zhang, Jerry Cc: Zhou, David(ChunMing), Ayyappa Ch, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, Christian König, helgaas-DgEjT+Ai2ygdnm+yROfE0A, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: >> -----Original Message----- >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of >> Christian K?nig >> Sent: Wednesday, March 15, 2017 17:29 >> To: Zhou, David(ChunMing); Ayyappa Ch >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> helgaas@kernel.org; dri-devel@lists.freedesktop.org >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> >> Yes, exactly that. > > (I'm not familiar with PCI too much.) > Is there any restrict for PCI device? > I'm concerning if any PCI couldn't support it on some motherboard. It depends on the PCI root bridge. This patch set only implements support for AMD root bridges. Intel and other vendors would need similar code. Alex > >> >> Christian. >> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): >> > Does that means we don't need invisible vram later? >> > >> > David >> > >> > -----Original Message----- >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >> > Behalf Of Christian K?nig >> > Sent: Wednesday, March 15, 2017 3:38 PM >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there. >> > The CPU can access the full stolen VRAM directly on that hardware. >> > >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants. >> > >> > Christian. >> > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> >> <deathsimple@vodafone.de> wrote: >> >>> From: Christian König <christian.koenig@amd.com> >> >>> >> >>> Try to resize BAR0 to let CPU access all of VRAM. >> >>> >> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >> >>> --- >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >> +++++++++++++++++++++++++++++ >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >> >>> 4 files changed, 40 insertions(+), 6 deletions(-) >> >>> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> index 3b81ded..905ded9 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >> amdgpu_device *adev, struct ttm_tt *ttm, >> >>> struct ttm_mem_reg *mem); >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct >> amdgpu_mc *mc, u64 base); >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >> >>> amdgpu_mc *mc); >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, >> u64 size); >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> index 118f4e6..92955fe 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device >> *adev, struct amdgpu_mc *mc) >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >> >>> } >> >>> >> >>> +/** >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 >> >>> + * >> >>> + * @adev: amdgpu_device pointer >> >>> + * >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >> >>> + */ >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >> >>> + int r; >> >>> + >> >>> + r = pci_resize_resource(adev->pdev, 0, size); >> >>> + >> >>> + if (r == -ENOTSUPP) { >> >>> + /* The hardware don't support the extension. */ >> >>> + return; >> >>> + >> >>> + } else if (r == -ENOSPC) { >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >> >>> + } else if (r) { >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >> >>> + } >> >>> + >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >> >>> + amdgpu_doorbell_fini(adev); >> >>> + BUG_ON(amdgpu_doorbell_init(adev)); >> >>> +} >> >>> + >> >>> /* >> >>> * GPU helpers function. >> >>> */ >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> index dc9b6d6..36a7aa5 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >> *adev) >> >>> break; >> >>> } >> >>> adev->mc.vram_width = numchan * chansize; >> >>> - /* Could aper size report 0 ? */ >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> /* size in MB on si */ >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >> 1024ULL; >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >> >>> 1024ULL >> >>> * 1024ULL; >> >>> >> >>> + if (!(adev->flags & AMD_IS_APU)) >> >>> + amdgpu_resize_bar0(adev); >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> + >> >>> #ifdef CONFIG_X86_64 >> >>> if (adev->flags & AMD_IS_APU) { >> >>> adev->mc.aper_base = >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> index c087b00..7761ad3 100644 >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >> *adev) >> >>> break; >> >>> } >> >>> adev->mc.vram_width = numchan * chansize; >> >>> - /* Could aper size report 0 ? */ >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> /* size in MB on si */ >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >> 1024ULL; >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >> >>> 1024ULL >> >>> * 1024ULL; >> >>> >> >>> + if (!(adev->flags & AMD_IS_APU)) >> >>> + amdgpu_resize_bar0(adev); >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >> >>> + >> >>> #ifdef CONFIG_X86_64 >> >>> if (adev->flags & AMD_IS_APU) { >> >>> adev->mc.aper_base = >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >> >>> -- >> >>> 2.7.4 >> >>> >> >>> _______________________________________________ >> >>> dri-devel mailing list >> >>> dri-devel@lists.freedesktop.org >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >> > >> > _______________________________________________ >> > dri-devel mailing list >> > dri-devel@lists.freedesktop.org >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel >> > _______________________________________________ >> > amd-gfx mailing list >> > amd-gfx@lists.freedesktop.org >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx >> >> >> _______________________________________________ >> dri-devel mailing list >> dri-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-16 2:25 ` Alex Deucher @ 2017-03-16 2:41 ` Zhang, Jerry -1 siblings, 0 replies; 72+ messages in thread From: Zhang, Jerry @ 2017-03-16 2:41 UTC (permalink / raw) To: Alex Deucher Cc: Christian König, Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx Thanks for your info. I see. Regards, Jerry (Junwei Zhang) Linux Base Graphics SRDC Software Development _____________________________________ > -----Original Message----- > From: Alex Deucher [mailto:alexdeucher@gmail.com] > Sent: Thursday, March 16, 2017 10:25 > To: Zhang, Jerry > Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux- > pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri- > devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; amd-gfx@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: > >> -----Original Message----- > >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> Behalf Of Christian K?nig > >> Sent: Wednesday, March 15, 2017 17:29 > >> To: Zhou, David(ChunMing); Ayyappa Ch > >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > >> Yes, exactly that. > > > > (I'm not familiar with PCI too much.) > > Is there any restrict for PCI device? > > I'm concerning if any PCI couldn't support it on some motherboard. > > It depends on the PCI root bridge. This patch set only implements support for > AMD root bridges. Intel and other vendors would need similar code. > > Alex > > > > >> > >> Christian. > >> > >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > >> > Does that means we don't need invisible vram later? > >> > > >> > David > >> > > >> > -----Original Message----- > >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> > Behalf Of Christian K?nig > >> > Sent: Wednesday, March 15, 2017 3:38 PM > >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; > >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there. > >> > The CPU can access the full stolen VRAM directly on that hardware. > >> > > >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris > variants. > >> > > >> > Christian. > >> > > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > >> >> Is it possible on Carrizo asics? Or only supports on newer asics? > >> >> > >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König > >> >> <deathsimple@vodafone.de> wrote: > >> >>> From: Christian König <christian.koenig@amd.com> > >> >>> > >> >>> Try to resize BAR0 to let CPU access all of VRAM. > >> >>> > >> >>> Signed-off-by: Christian König <christian.koenig@amd.com> > >> >>> --- > >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > >> +++++++++++++++++++++++++++++ > >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >> >>> 4 files changed, 40 insertions(+), 6 deletions(-) > >> >>> > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> index 3b81ded..905ded9 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > >> amdgpu_device *adev, struct ttm_tt *ttm, > >> >>> struct ttm_mem_reg *mem); > >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct > >> amdgpu_mc *mc, u64 base); > >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > >> >>> amdgpu_mc *mc); > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device > >> >>> *adev, > >> u64 size); > >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev); > >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> index 118f4e6..92955fe 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct > >> >>> amdgpu_device > >> *adev, struct amdgpu_mc *mc) > >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >> >>> } > >> >>> > >> >>> +/** > >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 > >> >>> + * > >> >>> + * @adev: amdgpu_device pointer > >> >>> + * > >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. > >> >>> + */ > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { > >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >> >>> + int r; > >> >>> + > >> >>> + r = pci_resize_resource(adev->pdev, 0, size); > >> >>> + > >> >>> + if (r == -ENOTSUPP) { > >> >>> + /* The hardware don't support the extension. */ > >> >>> + return; > >> >>> + > >> >>> + } else if (r == -ENOSPC) { > >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >> >>> + } else if (r) { > >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >> >>> + } > >> >>> + > >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >> >>> + amdgpu_doorbell_fini(adev); > >> >>> + BUG_ON(amdgpu_doorbell_init(adev)); > >> >>> +} > >> >>> + > >> >>> /* > >> >>> * GPU helpers function. > >> >>> */ > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> index dc9b6d6..36a7aa5 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>> break; > >> >>> } > >> >>> adev->mc.vram_width = numchan * chansize; > >> >>> - /* Could aper size report 0 ? */ > >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> /* size in MB on si */ > >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> + if (!(adev->flags & AMD_IS_APU)) > >> >>> + amdgpu_resize_bar0(adev); > >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> + > >> >>> #ifdef CONFIG_X86_64 > >> >>> if (adev->flags & AMD_IS_APU) { > >> >>> adev->mc.aper_base = > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> index c087b00..7761ad3 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>> break; > >> >>> } > >> >>> adev->mc.vram_width = numchan * chansize; > >> >>> - /* Could aper size report 0 ? */ > >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> /* size in MB on si */ > >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> + if (!(adev->flags & AMD_IS_APU)) > >> >>> + amdgpu_resize_bar0(adev); > >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> + > >> >>> #ifdef CONFIG_X86_64 > >> >>> if (adev->flags & AMD_IS_APU) { > >> >>> adev->mc.aper_base = > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > >> >>> -- > >> >>> 2.7.4 > >> >>> > >> >>> _______________________________________________ > >> >>> dri-devel mailing list > >> >>> dri-devel@lists.freedesktop.org > >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > > >> > _______________________________________________ > >> > dri-devel mailing list > >> > dri-devel@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > _______________________________________________ > >> > amd-gfx mailing list > >> > amd-gfx@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > >> > >> > >> _______________________________________________ > >> dri-devel mailing list > >> dri-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-16 2:41 ` Zhang, Jerry 0 siblings, 0 replies; 72+ messages in thread From: Zhang, Jerry @ 2017-03-16 2:41 UTC (permalink / raw) To: Alex Deucher Cc: Christian König, Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx VGhhbmtzIGZvciB5b3VyIGluZm8uDQpJIHNlZS4NCg0KUmVnYXJkcywNCkplcnJ5IChKdW53ZWkg WmhhbmcpDQoNCkxpbnV4IEJhc2UgR3JhcGhpY3MNClNSREMgU29mdHdhcmUgRGV2ZWxvcG1lbnQN Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCg0KDQo+IC0tLS0tT3JpZ2lu YWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEFsZXggRGV1Y2hlciBbbWFpbHRvOmFsZXhkZXVjaGVy QGdtYWlsLmNvbV0NCj4gU2VudDogVGh1cnNkYXksIE1hcmNoIDE2LCAyMDE3IDEwOjI1DQo+IFRv OiBaaGFuZywgSmVycnkNCj4gQ2M6IENocmlzdGlhbiBLw7ZuaWc7IFpob3UsIERhdmlkKENodW5N aW5nKTsgQXl5YXBwYSBDaDsgbGludXgtDQo+IHBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtl cm5lbEB2Z2VyLmtlcm5lbC5vcmc7IGRyaS0NCj4gZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3Jn OyBwbGF0Zm9ybS1kcml2ZXIteDg2QHZnZXIua2VybmVsLm9yZzsNCj4gaGVsZ2Fhc0BrZXJuZWwu b3JnOyBhbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiBTdWJqZWN0OiBSZTogW1BBVENI IDQvNF0gZHJtL2FtZGdwdTogcmVzaXplIFZSQU0gQkFSIGZvciBDUFUgYWNjZXNzDQo+IA0KPiBP biBXZWQsIE1hciAxNSwgMjAxNyBhdCAxMDoxOSBQTSwgWmhhbmcsIEplcnJ5IDxKZXJyeS5aaGFu Z0BhbWQuY29tPiB3cm90ZToNCj4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4g RnJvbTogZHJpLWRldmVsIFttYWlsdG86ZHJpLWRldmVsLWJvdW5jZXNAbGlzdHMuZnJlZWRlc2t0 b3Aub3JnXSBPbg0KPiA+PiBCZWhhbGYgT2YgQ2hyaXN0aWFuIEs/bmlnDQo+ID4+IFNlbnQ6IFdl ZG5lc2RheSwgTWFyY2ggMTUsIDIwMTcgMTc6MjkNCj4gPj4gVG86IFpob3UsIERhdmlkKENodW5N aW5nKTsgQXl5YXBwYSBDaA0KPiA+PiBDYzogbGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsgbGlu dXgta2VybmVsQHZnZXIua2VybmVsLm9yZzsgYW1kLQ0KPiA+PiBnZnhAbGlzdHMuZnJlZWRlc2t0 b3Aub3JnOyBwbGF0Zm9ybS1kcml2ZXIteDg2QHZnZXIua2VybmVsLm9yZzsNCj4gPj4gaGVsZ2Fh c0BrZXJuZWwub3JnOyBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQo+ID4+IFN1Ympl Y3Q6IFJlOiBbUEFUQ0ggNC80XSBkcm0vYW1kZ3B1OiByZXNpemUgVlJBTSBCQVIgZm9yIENQVSBh Y2Nlc3MNCj4gPj4NCj4gPj4gWWVzLCBleGFjdGx5IHRoYXQuDQo+ID4NCj4gPiAoSSdtIG5vdCBm YW1pbGlhciB3aXRoIFBDSSB0b28gbXVjaC4pDQo+ID4gSXMgdGhlcmUgYW55IHJlc3RyaWN0IGZv ciBQQ0kgZGV2aWNlPw0KPiA+IEknbSBjb25jZXJuaW5nIGlmIGFueSBQQ0kgY291bGRuJ3Qgc3Vw cG9ydCBpdCBvbiBzb21lIG1vdGhlcmJvYXJkLg0KPiANCj4gSXQgZGVwZW5kcyBvbiB0aGUgUENJ IHJvb3QgYnJpZGdlLiAgVGhpcyBwYXRjaCBzZXQgb25seSBpbXBsZW1lbnRzIHN1cHBvcnQgZm9y DQo+IEFNRCByb290IGJyaWRnZXMuICBJbnRlbCBhbmQgb3RoZXIgdmVuZG9ycyB3b3VsZCBuZWVk IHNpbWlsYXIgY29kZS4NCj4gDQo+IEFsZXgNCj4gDQo+ID4NCj4gPj4NCj4gPj4gQ2hyaXN0aWFu Lg0KPiA+Pg0KPiA+PiBBbSAxNS4wMy4yMDE3IHVtIDA5OjI1IHNjaHJpZWIgWmhvdSwgRGF2aWQo Q2h1bk1pbmcpOg0KPiA+PiA+IERvZXMgdGhhdCBtZWFucyB3ZSBkb24ndCBuZWVkIGludmlzaWJs ZSB2cmFtIGxhdGVyPw0KPiA+PiA+DQo+ID4+ID4gRGF2aWQNCj4gPj4gPg0KPiA+PiA+IC0tLS0t T3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+ID4gRnJvbTogZHJpLWRldmVsIFttYWlsdG86ZHJp LWRldmVsLWJvdW5jZXNAbGlzdHMuZnJlZWRlc2t0b3Aub3JnXSBPbg0KPiA+PiA+IEJlaGFsZiBP ZiBDaHJpc3RpYW4gSz9uaWcNCj4gPj4gPiBTZW50OiBXZWRuZXNkYXksIE1hcmNoIDE1LCAyMDE3 IDM6MzggUE0NCj4gPj4gPiBUbzogQXl5YXBwYSBDaCA8YXl5YXBwYS5jaC5saW51eEBnbWFpbC5j b20+DQo+ID4+ID4gQ2M6IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5lbEB2 Z2VyLmtlcm5lbC5vcmc7DQo+ID4+ID4gYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7IHBs YXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwub3JnOw0KPiA+PiA+IGhlbGdhYXNAa2VybmVs Lm9yZzsgZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+PiA+IFN1YmplY3Q6IFJl OiBbUEFUQ0ggNC80XSBkcm0vYW1kZ3B1OiByZXNpemUgVlJBTSBCQVIgZm9yIENQVSBhY2Nlc3MN Cj4gPj4gPg0KPiA+PiA+IENhcml6em8gaXMgYW4gQVBVIGFuZCByZXNpemluZyBCQVJzIGlzbid0 IG5lZWRlZCBub3Igc3VwcG9ydGVkIHRoZXJlLg0KPiA+PiA+IFRoZSBDUFUgY2FuIGFjY2VzcyB0 aGUgZnVsbCBzdG9sZW4gVlJBTSBkaXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KPiA+PiA+DQo+ ID4+ID4gQXMgZmFyIGFzIEkga25vdyBBU0lDcyB3aXRoIHN1cHBvcnQgZm9yIHRoaXMgYXJlIFRv bmdhLCBGaWppIGFuZCBhbGwgUG9sYXJpcw0KPiB2YXJpYW50cy4NCj4gPj4gPg0KPiA+PiA+IENo cmlzdGlhbi4NCj4gPj4gPg0KPiA+PiA+IEFtIDE1LjAzLjIwMTcgdW0gMDg6MjMgc2NocmllYiBB eXlhcHBhIENoOg0KPiA+PiA+PiBJcyBpdCBwb3NzaWJsZSBvbiBDYXJyaXpvIGFzaWNzPyBPciBv bmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNzPw0KPiA+PiA+Pg0KPiA+PiA+PiBPbiBNb24sIE1h ciAxMywgMjAxNyBhdCA2OjExIFBNLCBDaHJpc3RpYW4gS8O2bmlnDQo+ID4+ID4+IDxkZWF0aHNp bXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6DQo+ID4+ID4+PiBGcm9tOiBDaHJpc3RpYW4gS8O2bmln IDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+DQo+ID4+ID4+Pg0KPiA+PiA+Pj4gVHJ5IHRvIHJl c2l6ZSBCQVIwIHRvIGxldCBDUFUgYWNjZXNzIGFsbCBvZiBWUkFNLg0KPiA+PiA+Pj4NCj4gPj4g Pj4+IFNpZ25lZC1vZmYtYnk6IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1k LmNvbT4NCj4gPj4gPj4+IC0tLQ0KPiA+PiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRn cHUvYW1kZ3B1LmggICAgICAgIHwgIDEgKw0KPiA+PiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2Ft ZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jIHwgMjkNCj4gPj4gKysrKysrKysrKysrKysrKysrKysr KysrKysrKysNCj4gPj4gPj4+ICAgIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18w LmMgICAgICB8ICA4ICsrKysrLS0tDQo+ID4+ID4+PiAgICBkcml2ZXJzL2dwdS9kcm0vYW1kL2Ft ZGdwdS9nbWNfdjhfMC5jICAgICAgfCAgOCArKysrKy0tLQ0KPiA+PiA+Pj4gICAgNCBmaWxlcyBj aGFuZ2VkLCA0MCBpbnNlcnRpb25zKCspLCA2IGRlbGV0aW9ucygtKQ0KPiA+PiA+Pj4NCj4gPj4g Pj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+ PiA+Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+PiA+Pj4gaW5k ZXggM2I4MWRlZC4uOTA1ZGVkOSAxMDA2NDQNCj4gPj4gPj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2Ry bS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+ID4+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1k L2FtZGdwdS9hbWRncHUuaA0KPiA+PiA+Pj4gQEAgLTE3MTksNiArMTcxOSw3IEBAIHVpbnQ2NF90 IGFtZGdwdV90dG1fdHRfcHRlX2ZsYWdzKHN0cnVjdA0KPiA+PiBhbWRncHVfZGV2aWNlICphZGV2 LCBzdHJ1Y3QgdHRtX3R0ICp0dG0sDQo+ID4+ID4+PiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIHN0cnVjdCB0dG1fbWVtX3JlZyAqbWVtKTsNCj4gPj4gPj4+ICAgIHZvaWQgYW1k Z3B1X3ZyYW1fbG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiA+ PiBhbWRncHVfbWMgKm1jLCB1NjQgYmFzZSk7DQo+ID4+ID4+PiAgICB2b2lkIGFtZGdwdV9ndHRf bG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiA+PiA+Pj4gYW1k Z3B1X21jICptYyk7DQo+ID4+ID4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFt ZGdwdV9kZXZpY2UgKmFkZXYpOw0KPiA+PiA+Pj4gICAgdm9pZCBhbWRncHVfdHRtX3NldF9hY3Rp dmVfdnJhbV9zaXplKHN0cnVjdCBhbWRncHVfZGV2aWNlDQo+ID4+ID4+PiAqYWRldiwNCj4gPj4g dTY0IHNpemUpOw0KPiA+PiA+Pj4gICAgaW50IGFtZGdwdV90dG1faW5pdChzdHJ1Y3QgYW1kZ3B1 X2RldmljZSAqYWRldik7DQo+ID4+ID4+PiAgICB2b2lkIGFtZGdwdV90dG1fZmluaShzdHJ1Y3Qg YW1kZ3B1X2RldmljZSAqYWRldik7IGRpZmYgLS1naXQNCj4gPj4gPj4+IGEvZHJpdmVycy9ncHUv ZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+ID4+ID4+PiBiL2RyaXZlcnMvZ3B1L2Ry bS9hbWQvYW1kZ3B1L2FtZGdwdV9kZXZpY2UuYw0KPiA+PiA+Pj4gaW5kZXggMTE4ZjRlNi4uOTI5 NTVmZSAxMDA2NDQNCj4gPj4gPj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2Ft ZGdwdV9kZXZpY2UuYw0KPiA+PiA+Pj4gKysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv YW1kZ3B1X2RldmljZS5jDQo+ID4+ID4+PiBAQCAtNjkyLDYgKzY5MiwzNSBAQCB2b2lkIGFtZGdw dV9ndHRfbG9jYXRpb24oc3RydWN0DQo+ID4+ID4+PiBhbWRncHVfZGV2aWNlDQo+ID4+ICphZGV2 LCBzdHJ1Y3QgYW1kZ3B1X21jICptYykNCj4gPj4gPj4+ICAgICAgICAgICAgICAgICAgICAgICAg ICAgbWMtPmd0dF9zaXplID4+IDIwLCBtYy0+Z3R0X3N0YXJ0LCBtYy0+Z3R0X2VuZCk7DQo+ID4+ ID4+PiAgICB9DQo+ID4+ID4+Pg0KPiA+PiA+Pj4gKy8qKg0KPiA+PiA+Pj4gKyAqIGFtZGdwdV9y ZXNpemVfYmFyMCAtIHRyeSB0byByZXNpemUgQkFSMA0KPiA+PiA+Pj4gKyAqDQo+ID4+ID4+PiAr ICogQGFkZXY6IGFtZGdwdV9kZXZpY2UgcG9pbnRlcg0KPiA+PiA+Pj4gKyAqDQo+ID4+ID4+PiAr ICogVHJ5IHRvIHJlc2l6ZSBCQVIwIHRvIG1ha2UgYWxsIFZSQU0gQ1BVIGFjY2Vzc2libGUuDQo+ ID4+ID4+PiArICovDQo+ID4+ID4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFt ZGdwdV9kZXZpY2UgKmFkZXYpIHsNCj4gPj4gPj4+ICsgICAgICAgdTMyIHNpemUgPSBtYXgoaWxv ZzIoYWRldi0+bWMucmVhbF92cmFtX3NpemUgLSAxKSArIDEsIDIwKSAtIDIwOw0KPiA+PiA+Pj4g KyAgICAgICBpbnQgcjsNCj4gPj4gPj4+ICsNCj4gPj4gPj4+ICsgICAgICAgciA9IHBjaV9yZXNp emVfcmVzb3VyY2UoYWRldi0+cGRldiwgMCwgc2l6ZSk7DQo+ID4+ID4+PiArDQo+ID4+ID4+PiAr ICAgICAgIGlmIChyID09IC1FTk9UU1VQUCkgew0KPiA+PiA+Pj4gKyAgICAgICAgICAgICAgIC8q IFRoZSBoYXJkd2FyZSBkb24ndCBzdXBwb3J0IHRoZSBleHRlbnNpb24uICovDQo+ID4+ID4+PiAr ICAgICAgICAgICAgICAgcmV0dXJuOw0KPiA+PiA+Pj4gKw0KPiA+PiA+Pj4gKyAgICAgICB9IGVs c2UgaWYgKHIgPT0gLUVOT1NQQykgew0KPiA+PiA+Pj4gKyAgICAgICAgICAgICAgIERSTV9JTkZP KCJOb3QgZW5vaWdoIFBDSSBhZGRyZXNzIHNwYWNlIGZvciBhIGxhcmdlIEJBUi4iKTsNCj4gPj4g Pj4+ICsgICAgICAgfSBlbHNlIGlmIChyKSB7DQo+ID4+ID4+PiArICAgICAgICAgICAgICAgRFJN X0VSUk9SKCJQcm9ibGVtIHJlc2l6aW5nIEJBUjAgKCVkKS4iLCByKTsNCj4gPj4gPj4+ICsgICAg ICAgfQ0KPiA+PiA+Pj4gKw0KPiA+PiA+Pj4gKyAgICAgICAvKiBSZWluaXQgdGhlIGRvb3JiZWxs IG1hcHBpbmcsIGl0IGlzIG1vc3QgbGlrZWx5IG1vdmVkIGFzIHdlbGwgKi8NCj4gPj4gPj4+ICsg ICAgICAgYW1kZ3B1X2Rvb3JiZWxsX2ZpbmkoYWRldik7DQo+ID4+ID4+PiArICAgICAgIEJVR19P TihhbWRncHVfZG9vcmJlbGxfaW5pdChhZGV2KSk7DQo+ID4+ID4+PiArfQ0KPiA+PiA+Pj4gKw0K PiA+PiA+Pj4gICAgLyoNCj4gPj4gPj4+ICAgICAqIEdQVSBoZWxwZXJzIGZ1bmN0aW9uLg0KPiA+ PiA+Pj4gICAgICovDQo+ID4+ID4+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9h bWRncHUvZ21jX3Y3XzAuYw0KPiA+PiA+Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9n bWNfdjdfMC5jDQo+ID4+ID4+PiBpbmRleCBkYzliNmQ2Li4zNmE3YWE1IDEwMDY0NA0KPiA+PiA+ Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0KPiA+PiA+Pj4g KysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0KPiA+PiA+Pj4gQEAg LTM2NywxMyArMzY3LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y3XzBfbWNfaW5pdChzdHJ1Y3QNCj4g Pj4gPj4+IGFtZGdwdV9kZXZpY2UNCj4gPj4gKmFkZXYpDQo+ID4+ID4+PiAgICAgICAgICAgICAg ICAgICBicmVhazsNCj4gPj4gPj4+ICAgICAgICAgICB9DQo+ID4+ID4+PiAgICAgICAgICAgYWRl di0+bWMudnJhbV93aWR0aCA9IG51bWNoYW4gKiBjaGFuc2l6ZTsNCj4gPj4gPj4+IC0gICAgICAg LyogQ291bGQgYXBlciBzaXplIHJlcG9ydCAwID8gKi8NCj4gPj4gPj4+IC0gICAgICAgYWRldi0+ bWMuYXBlcl9iYXNlID0gcGNpX3Jlc291cmNlX3N0YXJ0KGFkZXYtPnBkZXYsIDApOw0KPiA+PiA+ Pj4gLSAgICAgICBhZGV2LT5tYy5hcGVyX3NpemUgPSBwY2lfcmVzb3VyY2VfbGVuKGFkZXYtPnBk ZXYsIDApOw0KPiA+PiA+Pj4gICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24gc2kgKi8NCj4gPj4g Pj4+ICAgICAgICAgICBhZGV2LT5tYy5tY192cmFtX3NpemUgPSBSUkVHMzIobW1DT05GSUdfTUVN U0laRSkgKg0KPiA+PiA+Pj4gMTAyNFVMTCAqDQo+ID4+IDEwMjRVTEw7DQo+ID4+ID4+PiAgICAg ICAgICAgYWRldi0+bWMucmVhbF92cmFtX3NpemUgPSBSUkVHMzIobW1DT05GSUdfTUVNU0laRSkg Kg0KPiA+PiA+Pj4gMTAyNFVMTA0KPiA+PiA+Pj4gKiAxMDI0VUxMOw0KPiA+PiA+Pj4NCj4gPj4g Pj4+ICsgICAgICAgaWYgKCEoYWRldi0+ZmxhZ3MgJiBBTURfSVNfQVBVKSkNCj4gPj4gPj4+ICsg ICAgICAgICAgICAgICBhbWRncHVfcmVzaXplX2JhcjAoYWRldik7DQo+ID4+ID4+PiArICAgICAg IGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsN Cj4gPj4gPj4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9zaXplID0gcGNpX3Jlc291cmNlX2xlbihh ZGV2LT5wZGV2LCAwKTsNCj4gPj4gPj4+ICsNCj4gPj4gPj4+ICAgICNpZmRlZiBDT05GSUdfWDg2 XzY0DQo+ID4+ID4+PiAgICAgICAgICAgaWYgKGFkZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkgew0K PiA+PiA+Pj4gICAgICAgICAgICAgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0NCj4gPj4gPj4+ ICgodTY0KVJSRUczMihtbU1DX1ZNX0ZCX09GRlNFVCkpIDw8IDIyOyBkaWZmIC0tZ2l0DQo+ID4+ ID4+PiBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMNCj4gPj4gPj4+IGIv ZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYw0KPiA+PiA+Pj4gaW5kZXggYzA4 N2IwMC4uNzc2MWFkMyAxMDA2NDQNCj4gPj4gPj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQv YW1kZ3B1L2dtY192OF8wLmMNCj4gPj4gPj4+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k Z3B1L2dtY192OF8wLmMNCj4gPj4gPj4+IEBAIC00NTksMTMgKzQ1OSwxNSBAQCBzdGF0aWMgaW50 IGdtY192OF8wX21jX2luaXQoc3RydWN0DQo+ID4+ID4+PiBhbWRncHVfZGV2aWNlDQo+ID4+ICph ZGV2KQ0KPiA+PiA+Pj4gICAgICAgICAgICAgICAgICAgYnJlYWs7DQo+ID4+ID4+PiAgICAgICAg ICAgfQ0KPiA+PiA+Pj4gICAgICAgICAgIGFkZXYtPm1jLnZyYW1fd2lkdGggPSBudW1jaGFuICog Y2hhbnNpemU7DQo+ID4+ID4+PiAtICAgICAgIC8qIENvdWxkIGFwZXIgc2l6ZSByZXBvcnQgMCA/ ICovDQo+ID4+ID4+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9yZXNvdXJjZV9z dGFydChhZGV2LT5wZGV2LCAwKTsNCj4gPj4gPj4+IC0gICAgICAgYWRldi0+bWMuYXBlcl9zaXpl ID0gcGNpX3Jlc291cmNlX2xlbihhZGV2LT5wZGV2LCAwKTsNCj4gPj4gPj4+ICAgICAgICAgICAv KiBzaXplIGluIE1CIG9uIHNpICovDQo+ID4+ID4+PiAgICAgICAgICAgYWRldi0+bWMubWNfdnJh bV9zaXplID0gUlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gPj4gPj4+IDEwMjRVTEwgKg0K PiA+PiAxMDI0VUxMOw0KPiA+PiA+Pj4gICAgICAgICAgIGFkZXYtPm1jLnJlYWxfdnJhbV9zaXpl ID0gUlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gPj4gPj4+IDEwMjRVTEwNCj4gPj4gPj4+ ICogMTAyNFVMTDsNCj4gPj4gPj4+DQo+ID4+ID4+PiArICAgICAgIGlmICghKGFkZXYtPmZsYWdz ICYgQU1EX0lTX0FQVSkpDQo+ID4+ID4+PiArICAgICAgICAgICAgICAgYW1kZ3B1X3Jlc2l6ZV9i YXIwKGFkZXYpOw0KPiA+PiA+Pj4gKyAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVz b3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCk7DQo+ID4+ID4+PiArICAgICAgIGFkZXYtPm1jLmFw ZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+ID4+PiArDQo+ ID4+ID4+PiAgICAjaWZkZWYgQ09ORklHX1g4Nl82NA0KPiA+PiA+Pj4gICAgICAgICAgIGlmIChh ZGV2LT5mbGFncyAmIEFNRF9JU19BUFUpIHsNCj4gPj4gPj4+ICAgICAgICAgICAgICAgICAgIGFk ZXYtPm1jLmFwZXJfYmFzZSA9DQo+ID4+ID4+PiAoKHU2NClSUkVHMzIobW1NQ19WTV9GQl9PRkZT RVQpKSA8PCAyMjsNCj4gPj4gPj4+IC0tDQo+ID4+ID4+PiAyLjcuNA0KPiA+PiA+Pj4NCj4gPj4g Pj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+ID4+ ID4+PiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+ID4+ID4+PiBkcmktZGV2ZWxAbGlzdHMuZnJl ZWRlc2t0b3Aub3JnDQo+ID4+ID4+PiBodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWls bWFuL2xpc3RpbmZvL2RyaS1kZXZlbA0KPiA+PiA+DQo+ID4+ID4gX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4gPiBkcmktZGV2ZWwgbWFpbGluZyBs aXN0DQo+ID4+ID4gZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+PiA+IGh0dHBz Oi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsDQo+ID4+ ID4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4g PiBhbWQtZ2Z4IG1haWxpbmcgbGlzdA0KPiA+PiA+IGFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Au b3JnDQo+ID4+ID4gaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m by9hbWQtZ2Z4DQo+ID4+DQo+ID4+DQo+ID4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fDQo+ID4+IGRyaS1kZXZlbCBtYWlsaW5nIGxpc3QNCj4gPj4gZHJp LWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+PiBodHRwczovL2xpc3RzLmZyZWVkZXNr dG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbA0KPiA+IF9fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+ID4gYW1kLWdmeCBtYWlsaW5nIGxpc3QN Cj4gPiBhbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+IGh0dHBzOi8vbGlzdHMuZnJl ZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vYW1kLWdmeA0K ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-16 2:41 ` Zhang, Jerry (?) @ 2017-03-23 14:30 ` Sagalovitch, Serguei -1 siblings, 0 replies; 72+ messages in thread From: Sagalovitch, Serguei @ 2017-03-23 14:30 UTC (permalink / raw) To: Zhang, Jerry, Alex Deucher, Christian König Cc: Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx Christian, - Are we going to support resizing BAR when kernel modesetting is not enabled and we are running in console under VBIOS control (VESA/VGA)? - Should we restore PCI configuration if amdgpu will be unloaded? - In function amdgpu_resize_bar0(): If resizing for "max" size failed should we try other sizes? What do you think? Sincerely yours, Serguei Sagalovitch From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com> Sent: March 15, 2017 10:41 PM To: Alex Deucher Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access Thanks for your info. I see. Regards, Jerry (Junwei Zhang) Linux Base Graphics SRDC Software Development _____________________________________ > -----Original Message----- > From: Alex Deucher [mailto:alexdeucher@gmail.com] > Sent: Thursday, March 16, 2017 10:25 > To: Zhang, Jerry > Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux- > pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri- > devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; amd-gfx@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: > >> -----Original Message----- > >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> Behalf Of Christian K?nig > >> Sent: Wednesday, March 15, 2017 17:29 > >> To: Zhou, David(ChunMing); Ayyappa Ch > >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > >> Yes, exactly that. > > > > (I'm not familiar with PCI too much.) > > Is there any restrict for PCI device? > > I'm concerning if any PCI couldn't support it on some motherboard. > > It depends on the PCI root bridge. This patch set only implements support for > AMD root bridges. Intel and other vendors would need similar code. > > Alex > > > > >> > >> Christian. > >> > >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > >> > Does that means we don't need invisible vram later? > >> > > >> > David > >> > > >> > -----Original Message----- > >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> > Behalf Of Christian K?nig > >> > Sent: Wednesday, March 15, 2017 3:38 PM > >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; > >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there. > >> > The CPU can access the full stolen VRAM directly on that hardware. > >> > > >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris > variants. > >> > > >> > Christian. > >> > > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > >> >> Is it possible on Carrizo asics? Or only supports on newer asics? > >> >> > >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König > >> >> <deathsimple@vodafone.de> wrote: > >> >>> From: Christian König <christian.koenig@amd.com> > >> >>> > >> >>> Try to resize BAR0 to let CPU access all of VRAM. > >> >>> > >> >>> Signed-off-by: Christian König <christian.koenig@amd.com> > >> >>> --- > >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > >> +++++++++++++++++++++++++++++ > >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >> >>> 4 files changed, 40 insertions(+), 6 deletions(-) > >> >>> > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> index 3b81ded..905ded9 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > >> amdgpu_device *adev, struct ttm_tt *ttm, > >> >>> struct ttm_mem_reg *mem); > >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct > >> amdgpu_mc *mc, u64 base); > >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > >> >>> amdgpu_mc *mc); > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device > >> >>> *adev, > >> u64 size); > >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev); > >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> index 118f4e6..92955fe 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct > >> >>> amdgpu_device > >> *adev, struct amdgpu_mc *mc) > >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >> >>> } > >> >>> > >> >>> +/** > >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 > >> >>> + * > >> >>> + * @adev: amdgpu_device pointer > >> >>> + * > >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. > >> >>> + */ > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { > >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >> >>> + int r; > >> >>> + > >> >>> + r = pci_resize_resource(adev->pdev, 0, size); > >> >>> + > >> >>> + if (r == -ENOTSUPP) { > >> >>> + /* The hardware don't support the extension. */ > >> >>> + return; > >> >>> + > >> >>> + } else if (r == -ENOSPC) { > >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >> >>> + } else if (r) { > >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >> >>> + } > >> >>> + > >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >> >>> + amdgpu_doorbell_fini(adev); > >> >>> + BUG_ON(amdgpu_doorbell_init(adev)); > >> >>> +} > >> >>> + > >> >>> /* > >> >>> * GPU helpers function. > >> >>> */ > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> index dc9b6d6..36a7aa5 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>> break; > >> >>> } > >> >>> adev->mc.vram_width = numchan * chansize; > >> >>> - /* Could aper size report 0 ? */ > >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> /* size in MB on si */ > >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> + if (!(adev->flags & AMD_IS_APU)) > >> >>> + amdgpu_resize_bar0(adev); > >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> + > >> >>> #ifdef CONFIG_X86_64 > >> >>> if (adev->flags & AMD_IS_APU) { > >> >>> adev->mc.aper_base = > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> index c087b00..7761ad3 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>> break; > >> >>> } > >> >>> adev->mc.vram_width = numchan * chansize; > >> >>> - /* Could aper size report 0 ? */ > >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> /* size in MB on si */ > >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> + if (!(adev->flags & AMD_IS_APU)) > >> >>> + amdgpu_resize_bar0(adev); > >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> + > >> >>> #ifdef CONFIG_X86_64 > >> >>> if (adev->flags & AMD_IS_APU) { > >> >>> adev->mc.aper_base = > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > >> >>> -- > >> >>> 2.7.4 > >> >>> > >> >>> _______________________________________________ > >> >>> dri-devel mailing list > >> >>> dri-devel@lists.freedesktop.org > >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > > >> > _______________________________________________ > >> > dri-devel mailing list > >> > dri-devel@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > _______________________________________________ > >> > amd-gfx mailing list > >> > amd-gfx@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > >> > >> > >> _______________________________________________ > >> dri-devel mailing list > >> dri-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-23 14:30 ` Sagalovitch, Serguei 0 siblings, 0 replies; 72+ messages in thread From: Sagalovitch, Serguei @ 2017-03-23 14:30 UTC (permalink / raw) To: Zhang, Jerry, Alex Deucher, Christian König Cc: Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx Christian, - Are we going to support resizing BAR when kernel=20 modesetting is not enabled and we are running in console=20 under VBIOS control (VESA/VGA)?=20 - Should we restore PCI configuration if amdgpu will be unloaded? - In function amdgpu_resize_bar0(): If resizing for "max" size failed should we try other=20 sizes? What do you think? Sincerely yours, Serguei Sagalovitch From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, J= erry <Jerry.Zhang@amd.com> Sent: March 15, 2017 10:41 PM To: Alex Deucher Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-ker= nel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@v= ger.kernel.org; Christian K=F6nig; helgaas@kernel.org; amd-gfx@lists.freede= sktop.org Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access =A0 =20 Thanks for your info. I see. Regards, Jerry (Junwei Zhang) Linux Base Graphics SRDC Software Development _____________________________________ > -----Original Message----- > From: Alex Deucher [mailto:alexdeucher@gmail.com] > Sent: Thursday, March 16, 2017 10:25 > To: Zhang, Jerry > Cc: Christian K=F6nig; Zhou, David(ChunMing); Ayyappa Ch; linux- > pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri- > devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; amd-gfx@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >=20 > On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrot= e: > >> -----Original Message----- > >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> Behalf Of Christian K?nig > >> Sent: Wednesday, March 15, 2017 17:29 > >> To: Zhou, David(ChunMing); Ayyappa Ch > >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > >> Yes, exactly that. > > > > (I'm not familiar with PCI too much.) > > Is there any restrict for PCI device? > > I'm concerning if any PCI couldn't support it on some motherboard. >=20 > It depends on the PCI root bridge.=A0 This patch set only implements supp= ort for > AMD root bridges.=A0 Intel and other vendors would need similar code. >=20 > Alex >=20 > > > >> > >> Christian. > >> > >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > >> > Does that means we don't need invisible vram later? > >> > > >> > David > >> > > >> > -----Original Message----- > >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> > Behalf Of Christian K?nig > >> > Sent: Wednesday, March 15, 2017 3:38 PM > >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; > >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there= . > >> > The CPU can access the full stolen VRAM directly on that hardware. > >> > > >> > As far as I know ASICs with support for this are Tonga, Fiji and all= Polaris > variants. > >> > > >> > Christian. > >> > > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > >> >> Is it possible on Carrizo asics? Or only supports on newer asics? > >> >> > >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian K=F6nig > >> >> <deathsimple@vodafone.de> wrote: > >> >>> From: Christian K=F6nig <christian.koenig@amd.com> > >> >>> > >> >>> Try to resize BAR0 to let CPU access all of VRAM. > >> >>> > >> >>> Signed-off-by: Christian K=F6nig <christian.koenig@amd.com> > >> >>> --- > >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/amdgpu.h=A0=A0=A0=A0=A0=A0=A0 = |=A0 1 + > >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > >> +++++++++++++++++++++++++++++ > >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c=A0=A0=A0=A0=A0 |=A0= 8 +++++--- > >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c=A0=A0=A0=A0=A0 |=A0= 8 +++++--- > >> >>>=A0=A0=A0 4 files changed, 40 insertions(+), 6 deletions(-) > >> >>> > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> index 3b81ded..905ded9 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > >> amdgpu_device *adev, struct ttm_tt *ttm, > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 struct ttm_mem_reg *mem); > >> >>>=A0=A0=A0 void amdgpu_vram_location(struct amdgpu_device *adev, str= uct > >> amdgpu_mc *mc, u64 base); > >> >>>=A0=A0=A0 void amdgpu_gtt_location(struct amdgpu_device *adev, stru= ct > >> >>> amdgpu_mc *mc); > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >> >>>=A0=A0=A0 void amdgpu_ttm_set_active_vram_size(struct amdgpu_device > >> >>> *adev, > >> u64 size); > >> >>>=A0=A0=A0 int amdgpu_ttm_init(struct amdgpu_device *adev); > >> >>>=A0=A0=A0 void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --= git > >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> index 118f4e6..92955fe 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct > >> >>> amdgpu_device > >> *adev, struct amdgpu_mc *mc) > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0= =A0=A0=A0=A0 mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >> >>>=A0=A0=A0 } > >> >>> > >> >>> +/** > >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 > >> >>> + * > >> >>> + * @adev: amdgpu_device pointer > >> >>> + * > >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. > >> >>> + */ > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { > >> >>> +=A0=A0=A0=A0=A0=A0 u32 size =3D max(ilog2(adev->mc.real_vram_size= - 1) + 1, 20) - 20; > >> >>> +=A0=A0=A0=A0=A0=A0 int r; > >> >>> + > >> >>> +=A0=A0=A0=A0=A0=A0 r =3D pci_resize_resource(adev->pdev, 0, size)= ; > >> >>> + > >> >>> +=A0=A0=A0=A0=A0=A0 if (r =3D=3D -ENOTSUPP) { > >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* The hardware don't = support the extension. */ > >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 return; > >> >>> + > >> >>> +=A0=A0=A0=A0=A0=A0 } else if (r =3D=3D -ENOSPC) { > >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 DRM_INFO("Not enoigh P= CI address space for a large BAR."); > >> >>> +=A0=A0=A0=A0=A0=A0 } else if (r) { > >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 DRM_ERROR("Problem res= izing BAR0 (%d).", r); > >> >>> +=A0=A0=A0=A0=A0=A0 } > >> >>> + > >> >>> +=A0=A0=A0=A0=A0=A0 /* Reinit the doorbell mapping, it is most lik= ely moved as well */ > >> >>> +=A0=A0=A0=A0=A0=A0 amdgpu_doorbell_fini(adev); > >> >>> +=A0=A0=A0=A0=A0=A0 BUG_ON(amdgpu_doorbell_init(adev)); > >> >>> +} > >> >>> + > >> >>>=A0=A0=A0 /* > >> >>>=A0=A0=A0=A0 * GPU helpers function. > >> >>>=A0=A0=A0=A0 */ > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> index dc9b6d6..36a7aa5 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break; > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.vram_width =3D numchan * ch= ansize; > >> >>> -=A0=A0=A0=A0=A0=A0 /* Could aper size report 0 ? */ > >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev= ->pdev, 0); > >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->= pdev, 0); > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* size in MB on si */ > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.mc_vram_size =3D RREG32(mmC= ONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.real_vram_size =3D RREG32(m= mCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> +=A0=A0=A0=A0=A0=A0 if (!(adev->flags & AMD_IS_APU)) > >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 amdgpu_resize_bar0(ade= v); > >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev= ->pdev, 0); > >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->= pdev, 0); > >> >>> + > >> >>>=A0=A0=A0 #ifdef CONFIG_X86_64 > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (adev->flags & AMD_IS_APU) { > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.ape= r_base =3D > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> index c087b00..7761ad3 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break; > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 } > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.vram_width =3D numchan * ch= ansize; > >> >>> -=A0=A0=A0=A0=A0=A0 /* Could aper size report 0 ? */ > >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev= ->pdev, 0); > >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->= pdev, 0); > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* size in MB on si */ > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.mc_vram_size =3D RREG32(mmC= ONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.real_vram_size =3D RREG32(m= mCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> +=A0=A0=A0=A0=A0=A0 if (!(adev->flags & AMD_IS_APU)) > >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 amdgpu_resize_bar0(ade= v); > >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev= ->pdev, 0); > >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->= pdev, 0); > >> >>> + > >> >>>=A0=A0=A0 #ifdef CONFIG_X86_64 > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (adev->flags & AMD_IS_APU) { > >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.ape= r_base =3D > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > >> >>> -- > >> >>> 2.7.4 > >> >>> > >> >>> _______________________________________________ > >> >>> dri-devel mailing list > >> >>> dri-devel@lists.freedesktop.org > >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > > >> > _______________________________________________ > >> > dri-devel mailing list > >> > dri-devel@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > _______________________________________________ > >> > amd-gfx mailing list > >> > amd-gfx@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > >> > >> > >> _______________________________________________ > >> dri-devel mailing list > >> dri-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx = ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-23 14:30 ` Sagalovitch, Serguei 0 siblings, 0 replies; 72+ messages in thread From: Sagalovitch, Serguei @ 2017-03-23 14:30 UTC (permalink / raw) To: Zhang, Jerry, Alex Deucher, Christian König Cc: Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx Christian, - Are we going to support resizing BAR when kernel modesetting is not enabled and we are running in console under VBIOS control (VESA/VGA)? - Should we restore PCI configuration if amdgpu will be unloaded? - In function amdgpu_resize_bar0(): If resizing for "max" size failed should we try other sizes? What do you think? Sincerely yours, Serguei Sagalovitch From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com> Sent: March 15, 2017 10:41 PM To: Alex Deucher Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access Thanks for your info. I see. Regards, Jerry (Junwei Zhang) Linux Base Graphics SRDC Software Development _____________________________________ > -----Original Message----- > From: Alex Deucher [mailto:alexdeucher@gmail.com] > Sent: Thursday, March 16, 2017 10:25 > To: Zhang, Jerry > Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux- > pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri- > devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; amd-gfx@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: > >> -----Original Message----- > >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> Behalf Of Christian K?nig > >> Sent: Wednesday, March 15, 2017 17:29 > >> To: Zhou, David(ChunMing); Ayyappa Ch > >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > >> Yes, exactly that. > > > > (I'm not familiar with PCI too much.) > > Is there any restrict for PCI device? > > I'm concerning if any PCI couldn't support it on some motherboard. > > It depends on the PCI root bridge. This patch set only implements support for > AMD root bridges. Intel and other vendors would need similar code. > > Alex > > > > >> > >> Christian. > >> > >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): > >> > Does that means we don't need invisible vram later? > >> > > >> > David > >> > > >> > -----Original Message----- > >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On > >> > Behalf Of Christian K?nig > >> > Sent: Wednesday, March 15, 2017 3:38 PM > >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> > >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; > >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org > >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > >> > > >> > Carizzo is an APU and resizing BARs isn't needed nor supported there. > >> > The CPU can access the full stolen VRAM directly on that hardware. > >> > > >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris > variants. > >> > > >> > Christian. > >> > > >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > >> >> Is it possible on Carrizo asics? Or only supports on newer asics? > >> >> > >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König > >> >> <deathsimple@vodafone.de> wrote: > >> >>> From: Christian König <christian.koenig@amd.com> > >> >>> > >> >>> Try to resize BAR0 to let CPU access all of VRAM. > >> >>> > >> >>> Signed-off-by: Christian König <christian.koenig@amd.com> > >> >>> --- > >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > >> +++++++++++++++++++++++++++++ > >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >> >>> 4 files changed, 40 insertions(+), 6 deletions(-) > >> >>> > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> index 3b81ded..905ded9 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > >> amdgpu_device *adev, struct ttm_tt *ttm, > >> >>> struct ttm_mem_reg *mem); > >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct > >> amdgpu_mc *mc, u64 base); > >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > >> >>> amdgpu_mc *mc); > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device > >> >>> *adev, > >> u64 size); > >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev); > >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> index 118f4e6..92955fe 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct > >> >>> amdgpu_device > >> *adev, struct amdgpu_mc *mc) > >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >> >>> } > >> >>> > >> >>> +/** > >> >>> + * amdgpu_resize_bar0 - try to resize BAR0 > >> >>> + * > >> >>> + * @adev: amdgpu_device pointer > >> >>> + * > >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible. > >> >>> + */ > >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { > >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >> >>> + int r; > >> >>> + > >> >>> + r = pci_resize_resource(adev->pdev, 0, size); > >> >>> + > >> >>> + if (r == -ENOTSUPP) { > >> >>> + /* The hardware don't support the extension. */ > >> >>> + return; > >> >>> + > >> >>> + } else if (r == -ENOSPC) { > >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >> >>> + } else if (r) { > >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >> >>> + } > >> >>> + > >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >> >>> + amdgpu_doorbell_fini(adev); > >> >>> + BUG_ON(amdgpu_doorbell_init(adev)); > >> >>> +} > >> >>> + > >> >>> /* > >> >>> * GPU helpers function. > >> >>> */ > >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> index dc9b6d6..36a7aa5 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>> break; > >> >>> } > >> >>> adev->mc.vram_width = numchan * chansize; > >> >>> - /* Could aper size report 0 ? */ > >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> /* size in MB on si */ > >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> + if (!(adev->flags & AMD_IS_APU)) > >> >>> + amdgpu_resize_bar0(adev); > >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> + > >> >>> #ifdef CONFIG_X86_64 > >> >>> if (adev->flags & AMD_IS_APU) { > >> >>> adev->mc.aper_base = > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git > >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> index c087b00..7761ad3 100644 > >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct > >> >>> amdgpu_device > >> *adev) > >> >>> break; > >> >>> } > >> >>> adev->mc.vram_width = numchan * chansize; > >> >>> - /* Could aper size report 0 ? */ > >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> /* size in MB on si */ > >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL * > >> 1024ULL; > >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > >> >>> 1024ULL > >> >>> * 1024ULL; > >> >>> > >> >>> + if (!(adev->flags & AMD_IS_APU)) > >> >>> + amdgpu_resize_bar0(adev); > >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> >>> + > >> >>> #ifdef CONFIG_X86_64 > >> >>> if (adev->flags & AMD_IS_APU) { > >> >>> adev->mc.aper_base = > >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; > >> >>> -- > >> >>> 2.7.4 > >> >>> > >> >>> _______________________________________________ > >> >>> dri-devel mailing list > >> >>> dri-devel@lists.freedesktop.org > >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > > >> > _______________________________________________ > >> > dri-devel mailing list > >> > dri-devel@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel > >> > _______________________________________________ > >> > amd-gfx mailing list > >> > amd-gfx@lists.freedesktop.org > >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > >> > >> > >> _______________________________________________ > >> dri-devel mailing list > >> dri-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > _______________________________________________ > > amd-gfx mailing list > > amd-gfx@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/amd-gfx _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-23 14:30 ` Sagalovitch, Serguei @ 2017-03-23 15:56 ` Christian König -1 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-23 15:56 UTC (permalink / raw) To: Sagalovitch, Serguei, Zhang, Jerry, Alex Deucher Cc: Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx > - Are we going to support resizing BAR when kernel > modesetting is not enabled and we are running in console > under VBIOS control (VESA/VGA)? No, initial I've tried to resize the PCI BAR during probing without the help of the driver at all. But the VESA/EFI/VBIOS don't seem to be able to handle addresses above 4GB for some reason. So the approach is to let the driver kick the VESA/EFI drivers out and then resize when we know that nobody is accessing the BAR. That's the only approach I've found without either blacklisting VESA/EFI drivers or crashing the system during the resize. > - Should we restore PCI configuration if amdgpu > will be unloaded? Yeah, thought about the as well. I'm just not sure how to do it. There is a lot of stuff we need to save and reset when the driver unloads for not much gain. > - In function amdgpu_resize_bar0(): > If resizing for "max" size failed should we try other > sizes? What do you think? Probably not worth it. If we get the BAR moved to a 64bit address we should have enough address space in almost all cases, so setting it to the maximum should succeed. But I think we could add another parameter to allow limiting the resized size for all corner cases and for testing. Regards, Christian. Am 23.03.2017 um 15:30 schrieb Sagalovitch, Serguei: > Christian, > > - Are we going to support resizing BAR when kernel > modesetting is not enabled and we are running in console > under VBIOS control (VESA/VGA)? > > - Should we restore PCI configuration if amdgpu > will be unloaded? > > - In function amdgpu_resize_bar0(): > If resizing for "max" size failed should we try other > sizes? What do you think? > > > Sincerely yours, > Serguei Sagalovitch > > > From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com> > Sent: March 15, 2017 10:41 PM > To: Alex Deucher > Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org > Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Thanks for your info. > I see. > > Regards, > Jerry (Junwei Zhang) > > Linux Base Graphics > SRDC Software Development > _____________________________________ > > >> -----Original Message----- >> From: Alex Deucher [mailto:alexdeucher@gmail.com] >> Sent: Thursday, March 16, 2017 10:25 >> To: Zhang, Jerry >> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux- >> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri- >> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> helgaas@kernel.org; amd-gfx@lists.freedesktop.org >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> >> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: >>>> -----Original Message----- >>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >>>> Behalf Of Christian K?nig >>>> Sent: Wednesday, March 15, 2017 17:29 >>>> To: Zhou, David(ChunMing); Ayyappa Ch >>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- >>>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org >>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >>>> >>>> Yes, exactly that. >>> (I'm not familiar with PCI too much.) >>> Is there any restrict for PCI device? >>> I'm concerning if any PCI couldn't support it on some motherboard. >> It depends on the PCI root bridge. This patch set only implements support for >> AMD root bridges. Intel and other vendors would need similar code. >> >> Alex >> >>>> Christian. >>>> >>>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): >>>>> Does that means we don't need invisible vram later? >>>>> >>>>> David >>>>> >>>>> -----Original Message----- >>>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >>>>> Behalf Of Christian K?nig >>>>> Sent: Wednesday, March 15, 2017 3:38 PM >>>>> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> >>>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; >>>>> amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >>>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org >>>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >>>>> >>>>> Carizzo is an APU and resizing BARs isn't needed nor supported there. >>>>> The CPU can access the full stolen VRAM directly on that hardware. >>>>> >>>>> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris >> variants. >>>>> Christian. >>>>> >>>>> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >>>>>> Is it possible on Carrizo asics? Or only supports on newer asics? >>>>>> >>>>>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >>>>>> <deathsimple@vodafone.de> wrote: >>>>>>> From: Christian König <christian.koenig@amd.com> >>>>>>> >>>>>>> Try to resize BAR0 to let CPU access all of VRAM. >>>>>>> >>>>>>> Signed-off-by: Christian König <christian.koenig@amd.com> >>>>>>> --- >>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>>> +++++++++++++++++++++++++++++ >>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>>>>>> 4 files changed, 40 insertions(+), 6 deletions(-) >>>>>>> >>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> index 3b81ded..905ded9 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>>> amdgpu_device *adev, struct ttm_tt *ttm, >>>>>>> struct ttm_mem_reg *mem); >>>>>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct >>>> amdgpu_mc *mc, u64 base); >>>>>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >>>>>>> amdgpu_mc *mc); >>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>>>>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device >>>>>>> *adev, >>>> u64 size); >>>>>>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>>>>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >>>>>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> index 118f4e6..92955fe 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct >>>>>>> amdgpu_device >>>> *adev, struct amdgpu_mc *mc) >>>>>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>>>>>> } >>>>>>> >>>>>>> +/** >>>>>>> + * amdgpu_resize_bar0 - try to resize BAR0 >>>>>>> + * >>>>>>> + * @adev: amdgpu_device pointer >>>>>>> + * >>>>>>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>>>>>> + */ >>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >>>>>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>>>>>> + int r; >>>>>>> + >>>>>>> + r = pci_resize_resource(adev->pdev, 0, size); >>>>>>> + >>>>>>> + if (r == -ENOTSUPP) { >>>>>>> + /* The hardware don't support the extension. */ >>>>>>> + return; >>>>>>> + >>>>>>> + } else if (r == -ENOSPC) { >>>>>>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >>>>>>> + } else if (r) { >>>>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>>>>>> + } >>>>>>> + >>>>>>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >>>>>>> + amdgpu_doorbell_fini(adev); >>>>>>> + BUG_ON(amdgpu_doorbell_init(adev)); >>>>>>> +} >>>>>>> + >>>>>>> /* >>>>>>> * GPU helpers function. >>>>>>> */ >>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> index dc9b6d6..36a7aa5 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct >>>>>>> amdgpu_device >>>> *adev) >>>>>>> break; >>>>>>> } >>>>>>> adev->mc.vram_width = numchan * chansize; >>>>>>> - /* Could aper size report 0 ? */ >>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> /* size in MB on si */ >>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL * >>>> 1024ULL; >>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL >>>>>>> * 1024ULL; >>>>>>> >>>>>>> + if (!(adev->flags & AMD_IS_APU)) >>>>>>> + amdgpu_resize_bar0(adev); >>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> + >>>>>>> #ifdef CONFIG_X86_64 >>>>>>> if (adev->flags & AMD_IS_APU) { >>>>>>> adev->mc.aper_base = >>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >>>>>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> index c087b00..7761ad3 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct >>>>>>> amdgpu_device >>>> *adev) >>>>>>> break; >>>>>>> } >>>>>>> adev->mc.vram_width = numchan * chansize; >>>>>>> - /* Could aper size report 0 ? */ >>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> /* size in MB on si */ >>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL * >>>> 1024ULL; >>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL >>>>>>> * 1024ULL; >>>>>>> >>>>>>> + if (!(adev->flags & AMD_IS_APU)) >>>>>>> + amdgpu_resize_bar0(adev); >>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> + >>>>>>> #ifdef CONFIG_X86_64 >>>>>>> if (adev->flags & AMD_IS_APU) { >>>>>>> adev->mc.aper_base = >>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >>>>>>> -- >>>>>>> 2.7.4 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> dri-devel mailing list >>>>>>> dri-devel@lists.freedesktop.org >>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>>>> _______________________________________________ >>>>> dri-devel mailing list >>>>> dri-devel@lists.freedesktop.org >>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>>>> _______________________________________________ >>>>> amd-gfx mailing list >>>>> amd-gfx@lists.freedesktop.org >>>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx >>>> >>>> _______________________________________________ >>>> dri-devel mailing list >>>> dri-devel@lists.freedesktop.org >>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>> _______________________________________________ >>> amd-gfx mailing list >>> amd-gfx@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-23 15:56 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-23 15:56 UTC (permalink / raw) To: Sagalovitch, Serguei, Zhang, Jerry, Alex Deucher Cc: Zhou, David(ChunMing), Ayyappa Ch, linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx > - Are we going to support resizing BAR when kernel > modesetting is not enabled and we are running in console > under VBIOS control (VESA/VGA)? No, initial I've tried to resize the PCI BAR during probing without the help of the driver at all. But the VESA/EFI/VBIOS don't seem to be able to handle addresses above 4GB for some reason. So the approach is to let the driver kick the VESA/EFI drivers out and then resize when we know that nobody is accessing the BAR. That's the only approach I've found without either blacklisting VESA/EFI drivers or crashing the system during the resize. > - Should we restore PCI configuration if amdgpu > will be unloaded? Yeah, thought about the as well. I'm just not sure how to do it. There is a lot of stuff we need to save and reset when the driver unloads for not much gain. > - In function amdgpu_resize_bar0(): > If resizing for "max" size failed should we try other > sizes? What do you think? Probably not worth it. If we get the BAR moved to a 64bit address we should have enough address space in almost all cases, so setting it to the maximum should succeed. But I think we could add another parameter to allow limiting the resized size for all corner cases and for testing. Regards, Christian. Am 23.03.2017 um 15:30 schrieb Sagalovitch, Serguei: > Christian, > > - Are we going to support resizing BAR when kernel > modesetting is not enabled and we are running in console > under VBIOS control (VESA/VGA)? > > - Should we restore PCI configuration if amdgpu > will be unloaded? > > - In function amdgpu_resize_bar0(): > If resizing for "max" size failed should we try other > sizes? What do you think? > > > Sincerely yours, > Serguei Sagalovitch > > > From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com> > Sent: March 15, 2017 10:41 PM > To: Alex Deucher > Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org > Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Thanks for your info. > I see. > > Regards, > Jerry (Junwei Zhang) > > Linux Base Graphics > SRDC Software Development > _____________________________________ > > >> -----Original Message----- >> From: Alex Deucher [mailto:alexdeucher@gmail.com] >> Sent: Thursday, March 16, 2017 10:25 >> To: Zhang, Jerry >> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux- >> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri- >> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >> helgaas@kernel.org; amd-gfx@lists.freedesktop.org >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >> >> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote: >>>> -----Original Message----- >>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >>>> Behalf Of Christian K?nig >>>> Sent: Wednesday, March 15, 2017 17:29 >>>> To: Zhou, David(ChunMing); Ayyappa Ch >>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- >>>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org >>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >>>> >>>> Yes, exactly that. >>> (I'm not familiar with PCI too much.) >>> Is there any restrict for PCI device? >>> I'm concerning if any PCI couldn't support it on some motherboard. >> It depends on the PCI root bridge. This patch set only implements support for >> AMD root bridges. Intel and other vendors would need similar code. >> >> Alex >> >>>> Christian. >>>> >>>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing): >>>>> Does that means we don't need invisible vram later? >>>>> >>>>> David >>>>> >>>>> -----Original Message----- >>>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On >>>>> Behalf Of Christian K?nig >>>>> Sent: Wednesday, March 15, 2017 3:38 PM >>>>> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com> >>>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; >>>>> amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; >>>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org >>>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access >>>>> >>>>> Carizzo is an APU and resizing BARs isn't needed nor supported there. >>>>> The CPU can access the full stolen VRAM directly on that hardware. >>>>> >>>>> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris >> variants. >>>>> Christian. >>>>> >>>>> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >>>>>> Is it possible on Carrizo asics? Or only supports on newer asics? >>>>>> >>>>>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >>>>>> <deathsimple@vodafone.de> wrote: >>>>>>> From: Christian König <christian.koenig@amd.com> >>>>>>> >>>>>>> Try to resize BAR0 to let CPU access all of VRAM. >>>>>>> >>>>>>> Signed-off-by: Christian König <christian.koenig@amd.com> >>>>>>> --- >>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>>> +++++++++++++++++++++++++++++ >>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>>>>>> 4 files changed, 40 insertions(+), 6 deletions(-) >>>>>>> >>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> index 3b81ded..905ded9 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>>>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>>> amdgpu_device *adev, struct ttm_tt *ttm, >>>>>>> struct ttm_mem_reg *mem); >>>>>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct >>>> amdgpu_mc *mc, u64 base); >>>>>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct >>>>>>> amdgpu_mc *mc); >>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>>>>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device >>>>>>> *adev, >>>> u64 size); >>>>>>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>>>>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git >>>>>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> index 118f4e6..92955fe 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>>>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct >>>>>>> amdgpu_device >>>> *adev, struct amdgpu_mc *mc) >>>>>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>>>>>> } >>>>>>> >>>>>>> +/** >>>>>>> + * amdgpu_resize_bar0 - try to resize BAR0 >>>>>>> + * >>>>>>> + * @adev: amdgpu_device pointer >>>>>>> + * >>>>>>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>>>>>> + */ >>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) { >>>>>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>>>>>> + int r; >>>>>>> + >>>>>>> + r = pci_resize_resource(adev->pdev, 0, size); >>>>>>> + >>>>>>> + if (r == -ENOTSUPP) { >>>>>>> + /* The hardware don't support the extension. */ >>>>>>> + return; >>>>>>> + >>>>>>> + } else if (r == -ENOSPC) { >>>>>>> + DRM_INFO("Not enoigh PCI address space for a large BAR."); >>>>>>> + } else if (r) { >>>>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>>>>>> + } >>>>>>> + >>>>>>> + /* Reinit the doorbell mapping, it is most likely moved as well */ >>>>>>> + amdgpu_doorbell_fini(adev); >>>>>>> + BUG_ON(amdgpu_doorbell_init(adev)); >>>>>>> +} >>>>>>> + >>>>>>> /* >>>>>>> * GPU helpers function. >>>>>>> */ >>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> index dc9b6d6..36a7aa5 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>>>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct >>>>>>> amdgpu_device >>>> *adev) >>>>>>> break; >>>>>>> } >>>>>>> adev->mc.vram_width = numchan * chansize; >>>>>>> - /* Could aper size report 0 ? */ >>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> /* size in MB on si */ >>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL * >>>> 1024ULL; >>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL >>>>>>> * 1024ULL; >>>>>>> >>>>>>> + if (!(adev->flags & AMD_IS_APU)) >>>>>>> + amdgpu_resize_bar0(adev); >>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> + >>>>>>> #ifdef CONFIG_X86_64 >>>>>>> if (adev->flags & AMD_IS_APU) { >>>>>>> adev->mc.aper_base = >>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git >>>>>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> index c087b00..7761ad3 100644 >>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>>>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct >>>>>>> amdgpu_device >>>> *adev) >>>>>>> break; >>>>>>> } >>>>>>> adev->mc.vram_width = numchan * chansize; >>>>>>> - /* Could aper size report 0 ? */ >>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> /* size in MB on si */ >>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL * >>>> 1024ULL; >>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * >>>>>>> 1024ULL >>>>>>> * 1024ULL; >>>>>>> >>>>>>> + if (!(adev->flags & AMD_IS_APU)) >>>>>>> + amdgpu_resize_bar0(adev); >>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>>>>> + >>>>>>> #ifdef CONFIG_X86_64 >>>>>>> if (adev->flags & AMD_IS_APU) { >>>>>>> adev->mc.aper_base = >>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; >>>>>>> -- >>>>>>> 2.7.4 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> dri-devel mailing list >>>>>>> dri-devel@lists.freedesktop.org >>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>>>> _______________________________________________ >>>>> dri-devel mailing list >>>>> dri-devel@lists.freedesktop.org >>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>>>> _______________________________________________ >>>>> amd-gfx mailing list >>>>> amd-gfx@lists.freedesktop.org >>>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx >>>> >>>> _______________________________________________ >>>> dri-devel mailing list >>>> dri-devel@lists.freedesktop.org >>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >>> _______________________________________________ >>> amd-gfx mailing list >>> amd-gfx@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-15 7:37 ` Christian König (?) @ 2017-03-15 10:42 ` Ayyappa Ch -1 siblings, 0 replies; 72+ messages in thread From: Ayyappa Ch @ 2017-03-15 10:42 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel It also needs any support from VBIOS side ? I mean PCIe large bar support? Thanks, Ayyappa. On Wed, Mar 15, 2017 at 1:07 PM, Christian König <deathsimple@vodafone.de> wrote: > Carizzo is an APU and resizing BARs isn't needed nor supported there. The > CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris > variants. > > Christian. > > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> <deathsimple@vodafone.de> wrote: >>> >>> From: Christian König <christian.koenig@amd.com> >>> >>> Try to resize BAR0 to let CPU access all of VRAM. >>> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>> +++++++++++++++++++++++++++++ >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>> 4 files changed, 40 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> index 3b81ded..905ded9 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>> amdgpu_device *adev, struct ttm_tt *ttm, >>> struct ttm_mem_reg *mem); >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc >>> *mc, u64 base); >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc >>> *mc); >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 >>> size); >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> index 118f4e6..92955fe 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, >>> struct amdgpu_mc *mc) >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>> } >>> >>> +/** >>> + * amdgpu_resize_bar0 - try to resize BAR0 >>> + * >>> + * @adev: amdgpu_device pointer >>> + * >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>> + */ >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >>> +{ >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>> + int r; >>> + >>> + r = pci_resize_resource(adev->pdev, 0, size); >>> + >>> + if (r == -ENOTSUPP) { >>> + /* The hardware don't support the extension. */ >>> + return; >>> + >>> + } else if (r == -ENOSPC) { >>> + DRM_INFO("Not enoigh PCI address space for a large >>> BAR."); >>> + } else if (r) { >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>> + } >>> + >>> + /* Reinit the doorbell mapping, it is most likely moved as well >>> */ >>> + amdgpu_doorbell_fini(adev); >>> + BUG_ON(amdgpu_doorbell_init(adev)); >>> +} >>> + >>> /* >>> * GPU helpers function. >>> */ >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> index dc9b6d6..36a7aa5 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >>> *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>> 22; >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> index c087b00..7761ad3 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >>> *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>> 22; >>> -- >>> 2.7.4 >>> >>> _______________________________________________ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 10:42 ` Ayyappa Ch 0 siblings, 0 replies; 72+ messages in thread From: Ayyappa Ch @ 2017-03-15 10:42 UTC (permalink / raw) To: Christian König Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel It also needs any support from VBIOS side ? I mean PCIe large bar support? Thanks, Ayyappa. On Wed, Mar 15, 2017 at 1:07 PM, Christian K=C3=B6nig <deathsimple@vodafone.de> wrote: > Carizzo is an APU and resizing BARs isn't needed nor supported there. The > CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all Pola= ris > variants. > > Christian. > > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian K=C3=B6nig >> <deathsimple@vodafone.de> wrote: >>> >>> From: Christian K=C3=B6nig <christian.koenig@amd.com> >>> >>> Try to resize BAR0 to let CPU access all of VRAM. >>> >>> Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com> >>> --- >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>> +++++++++++++++++++++++++++++ >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>> 4 files changed, 40 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> index 3b81ded..905ded9 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>> amdgpu_device *adev, struct ttm_tt *ttm, >>> struct ttm_mem_reg *mem); >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_m= c >>> *mc, u64 base); >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc >>> *mc); >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 >>> size); >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> index 118f4e6..92955fe 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *ade= v, >>> struct amdgpu_mc *mc) >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end= ); >>> } >>> >>> +/** >>> + * amdgpu_resize_bar0 - try to resize BAR0 >>> + * >>> + * @adev: amdgpu_device pointer >>> + * >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>> + */ >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >>> +{ >>> + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - = 20; >>> + int r; >>> + >>> + r =3D pci_resize_resource(adev->pdev, 0, size); >>> + >>> + if (r =3D=3D -ENOTSUPP) { >>> + /* The hardware don't support the extension. */ >>> + return; >>> + >>> + } else if (r =3D=3D -ENOSPC) { >>> + DRM_INFO("Not enoigh PCI address space for a large >>> BAR."); >>> + } else if (r) { >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>> + } >>> + >>> + /* Reinit the doorbell mapping, it is most likely moved as well >>> */ >>> + amdgpu_doorbell_fini(adev); >>> + BUG_ON(amdgpu_doorbell_init(adev)); >>> +} >>> + >>> /* >>> * GPU helpers function. >>> */ >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> index dc9b6d6..36a7aa5 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >>> *adev) >>> break; >>> } >>> adev->mc.vram_width =3D numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL= * >>> 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)= ) << >>> 22; >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> index c087b00..7761ad3 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >>> *adev) >>> break; >>> } >>> adev->mc.vram_width =3D numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL= * >>> 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)= ) << >>> 22; >>> -- >>> 2.7.4 >>> >>> _______________________________________________ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 10:42 ` Ayyappa Ch 0 siblings, 0 replies; 72+ messages in thread From: Ayyappa Ch @ 2017-03-15 10:42 UTC (permalink / raw) To: Christian König Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel It also needs any support from VBIOS side ? I mean PCIe large bar support? Thanks, Ayyappa. On Wed, Mar 15, 2017 at 1:07 PM, Christian König <deathsimple@vodafone.de> wrote: > Carizzo is an APU and resizing BARs isn't needed nor supported there. The > CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris > variants. > > Christian. > > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >> >> Is it possible on Carrizo asics? Or only supports on newer asics? >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >> <deathsimple@vodafone.de> wrote: >>> >>> From: Christian König <christian.koenig@amd.com> >>> >>> Try to resize BAR0 to let CPU access all of VRAM. >>> >>> Signed-off-by: Christian König <christian.koenig@amd.com> >>> --- >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>> +++++++++++++++++++++++++++++ >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>> 4 files changed, 40 insertions(+), 6 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> index 3b81ded..905ded9 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>> amdgpu_device *adev, struct ttm_tt *ttm, >>> struct ttm_mem_reg *mem); >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc >>> *mc, u64 base); >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc >>> *mc); >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 >>> size); >>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> index 118f4e6..92955fe 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, >>> struct amdgpu_mc *mc) >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>> } >>> >>> +/** >>> + * amdgpu_resize_bar0 - try to resize BAR0 >>> + * >>> + * @adev: amdgpu_device pointer >>> + * >>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>> + */ >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >>> +{ >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>> + int r; >>> + >>> + r = pci_resize_resource(adev->pdev, 0, size); >>> + >>> + if (r == -ENOTSUPP) { >>> + /* The hardware don't support the extension. */ >>> + return; >>> + >>> + } else if (r == -ENOSPC) { >>> + DRM_INFO("Not enoigh PCI address space for a large >>> BAR."); >>> + } else if (r) { >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>> + } >>> + >>> + /* Reinit the doorbell mapping, it is most likely moved as well >>> */ >>> + amdgpu_doorbell_fini(adev); >>> + BUG_ON(amdgpu_doorbell_init(adev)); >>> +} >>> + >>> /* >>> * GPU helpers function. >>> */ >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> index dc9b6d6..36a7aa5 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >>> *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>> 22; >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> index c087b00..7761ad3 100644 >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >>> *adev) >>> break; >>> } >>> adev->mc.vram_width = numchan * chansize; >>> - /* Could aper size report 0 ? */ >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> /* size in MB on si */ >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>> 1024ULL; >>> >>> + if (!(adev->flags & AMD_IS_APU)) >>> + amdgpu_resize_bar0(adev); >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>> + >>> #ifdef CONFIG_X86_64 >>> if (adev->flags & AMD_IS_APU) { >>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>> 22; >>> -- >>> 2.7.4 >>> >>> _______________________________________________ >>> dri-devel mailing list >>> dri-devel@lists.freedesktop.org >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-15 10:42 ` Ayyappa Ch @ 2017-03-15 11:03 ` Christian König -1 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 11:03 UTC (permalink / raw) To: Ayyappa Ch Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel No, we resize the BAR on the fly during driver load without help from the BIOS or VBIOS. Christian. Am 15.03.2017 um 11:42 schrieb Ayyappa Ch: > It also needs any support from VBIOS side ? I mean PCIe large bar support? > > Thanks, > Ayyappa. > > On Wed, Mar 15, 2017 at 1:07 PM, Christian König > <deathsimple@vodafone.de> wrote: >> Carizzo is an APU and resizing BARs isn't needed nor supported there. The >> CPU can access the full stolen VRAM directly on that hardware. >> >> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris >> variants. >> >> Christian. >> >> >> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >>> Is it possible on Carrizo asics? Or only supports on newer asics? >>> >>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >>> <deathsimple@vodafone.de> wrote: >>>> From: Christian König <christian.koenig@amd.com> >>>> >>>> Try to resize BAR0 to let CPU access all of VRAM. >>>> >>>> Signed-off-by: Christian König <christian.koenig@amd.com> >>>> --- >>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>>> +++++++++++++++++++++++++++++ >>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>>> 4 files changed, 40 insertions(+), 6 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> index 3b81ded..905ded9 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>>> amdgpu_device *adev, struct ttm_tt *ttm, >>>> struct ttm_mem_reg *mem); >>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc >>>> *mc, u64 base); >>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc >>>> *mc); >>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 >>>> size); >>>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> index 118f4e6..92955fe 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, >>>> struct amdgpu_mc *mc) >>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>>> } >>>> >>>> +/** >>>> + * amdgpu_resize_bar0 - try to resize BAR0 >>>> + * >>>> + * @adev: amdgpu_device pointer >>>> + * >>>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>>> + */ >>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >>>> +{ >>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>>> + int r; >>>> + >>>> + r = pci_resize_resource(adev->pdev, 0, size); >>>> + >>>> + if (r == -ENOTSUPP) { >>>> + /* The hardware don't support the extension. */ >>>> + return; >>>> + >>>> + } else if (r == -ENOSPC) { >>>> + DRM_INFO("Not enoigh PCI address space for a large >>>> BAR."); >>>> + } else if (r) { >>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>>> + } >>>> + >>>> + /* Reinit the doorbell mapping, it is most likely moved as well >>>> */ >>>> + amdgpu_doorbell_fini(adev); >>>> + BUG_ON(amdgpu_doorbell_init(adev)); >>>> +} >>>> + >>>> /* >>>> * GPU helpers function. >>>> */ >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> index dc9b6d6..36a7aa5 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >>>> *adev) >>>> break; >>>> } >>>> adev->mc.vram_width = numchan * chansize; >>>> - /* Could aper size report 0 ? */ >>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> /* size in MB on si */ >>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> >>>> + if (!(adev->flags & AMD_IS_APU)) >>>> + amdgpu_resize_bar0(adev); >>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> + >>>> #ifdef CONFIG_X86_64 >>>> if (adev->flags & AMD_IS_APU) { >>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>>> 22; >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> index c087b00..7761ad3 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >>>> *adev) >>>> break; >>>> } >>>> adev->mc.vram_width = numchan * chansize; >>>> - /* Could aper size report 0 ? */ >>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> /* size in MB on si */ >>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> >>>> + if (!(adev->flags & AMD_IS_APU)) >>>> + amdgpu_resize_bar0(adev); >>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> + >>>> #ifdef CONFIG_X86_64 >>>> if (adev->flags & AMD_IS_APU) { >>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>>> 22; >>>> -- >>>> 2.7.4 >>>> >>>> _______________________________________________ >>>> dri-devel mailing list >>>> dri-devel@lists.freedesktop.org >>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >> >> ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 11:03 ` Christian König 0 siblings, 0 replies; 72+ messages in thread From: Christian König @ 2017-03-15 11:03 UTC (permalink / raw) To: Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel No, we resize the BAR on the fly during driver load without help from the BIOS or VBIOS. Christian. Am 15.03.2017 um 11:42 schrieb Ayyappa Ch: > It also needs any support from VBIOS side ? I mean PCIe large bar support? > > Thanks, > Ayyappa. > > On Wed, Mar 15, 2017 at 1:07 PM, Christian König > <deathsimple@vodafone.de> wrote: >> Carizzo is an APU and resizing BARs isn't needed nor supported there. The >> CPU can access the full stolen VRAM directly on that hardware. >> >> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris >> variants. >> >> Christian. >> >> >> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: >>> Is it possible on Carrizo asics? Or only supports on newer asics? >>> >>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König >>> <deathsimple@vodafone.de> wrote: >>>> From: Christian König <christian.koenig@amd.com> >>>> >>>> Try to resize BAR0 to let CPU access all of VRAM. >>>> >>>> Signed-off-by: Christian König <christian.koenig@amd.com> >>>> --- >>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + >>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 >>>> +++++++++++++++++++++++++++++ >>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- >>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- >>>> 4 files changed, 40 insertions(+), 6 deletions(-) >>>> >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> index 3b81ded..905ded9 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h >>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct >>>> amdgpu_device *adev, struct ttm_tt *ttm, >>>> struct ttm_mem_reg *mem); >>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc >>>> *mc, u64 base); >>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc >>>> *mc); >>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev); >>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 >>>> size); >>>> int amdgpu_ttm_init(struct amdgpu_device *adev); >>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> index 118f4e6..92955fe 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c >>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, >>>> struct amdgpu_mc *mc) >>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); >>>> } >>>> >>>> +/** >>>> + * amdgpu_resize_bar0 - try to resize BAR0 >>>> + * >>>> + * @adev: amdgpu_device pointer >>>> + * >>>> + * Try to resize BAR0 to make all VRAM CPU accessible. >>>> + */ >>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) >>>> +{ >>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; >>>> + int r; >>>> + >>>> + r = pci_resize_resource(adev->pdev, 0, size); >>>> + >>>> + if (r == -ENOTSUPP) { >>>> + /* The hardware don't support the extension. */ >>>> + return; >>>> + >>>> + } else if (r == -ENOSPC) { >>>> + DRM_INFO("Not enoigh PCI address space for a large >>>> BAR."); >>>> + } else if (r) { >>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r); >>>> + } >>>> + >>>> + /* Reinit the doorbell mapping, it is most likely moved as well >>>> */ >>>> + amdgpu_doorbell_fini(adev); >>>> + BUG_ON(amdgpu_doorbell_init(adev)); >>>> +} >>>> + >>>> /* >>>> * GPU helpers function. >>>> */ >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> index dc9b6d6..36a7aa5 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c >>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device >>>> *adev) >>>> break; >>>> } >>>> adev->mc.vram_width = numchan * chansize; >>>> - /* Could aper size report 0 ? */ >>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> /* size in MB on si */ >>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> >>>> + if (!(adev->flags & AMD_IS_APU)) >>>> + amdgpu_resize_bar0(adev); >>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> + >>>> #ifdef CONFIG_X86_64 >>>> if (adev->flags & AMD_IS_APU) { >>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>>> 22; >>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> index c087b00..7761ad3 100644 >>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c >>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device >>>> *adev) >>>> break; >>>> } >>>> adev->mc.vram_width = numchan * chansize; >>>> - /* Could aper size report 0 ? */ >>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> /* size in MB on si */ >>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * >>>> 1024ULL; >>>> >>>> + if (!(adev->flags & AMD_IS_APU)) >>>> + amdgpu_resize_bar0(adev); >>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); >>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); >>>> + >>>> #ifdef CONFIG_X86_64 >>>> if (adev->flags & AMD_IS_APU) { >>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << >>>> 22; >>>> -- >>>> 2.7.4 >>>> >>>> _______________________________________________ >>>> dri-devel mailing list >>>> dri-devel@lists.freedesktop.org >>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel >> >> _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 16:08 ` Deucher, Alexander 0 siblings, 0 replies; 72+ messages in thread From: Deucher, Alexander @ 2017-03-15 16:08 UTC (permalink / raw) To: 'Christian König', Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel > -----Original Message----- > From: amd-gfx [mailto:amd-gfx-bounces@lists.freedesktop.org] On Behalf > Of Christian König > Sent: Wednesday, March 15, 2017 3:38 AM > To: Ayyappa Ch > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > The CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all > Polaris variants. I think resizable BARs are supported as far back as evergreen or NI. Alex > > Christian. > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > > Is it possible on Carrizo asics? Or only supports on newer asics? > > > > On Mon, Mar 13, 2017 at 6:11 PM, Christian König > > <deathsimple@vodafone.de> wrote: > >> From: Christian König <christian.koenig@amd.com> > >> > >> Try to resize BAR0 to let CPU access all of VRAM. > >> > >> Signed-off-by: Christian König <christian.koenig@amd.com> > >> --- > >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > +++++++++++++++++++++++++++++ > >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >> 4 files changed, 40 insertions(+), 6 deletions(-) > >> > >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> index 3b81ded..905ded9 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > amdgpu_device *adev, struct ttm_tt *ttm, > >> struct ttm_mem_reg *mem); > >> void amdgpu_vram_location(struct amdgpu_device *adev, struct > amdgpu_mc *mc, u64 base); > >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > amdgpu_mc *mc); > >> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, > u64 size); > >> int amdgpu_ttm_init(struct amdgpu_device *adev); > >> void amdgpu_ttm_fini(struct amdgpu_device *adev); > >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> index 118f4e6..92955fe 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device > *adev, struct amdgpu_mc *mc) > >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >> } > >> > >> +/** > >> + * amdgpu_resize_bar0 - try to resize BAR0 > >> + * > >> + * @adev: amdgpu_device pointer > >> + * > >> + * Try to resize BAR0 to make all VRAM CPU accessible. > >> + */ > >> +void amdgpu_resize_bar0(struct amdgpu_device *adev) > >> +{ > >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >> + int r; > >> + > >> + r = pci_resize_resource(adev->pdev, 0, size); > >> + > >> + if (r == -ENOTSUPP) { > >> + /* The hardware don't support the extension. */ > >> + return; > >> + > >> + } else if (r == -ENOSPC) { > >> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >> + } else if (r) { > >> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >> + } > >> + > >> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >> + amdgpu_doorbell_fini(adev); > >> + BUG_ON(amdgpu_doorbell_init(adev)); > >> +} > >> + > >> /* > >> * GPU helpers function. > >> */ > >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> index dc9b6d6..36a7aa5 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct > amdgpu_device *adev) > >> break; > >> } > >> adev->mc.vram_width = numchan * chansize; > >> - /* Could aper size report 0 ? */ > >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> /* size in MB on si */ > >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> > >> + if (!(adev->flags & AMD_IS_APU)) > >> + amdgpu_resize_bar0(adev); > >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> + > >> #ifdef CONFIG_X86_64 > >> if (adev->flags & AMD_IS_APU) { > >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) > << 22; > >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> index c087b00..7761ad3 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct > amdgpu_device *adev) > >> break; > >> } > >> adev->mc.vram_width = numchan * chansize; > >> - /* Could aper size report 0 ? */ > >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> /* size in MB on si */ > >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> > >> + if (!(adev->flags & AMD_IS_APU)) > >> + amdgpu_resize_bar0(adev); > >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> + > >> #ifdef CONFIG_X86_64 > >> if (adev->flags & AMD_IS_APU) { > >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) > << 22; > >> -- > >> 2.7.4 > >> > >> _______________________________________________ > >> dri-devel mailing list > >> dri-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 16:08 ` Deucher, Alexander 0 siblings, 0 replies; 72+ messages in thread From: Deucher, Alexander @ 2017-03-15 16:08 UTC (permalink / raw) To: 'Christian König', Ayyappa Ch Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas, dri-devel PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBhbWQtZ2Z4IFttYWlsdG86YW1k LWdmeC1ib3VuY2VzQGxpc3RzLmZyZWVkZXNrdG9wLm9yZ10gT24gQmVoYWxmDQo+IE9mIENocmlz dGlhbiBLw7ZuaWcNCj4gU2VudDogV2VkbmVzZGF5LCBNYXJjaCAxNSwgMjAxNyAzOjM4IEFNDQo+ IFRvOiBBeXlhcHBhIENoDQo+IENjOiBsaW51eC1wY2lAdmdlci5rZXJuZWwub3JnOyBsaW51eC1r ZXJuZWxAdmdlci5rZXJuZWwub3JnOyBhbWQtDQo+IGdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7 IHBsYXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwub3JnOw0KPiBoZWxnYWFzQGtlcm5lbC5v cmc7IGRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcNCj4gU3ViamVjdDogUmU6IFtQQVRD SCA0LzRdIGRybS9hbWRncHU6IHJlc2l6ZSBWUkFNIEJBUiBmb3IgQ1BVIGFjY2Vzcw0KPiANCj4g Q2FyaXp6byBpcyBhbiBBUFUgYW5kIHJlc2l6aW5nIEJBUnMgaXNuJ3QgbmVlZGVkIG5vciBzdXBw b3J0ZWQgdGhlcmUuDQo+IFRoZSBDUFUgY2FuIGFjY2VzcyB0aGUgZnVsbCBzdG9sZW4gVlJBTSBk aXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KPiANCj4gQXMgZmFyIGFzIEkga25vdyBBU0lDcyB3 aXRoIHN1cHBvcnQgZm9yIHRoaXMgYXJlIFRvbmdhLCBGaWppIGFuZCBhbGwNCj4gUG9sYXJpcyB2 YXJpYW50cy4NCg0KSSB0aGluayByZXNpemFibGUgQkFScyBhcmUgc3VwcG9ydGVkIGFzIGZhciBi YWNrIGFzIGV2ZXJncmVlbiBvciBOSS4NCg0KQWxleA0KDQo+IA0KPiBDaHJpc3RpYW4uDQo+IA0K PiBBbSAxNS4wMy4yMDE3IHVtIDA4OjIzIHNjaHJpZWIgQXl5YXBwYSBDaDoNCj4gPiBJcyBpdCBw b3NzaWJsZSBvbiBDYXJyaXpvIGFzaWNzPyBPciBvbmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNz Pw0KPiA+DQo+ID4gT24gTW9uLCBNYXIgMTMsIDIwMTcgYXQgNjoxMSBQTSwgQ2hyaXN0aWFuIEvD tm5pZw0KPiA+IDxkZWF0aHNpbXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6DQo+ID4+IEZyb206IENo cmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4NCj4gPj4NCj4gPj4gVHJ5 IHRvIHJlc2l6ZSBCQVIwIHRvIGxldCBDUFUgYWNjZXNzIGFsbCBvZiBWUkFNLg0KPiA+Pg0KPiA+ PiBTaWduZWQtb2ZmLWJ5OiBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5j b20+DQo+ID4+IC0tLQ0KPiA+PiAgIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdS5o ICAgICAgICB8ICAxICsNCj4gPj4gICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVf ZGV2aWNlLmMgfCAyOQ0KPiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPiA+PiAgIGRy aXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMgICAgICB8ICA4ICsrKysrLS0tDQo+ ID4+ICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYyAgICAgIHwgIDggKysr KystLS0NCj4gPj4gICA0IGZpbGVzIGNoYW5nZWQsIDQwIGluc2VydGlvbnMoKyksIDYgZGVsZXRp b25zKC0pDQo+ID4+DQo+ID4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdw dS9hbWRncHUuaA0KPiBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+ IGluZGV4IDNiODFkZWQuLjkwNWRlZDkgMTAwNjQ0DQo+ID4+IC0tLSBhL2RyaXZlcnMvZ3B1L2Ry bS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k Z3B1L2FtZGdwdS5oDQo+ID4+IEBAIC0xNzE5LDYgKzE3MTksNyBAQCB1aW50NjRfdCBhbWRncHVf dHRtX3R0X3B0ZV9mbGFncyhzdHJ1Y3QNCj4gYW1kZ3B1X2RldmljZSAqYWRldiwgc3RydWN0IHR0 bV90dCAqdHRtLA0KPiA+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0 IHR0bV9tZW1fcmVnICptZW0pOw0KPiA+PiAgIHZvaWQgYW1kZ3B1X3ZyYW1fbG9jYXRpb24oc3Ry dWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiBhbWRncHVfbWMgKm1jLCB1NjQgYmFz ZSk7DQo+ID4+ICAgdm9pZCBhbWRncHVfZ3R0X2xvY2F0aW9uKHN0cnVjdCBhbWRncHVfZGV2aWNl ICphZGV2LCBzdHJ1Y3QNCj4gYW1kZ3B1X21jICptYyk7DQo+ID4+ICt2b2lkIGFtZGdwdV9yZXNp emVfYmFyMChzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldik7DQo+ID4+ICAgdm9pZCBhbWRncHVf dHRtX3NldF9hY3RpdmVfdnJhbV9zaXplKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LA0KPiB1 NjQgc2l6ZSk7DQo+ID4+ICAgaW50IGFtZGdwdV90dG1faW5pdChzdHJ1Y3QgYW1kZ3B1X2Rldmlj ZSAqYWRldik7DQo+ID4+ICAgdm9pZCBhbWRncHVfdHRtX2Zpbmkoc3RydWN0IGFtZGdwdV9kZXZp Y2UgKmFkZXYpOw0KPiA+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv YW1kZ3B1X2RldmljZS5jDQo+IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2Rl dmljZS5jDQo+ID4+IGluZGV4IDExOGY0ZTYuLjkyOTU1ZmUgMTAwNjQ0DQo+ID4+IC0tLSBhL2Ry aXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9kZXZpY2UuYw0KPiA+PiArKysgYi9kcml2 ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMNCj4gPj4gQEAgLTY5Miw2ICs2 OTIsMzUgQEAgdm9pZCBhbWRncHVfZ3R0X2xvY2F0aW9uKHN0cnVjdCBhbWRncHVfZGV2aWNlDQo+ ICphZGV2LCBzdHJ1Y3QgYW1kZ3B1X21jICptYykNCj4gPj4gICAgICAgICAgICAgICAgICAgICAg ICAgIG1jLT5ndHRfc2l6ZSA+PiAyMCwgbWMtPmd0dF9zdGFydCwgbWMtPmd0dF9lbmQpOw0KPiA+ PiAgIH0NCj4gPj4NCj4gPj4gKy8qKg0KPiA+PiArICogYW1kZ3B1X3Jlc2l6ZV9iYXIwIC0gdHJ5 IHRvIHJlc2l6ZSBCQVIwDQo+ID4+ICsgKg0KPiA+PiArICogQGFkZXY6IGFtZGdwdV9kZXZpY2Ug cG9pbnRlcg0KPiA+PiArICoNCj4gPj4gKyAqIFRyeSB0byByZXNpemUgQkFSMCB0byBtYWtlIGFs bCBWUkFNIENQVSBhY2Nlc3NpYmxlLg0KPiA+PiArICovDQo+ID4+ICt2b2lkIGFtZGdwdV9yZXNp emVfYmFyMChzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldikNCj4gPj4gK3sNCj4gPj4gKyAgICAg ICB1MzIgc2l6ZSA9IG1heChpbG9nMihhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSAtIDEpICsgMSwg MjApIC0gMjA7DQo+ID4+ICsgICAgICAgaW50IHI7DQo+ID4+ICsNCj4gPj4gKyAgICAgICByID0g cGNpX3Jlc2l6ZV9yZXNvdXJjZShhZGV2LT5wZGV2LCAwLCBzaXplKTsNCj4gPj4gKw0KPiA+PiAr ICAgICAgIGlmIChyID09IC1FTk9UU1VQUCkgew0KPiA+PiArICAgICAgICAgICAgICAgLyogVGhl IGhhcmR3YXJlIGRvbid0IHN1cHBvcnQgdGhlIGV4dGVuc2lvbi4gKi8NCj4gPj4gKyAgICAgICAg ICAgICAgIHJldHVybjsNCj4gPj4gKw0KPiA+PiArICAgICAgIH0gZWxzZSBpZiAociA9PSAtRU5P U1BDKSB7DQo+ID4+ICsgICAgICAgICAgICAgICBEUk1fSU5GTygiTm90IGVub2lnaCBQQ0kgYWRk cmVzcyBzcGFjZSBmb3IgYSBsYXJnZSBCQVIuIik7DQo+ID4+ICsgICAgICAgfSBlbHNlIGlmIChy KSB7DQo+ID4+ICsgICAgICAgICAgICAgICBEUk1fRVJST1IoIlByb2JsZW0gcmVzaXppbmcgQkFS MCAoJWQpLiIsIHIpOw0KPiA+PiArICAgICAgIH0NCj4gPj4gKw0KPiA+PiArICAgICAgIC8qIFJl aW5pdCB0aGUgZG9vcmJlbGwgbWFwcGluZywgaXQgaXMgbW9zdCBsaWtlbHkgbW92ZWQgYXMgd2Vs bCAqLw0KPiA+PiArICAgICAgIGFtZGdwdV9kb29yYmVsbF9maW5pKGFkZXYpOw0KPiA+PiArICAg ICAgIEJVR19PTihhbWRncHVfZG9vcmJlbGxfaW5pdChhZGV2KSk7DQo+ID4+ICt9DQo+ID4+ICsN Cj4gPj4gICAvKg0KPiA+PiAgICAqIEdQVSBoZWxwZXJzIGZ1bmN0aW9uLg0KPiA+PiAgICAqLw0K PiA+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0K PiBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMNCj4gPj4gaW5kZXggZGM5 YjZkNi4uMzZhN2FhNSAxMDA2NDQNCj4gPj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRn cHUvZ21jX3Y3XzAuYw0KPiA+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNf djdfMC5jDQo+ID4+IEBAIC0zNjcsMTMgKzM2NywxNSBAQCBzdGF0aWMgaW50IGdtY192N18wX21j X2luaXQoc3RydWN0DQo+IGFtZGdwdV9kZXZpY2UgKmFkZXYpDQo+ID4+ICAgICAgICAgICAgICAg ICAgYnJlYWs7DQo+ID4+ICAgICAgICAgIH0NCj4gPj4gICAgICAgICAgYWRldi0+bWMudnJhbV93 aWR0aCA9IG51bWNoYW4gKiBjaGFuc2l6ZTsNCj4gPj4gLSAgICAgICAvKiBDb3VsZCBhcGVyIHNp emUgcmVwb3J0IDAgPyAqLw0KPiA+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9y ZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsNCj4gPj4gLSAgICAgICBhZGV2LT5tYy5hcGVy X3NpemUgPSBwY2lfcmVzb3VyY2VfbGVuKGFkZXYtPnBkZXYsIDApOw0KPiA+PiAgICAgICAgICAv KiBzaXplIGluIE1CIG9uIHNpICovDQo+ID4+ICAgICAgICAgIGFkZXYtPm1jLm1jX3ZyYW1fc2l6 ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqDQo+IDEwMjRVTEwgKiAxMDI0VUxMOw0KPiA+ PiAgICAgICAgICBhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1T SVpFKSAqDQo+IDEwMjRVTEwgKiAxMDI0VUxMOw0KPiA+Pg0KPiA+PiArICAgICAgIGlmICghKGFk ZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkpDQo+ID4+ICsgICAgICAgICAgICAgICBhbWRncHVfcmVz aXplX2JhcjAoYWRldik7DQo+ID4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jl c291cmNlX3N0YXJ0KGFkZXYtPnBkZXYsIDApOw0KPiA+PiArICAgICAgIGFkZXYtPm1jLmFwZXJf c2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+ICsNCj4gPj4gICAj aWZkZWYgQ09ORklHX1g4Nl82NA0KPiA+PiAgICAgICAgICBpZiAoYWRldi0+ZmxhZ3MgJiBBTURf SVNfQVBVKSB7DQo+ID4+ICAgICAgICAgICAgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gKCh1 NjQpUlJFRzMyKG1tTUNfVk1fRkJfT0ZGU0VUKSkNCj4gPDwgMjI7DQo+ID4+IGRpZmYgLS1naXQg YS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+IGIvZHJpdmVycy9ncHUv ZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYw0KPiA+PiBpbmRleCBjMDg3YjAwLi43NzYxYWQzIDEw MDY0NA0KPiA+PiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+ ID4+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMNCj4gPj4gQEAg LTQ1OSwxMyArNDU5LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y4XzBfbWNfaW5pdChzdHJ1Y3QNCj4g YW1kZ3B1X2RldmljZSAqYWRldikNCj4gPj4gICAgICAgICAgICAgICAgICBicmVhazsNCj4gPj4g ICAgICAgICAgfQ0KPiA+PiAgICAgICAgICBhZGV2LT5tYy52cmFtX3dpZHRoID0gbnVtY2hhbiAq IGNoYW5zaXplOw0KPiA+PiAtICAgICAgIC8qIENvdWxkIGFwZXIgc2l6ZSByZXBvcnQgMCA/ICov DQo+ID4+IC0gICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jlc291cmNlX3N0YXJ0KGFk ZXYtPnBkZXYsIDApOw0KPiA+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNv dXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+ICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24g c2kgKi8NCj4gPj4gICAgICAgICAgYWRldi0+bWMubWNfdnJhbV9zaXplID0gUlJFRzMyKG1tQ09O RklHX01FTVNJWkUpICoNCj4gMTAyNFVMTCAqIDEwMjRVTEw7DQo+ID4+ICAgICAgICAgIGFkZXYt Pm1jLnJlYWxfdnJhbV9zaXplID0gUlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gMTAyNFVM TCAqIDEwMjRVTEw7DQo+ID4+DQo+ID4+ICsgICAgICAgaWYgKCEoYWRldi0+ZmxhZ3MgJiBBTURf SVNfQVBVKSkNCj4gPj4gKyAgICAgICAgICAgICAgIGFtZGdwdV9yZXNpemVfYmFyMChhZGV2KTsN Cj4gPj4gKyAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRl di0+cGRldiwgMCk7DQo+ID4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9zaXplID0gcGNpX3Jlc291 cmNlX2xlbihhZGV2LT5wZGV2LCAwKTsNCj4gPj4gKw0KPiA+PiAgICNpZmRlZiBDT05GSUdfWDg2 XzY0DQo+ID4+ICAgICAgICAgIGlmIChhZGV2LT5mbGFncyAmIEFNRF9JU19BUFUpIHsNCj4gPj4g ICAgICAgICAgICAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSAoKHU2NClSUkVHMzIobW1NQ19W TV9GQl9PRkZTRVQpKQ0KPiA8PCAyMjsNCj4gPj4gLS0NCj4gPj4gMi43LjQNCj4gPj4NCj4gPj4g X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4gZHJp LWRldmVsIG1haWxpbmcgbGlzdA0KPiA+PiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3Jn DQo+ID4+IGh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJp LWRldmVsDQo+IA0KPiANCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18NCj4gYW1kLWdmeCBtYWlsaW5nIGxpc3QNCj4gYW1kLWdmeEBsaXN0cy5mcmVlZGVz a3RvcC5vcmcNCj4gaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m by9hbWQtZ2Z4DQo= ^ permalink raw reply [flat|nested] 72+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access @ 2017-03-15 16:08 ` Deucher, Alexander 0 siblings, 0 replies; 72+ messages in thread From: Deucher, Alexander @ 2017-03-15 16:08 UTC (permalink / raw) To: 'Christian König', Ayyappa Ch Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, helgaas-DgEjT+Ai2ygdnm+yROfE0A, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW > -----Original Message----- > From: amd-gfx [mailto:amd-gfx-bounces@lists.freedesktop.org] On Behalf > Of Christian König > Sent: Wednesday, March 15, 2017 3:38 AM > To: Ayyappa Ch > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd- > gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; > helgaas@kernel.org; dri-devel@lists.freedesktop.org > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access > > Carizzo is an APU and resizing BARs isn't needed nor supported there. > The CPU can access the full stolen VRAM directly on that hardware. > > As far as I know ASICs with support for this are Tonga, Fiji and all > Polaris variants. I think resizable BARs are supported as far back as evergreen or NI. Alex > > Christian. > > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch: > > Is it possible on Carrizo asics? Or only supports on newer asics? > > > > On Mon, Mar 13, 2017 at 6:11 PM, Christian König > > <deathsimple@vodafone.de> wrote: > >> From: Christian König <christian.koenig@amd.com> > >> > >> Try to resize BAR0 to let CPU access all of VRAM. > >> > >> Signed-off-by: Christian König <christian.koenig@amd.com> > >> --- > >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 > +++++++++++++++++++++++++++++ > >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > >> 4 files changed, 40 insertions(+), 6 deletions(-) > >> > >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> index 3b81ded..905ded9 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct > amdgpu_device *adev, struct ttm_tt *ttm, > >> struct ttm_mem_reg *mem); > >> void amdgpu_vram_location(struct amdgpu_device *adev, struct > amdgpu_mc *mc, u64 base); > >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct > amdgpu_mc *mc); > >> +void amdgpu_resize_bar0(struct amdgpu_device *adev); > >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, > u64 size); > >> int amdgpu_ttm_init(struct amdgpu_device *adev); > >> void amdgpu_ttm_fini(struct amdgpu_device *adev); > >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> index 118f4e6..92955fe 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device > *adev, struct amdgpu_mc *mc) > >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > >> } > >> > >> +/** > >> + * amdgpu_resize_bar0 - try to resize BAR0 > >> + * > >> + * @adev: amdgpu_device pointer > >> + * > >> + * Try to resize BAR0 to make all VRAM CPU accessible. > >> + */ > >> +void amdgpu_resize_bar0(struct amdgpu_device *adev) > >> +{ > >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > >> + int r; > >> + > >> + r = pci_resize_resource(adev->pdev, 0, size); > >> + > >> + if (r == -ENOTSUPP) { > >> + /* The hardware don't support the extension. */ > >> + return; > >> + > >> + } else if (r == -ENOSPC) { > >> + DRM_INFO("Not enoigh PCI address space for a large BAR."); > >> + } else if (r) { > >> + DRM_ERROR("Problem resizing BAR0 (%d).", r); > >> + } > >> + > >> + /* Reinit the doorbell mapping, it is most likely moved as well */ > >> + amdgpu_doorbell_fini(adev); > >> + BUG_ON(amdgpu_doorbell_init(adev)); > >> +} > >> + > >> /* > >> * GPU helpers function. > >> */ > >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> index dc9b6d6..36a7aa5 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c > >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct > amdgpu_device *adev) > >> break; > >> } > >> adev->mc.vram_width = numchan * chansize; > >> - /* Could aper size report 0 ? */ > >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> /* size in MB on si */ > >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> > >> + if (!(adev->flags & AMD_IS_APU)) > >> + amdgpu_resize_bar0(adev); > >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> + > >> #ifdef CONFIG_X86_64 > >> if (adev->flags & AMD_IS_APU) { > >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) > << 22; > >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> index c087b00..7761ad3 100644 > >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c > >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct > amdgpu_device *adev) > >> break; > >> } > >> adev->mc.vram_width = numchan * chansize; > >> - /* Could aper size report 0 ? */ > >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> /* size in MB on si */ > >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * > 1024ULL * 1024ULL; > >> > >> + if (!(adev->flags & AMD_IS_APU)) > >> + amdgpu_resize_bar0(adev); > >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0); > >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0); > >> + > >> #ifdef CONFIG_X86_64 > >> if (adev->flags & AMD_IS_APU) { > >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) > << 22; > >> -- > >> 2.7.4 > >> > >> _______________________________________________ > >> dri-devel mailing list > >> dri-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel > > > _______________________________________________ > amd-gfx mailing list > amd-gfx@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/amd-gfx _______________________________________________ amd-gfx mailing list amd-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/amd-gfx ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access 2017-03-13 12:41 ` Christian König ` (2 preceding siblings ...) (?) @ 2017-03-24 21:42 ` Bjorn Helgaas -1 siblings, 0 replies; 72+ messages in thread From: Bjorn Helgaas @ 2017-03-24 21:42 UTC (permalink / raw) To: Christian König Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel On Mon, Mar 13, 2017 at 01:41:36PM +0100, Christian König wrote: > From: Christian König <christian.koenig@amd.com> > > Try to resize BAR0 to let CPU access all of VRAM. > > Signed-off-by: Christian König <christian.koenig@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++ > drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++--- > drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++--- > 4 files changed, 40 insertions(+), 6 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > index 3b81ded..905ded9 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h > @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm, > struct ttm_mem_reg *mem); > void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base); > void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc); > +void amdgpu_resize_bar0(struct amdgpu_device *adev); > void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size); > int amdgpu_ttm_init(struct amdgpu_device *adev); > void amdgpu_ttm_fini(struct amdgpu_device *adev); > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > index 118f4e6..92955fe 100644 > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c > @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc) > mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end); > } > > +/** > + * amdgpu_resize_bar0 - try to resize BAR0 > + * > + * @adev: amdgpu_device pointer > + * > + * Try to resize BAR0 to make all VRAM CPU accessible. > + */ > +void amdgpu_resize_bar0(struct amdgpu_device *adev) > +{ > + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20; > + int r; > + > + r = pci_resize_resource(adev->pdev, 0, size); > + > + if (r == -ENOTSUPP) { > + /* The hardware don't support the extension. */ > + return; > + > + } else if (r == -ENOSPC) { > + DRM_INFO("Not enoigh PCI address space for a large BAR."); s/enoigh/enough/ > + } else if (r) { > + DRM_ERROR("Problem resizing BAR0 (%d).", r); > + } > + > + /* Reinit the doorbell mapping, it is most likely moved as well */ I think you should assume all BARs moved (I don't know how many you have; maybe this already covers all of them). > + amdgpu_doorbell_fini(adev); > + BUG_ON(amdgpu_doorbell_init(adev)); I think things inside BUG_ON() tend to get overlooked, so I avoid things that have side-effects. But that's just my personal preference. ^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors @ 2023-03-10 14:05 Brandi Gulbin 0 siblings, 0 replies; 72+ messages in thread From: Brandi Gulbin @ 2023-03-10 14:05 UTC (permalink / raw) To: deathsimple Cc: linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas, amd-gfx [-- Attachment #1: Type: text/plain, Size: 20 bytes --] -- Brandi B Gulbin [-- Attachment #2: Type: text/html, Size: 102 bytes --] ^ permalink raw reply [flat|nested] 72+ messages in thread
end of thread, other threads:[~2023-03-11 10:52 UTC | newest] Thread overview: 72+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-03-13 12:41 Resizeable PCI BAR support V3 Christian König 2017-03-13 12:41 ` Christian König 2017-03-13 12:41 ` [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 Christian König 2017-03-13 12:41 ` Christian König 2017-03-14 13:09 ` kbuild test robot 2017-03-14 13:09 ` kbuild test robot 2017-03-24 15:28 ` Bjorn Helgaas 2017-03-13 12:41 ` [PATCH 2/4] PCI: add functionality for resizing resources v2 Christian König 2017-03-13 12:41 ` Christian König 2017-03-13 16:43 ` Andy Shevchenko 2017-03-13 16:43 ` Andy Shevchenko 2017-03-13 16:43 ` Andy Shevchenko 2017-04-11 9:14 ` Christian König 2017-04-11 9:14 ` Christian König 2017-03-14 9:01 ` kbuild test robot 2017-03-24 21:34 ` Bjorn Helgaas 2017-04-11 15:37 ` Christian König 2017-04-12 16:37 ` Bjorn Helgaas 2017-03-13 12:41 ` [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors Christian König 2017-03-13 12:41 ` Christian König 2017-03-13 16:49 ` Andy Shevchenko 2017-03-13 16:49 ` Andy Shevchenko 2017-03-13 16:49 ` Andy Shevchenko 2017-04-11 9:21 ` Christian König 2017-04-11 9:21 ` Christian König 2017-03-14 9:25 ` kbuild test robot 2017-03-14 9:25 ` kbuild test robot 2017-03-24 15:47 ` Bjorn Helgaas 2017-03-24 15:47 ` Bjorn Helgaas 2017-04-11 15:48 ` Christian König 2017-04-11 15:48 ` Christian König 2017-04-12 16:55 ` Bjorn Helgaas 2017-04-25 13:01 ` Christian König 2017-05-17 21:36 ` Bjorn Helgaas 2017-03-13 12:41 ` [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access Christian König 2017-03-13 12:41 ` Christian König 2017-03-13 16:51 ` Andy Shevchenko 2017-03-13 16:51 ` Andy Shevchenko 2017-03-15 7:23 ` Ayyappa Ch 2017-03-15 7:23 ` Ayyappa Ch 2017-03-15 7:23 ` Ayyappa Ch 2017-03-15 7:37 ` Christian König 2017-03-15 7:37 ` Christian König 2017-03-15 8:25 ` Zhou, David(ChunMing) 2017-03-15 8:25 ` Zhou, David(ChunMing) 2017-03-15 8:25 ` Zhou, David(ChunMing) 2017-03-15 9:29 ` Christian König 2017-03-15 9:29 ` Christian König 2017-03-15 9:29 ` Christian König 2017-03-16 2:19 ` Zhang, Jerry 2017-03-16 2:19 ` Zhang, Jerry 2017-03-16 2:19 ` Zhang, Jerry 2017-03-16 2:25 ` Alex Deucher 2017-03-16 2:25 ` Alex Deucher 2017-03-16 2:25 ` Alex Deucher 2017-03-16 2:41 ` Zhang, Jerry 2017-03-16 2:41 ` Zhang, Jerry 2017-03-23 14:30 ` Sagalovitch, Serguei 2017-03-23 14:30 ` Sagalovitch, Serguei 2017-03-23 14:30 ` Sagalovitch, Serguei 2017-03-23 15:56 ` Christian König 2017-03-23 15:56 ` Christian König 2017-03-15 10:42 ` Ayyappa Ch 2017-03-15 10:42 ` Ayyappa Ch 2017-03-15 10:42 ` Ayyappa Ch 2017-03-15 11:03 ` Christian König 2017-03-15 11:03 ` Christian König 2017-03-15 16:08 ` Deucher, Alexander 2017-03-15 16:08 ` Deucher, Alexander 2017-03-15 16:08 ` Deucher, Alexander 2017-03-24 21:42 ` Bjorn Helgaas 2023-03-10 14:05 [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors Brandi Gulbin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.