* Resizeable PCI BAR support V3
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
Hi Bjorn and others on the lists,
it's over a year that I initially came up with that, but now I
finally have the time to finish it up.
This set of patches allows device drivers to resize and most likely also
relocate the PCI BAR of devices they manage to allow the CPU to access
all of the device local memory at once.
This is very useful for GFX device drivers where the default PCI BAR is only
about 256MB in size for compatibility reasons, but the device easily have
multiple gigabyte of local memory.
Additional to the pure resize functionality this set also adds a quirk to
enable a 64bit bar above 4GB on AMD Family 15h CPUs/APUs. Doing so is
actually trivial and I plan to extend this to all recent AMD CPUs/APUs.
Open questions:
1. Is this the right location to implement the 64bit BAR above 4GB for AMD CPUs?
I guess that this needs a bit more cleanup.
2. Any idea how to better handle intermediate bridges? Reprogramming them isn't
possible after drivers are loaded, but at least for GFX drivers we need to load
them first to know how large we actually want our BAR to be (and to kick our
vesafb/efifb).
Thanks in advance,
Christian.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Resizeable PCI BAR support V3
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
Hi Bjorn and others on the lists,
it's over a year that I initially came up with that, but now I
finally have the time to finish it up.
This set of patches allows device drivers to resize and most likely also
relocate the PCI BAR of devices they manage to allow the CPU to access
all of the device local memory at once.
This is very useful for GFX device drivers where the default PCI BAR is only
about 256MB in size for compatibility reasons, but the device easily have
multiple gigabyte of local memory.
Additional to the pure resize functionality this set also adds a quirk to
enable a 64bit bar above 4GB on AMD Family 15h CPUs/APUs. Doing so is
actually trivial and I plan to extend this to all recent AMD CPUs/APUs.
Open questions:
1. Is this the right location to implement the 64bit BAR above 4GB for AMD CPUs?
I guess that this needs a bit more cleanup.
2. Any idea how to better handle intermediate bridges? Reprogramming them isn't
possible after drivers are loaded, but at least for GFX drivers we need to load
them first to know how large we actually want our BAR to be (and to kick our
vesafb/efifb).
Thanks in advance,
Christian.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* [PATCH 1/4] PCI: add resizeable BAR infrastructure v3
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
From: Christian König <christian.koenig@amd.com>
Just the defines and helper functions to read the possible sizes of a BAR and
update it's size.
See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf.
This is useful for hardware with large local storage (mostly GFX) which only
expose 256MB BARs initially to be compatible with 32bit systems.
v2: provide read helper as well
v3: improve function names, use unsigned values, add better comments.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++
include/linux/pci.h | 3 ++
include/uapi/linux/pci_regs.h | 7 +++
3 files changed, 125 insertions(+)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ba34907..c2d9f78 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2944,6 +2944,121 @@ bool pci_acs_path_enabled(struct pci_dev *start,
}
/**
+ * pci_rbar_get_possible_sizes - get possible sizes for BAR
+ * @dev: PCI device
+ * @bar: BAR to query
+ *
+ * Get the possible sizes of a resizeable BAR as bitmask defined in the spec
+ * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable.
+ */
+u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar)
+{
+ unsigned pos, nbars;
+ u32 ctrl, cap;
+ unsigned i;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return 0;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
+
+ for (i = 0; i < nbars; ++i, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
+ PCI_REBAR_CTRL_BAR_IDX_SHIFT;
+ if (bar_idx != bar)
+ continue;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
+ return (cap & PCI_REBAR_CTRL_SIZES_MASK) >>
+ PCI_REBAR_CTRL_SIZES_SHIFT;
+ }
+
+ return 0;
+}
+
+/**
+ * pci_rbar_get_current_size - get the current size of a BAR
+ * @dev: PCI device
+ * @bar: BAR to set size to
+ *
+ * Read the size of a BAR from the resizeable BAR config.
+ * Returns size if found or negativ error code.
+ */
+int pci_rbar_get_current_size(struct pci_dev *pdev, int bar)
+{
+ unsigned pos, nbars;
+ u32 ctrl;
+ unsigned i;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return -ENOTSUPP;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
+
+ for (i = 0; i < nbars; ++i, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
+ PCI_REBAR_CTRL_BAR_IDX_SHIFT;
+ if (bar_idx != bar)
+ continue;
+
+ return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >>
+ PCI_REBAR_CTRL_BAR_SIZE_SHIFT;
+ }
+
+ return -ENOENT;
+}
+
+/**
+ * pci_rbar_set_size - set a new size for a BAR
+ * @dev: PCI device
+ * @bar: BAR to set size to
+ * @size: new size as defined in the spec (log2(size in bytes) - 20)
+ *
+ * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB).
+ * Returns true if resizing was successful, false otherwise.
+ */
+int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size)
+{
+ unsigned pos, nbars;
+ u32 ctrl;
+ unsigned i;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return -ENOTSUPP;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
+
+ for (i = 0; i < nbars; ++i, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
+ PCI_REBAR_CTRL_BAR_IDX_SHIFT;
+ if (bar_idx != bar)
+ continue;
+
+ ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE_MASK;
+ ctrl |= size << PCI_REBAR_CTRL_BAR_SIZE_SHIFT;
+ pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
+ return 0;
+ }
+
+ return -ENOENT;
+}
+
+/**
* pci_swizzle_interrupt_pin - swizzle INTx for device behind bridge
* @dev: the PCI device
* @pin: the INTx pin (1=INTA, 2=INTB, 3=INTC, 4=INTD)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index a38772a..6954e50 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1946,6 +1946,9 @@ void pci_request_acs(void);
bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags);
bool pci_acs_path_enabled(struct pci_dev *start,
struct pci_dev *end, u16 acs_flags);
+u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar);
+int pci_rbar_get_current_size(struct pci_dev *pdev, int bar);
+int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size);
#define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */
#define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT)
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index e5a2e68..6de29d6 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -932,9 +932,16 @@
#define PCI_SATA_SIZEOF_LONG 16
/* Resizable BARs */
+#define PCI_REBAR_CAP 4 /* capability register */
+#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */
+#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */
#define PCI_REBAR_CTRL 8 /* control register */
+#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for bar index */
+#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for bar index */
#define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */
#define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */
+#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for bar size */
+#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for bar size */
/* Dynamic Power Allocation */
#define PCI_DPA_CAP 4 /* capability register */
--
2.7.4
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 1/4] PCI: add resizeable BAR infrastructure v3
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas-DgEjT+Ai2ygdnm+yROfE0A, linux-pci-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
From: Christian König <christian.koenig@amd.com>
Just the defines and helper functions to read the possible sizes of a BAR and
update it's size.
See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf.
This is useful for hardware with large local storage (mostly GFX) which only
expose 256MB BARs initially to be compatible with 32bit systems.
v2: provide read helper as well
v3: improve function names, use unsigned values, add better comments.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/pci/pci.c | 115 ++++++++++++++++++++++++++++++++++++++++++
include/linux/pci.h | 3 ++
include/uapi/linux/pci_regs.h | 7 +++
3 files changed, 125 insertions(+)
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index ba34907..c2d9f78 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -2944,6 +2944,121 @@ bool pci_acs_path_enabled(struct pci_dev *start,
}
/**
+ * pci_rbar_get_possible_sizes - get possible sizes for BAR
+ * @dev: PCI device
+ * @bar: BAR to query
+ *
+ * Get the possible sizes of a resizeable BAR as bitmask defined in the spec
+ * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable.
+ */
+u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar)
+{
+ unsigned pos, nbars;
+ u32 ctrl, cap;
+ unsigned i;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return 0;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
+
+ for (i = 0; i < nbars; ++i, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
+ PCI_REBAR_CTRL_BAR_IDX_SHIFT;
+ if (bar_idx != bar)
+ continue;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
+ return (cap & PCI_REBAR_CTRL_SIZES_MASK) >>
+ PCI_REBAR_CTRL_SIZES_SHIFT;
+ }
+
+ return 0;
+}
+
+/**
+ * pci_rbar_get_current_size - get the current size of a BAR
+ * @dev: PCI device
+ * @bar: BAR to set size to
+ *
+ * Read the size of a BAR from the resizeable BAR config.
+ * Returns size if found or negativ error code.
+ */
+int pci_rbar_get_current_size(struct pci_dev *pdev, int bar)
+{
+ unsigned pos, nbars;
+ u32 ctrl;
+ unsigned i;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return -ENOTSUPP;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
+
+ for (i = 0; i < nbars; ++i, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
+ PCI_REBAR_CTRL_BAR_IDX_SHIFT;
+ if (bar_idx != bar)
+ continue;
+
+ return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >>
+ PCI_REBAR_CTRL_BAR_SIZE_SHIFT;
+ }
+
+ return -ENOENT;
+}
+
+/**
+ * pci_rbar_set_size - set a new size for a BAR
+ * @dev: PCI device
+ * @bar: BAR to set size to
+ * @size: new size as defined in the spec (log2(size in bytes) - 20)
+ *
+ * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB).
+ * Returns true if resizing was successful, false otherwise.
+ */
+int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size)
+{
+ unsigned pos, nbars;
+ u32 ctrl;
+ unsigned i;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
+ if (!pos)
+ return -ENOTSUPP;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
+
+ for (i = 0; i < nbars; ++i, pos += 8) {
+ int bar_idx;
+
+ pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
+ bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
+ PCI_REBAR_CTRL_BAR_IDX_SHIFT;
+ if (bar_idx != bar)
+ continue;
+
+ ctrl &= ~PCI_REBAR_CTRL_BAR_SIZE_MASK;
+ ctrl |= size << PCI_REBAR_CTRL_BAR_SIZE_SHIFT;
+ pci_write_config_dword(pdev, pos + PCI_REBAR_CTRL, ctrl);
+ return 0;
+ }
+
+ return -ENOENT;
+}
+
+/**
* pci_swizzle_interrupt_pin - swizzle INTx for device behind bridge
* @dev: the PCI device
* @pin: the INTx pin (1=INTA, 2=INTB, 3=INTC, 4=INTD)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index a38772a..6954e50 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1946,6 +1946,9 @@ void pci_request_acs(void);
bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags);
bool pci_acs_path_enabled(struct pci_dev *start,
struct pci_dev *end, u16 acs_flags);
+u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar);
+int pci_rbar_get_current_size(struct pci_dev *pdev, int bar);
+int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size);
#define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */
#define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT)
diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
index e5a2e68..6de29d6 100644
--- a/include/uapi/linux/pci_regs.h
+++ b/include/uapi/linux/pci_regs.h
@@ -932,9 +932,16 @@
#define PCI_SATA_SIZEOF_LONG 16
/* Resizable BARs */
+#define PCI_REBAR_CAP 4 /* capability register */
+#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */
+#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */
#define PCI_REBAR_CTRL 8 /* control register */
+#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for bar index */
+#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for bar index */
#define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */
#define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */
+#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for bar size */
+#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for bar size */
/* Dynamic Power Allocation */
#define PCI_DPA_CAP 4 /* capability register */
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 2/4] PCI: add functionality for resizing resources v2
2017-03-13 12:41 ` Christian König
@ 2017-03-13 12:41 ` Christian König
-1 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
From: Christian König <christian.koenig@amd.com>
This allows device drivers to request resizing their BARs.
The function only tries to reprogram the windows of the bridge directly above
the requesting device and only the BAR of the same type (usually mem, 64bit,
prefetchable). This is done to make sure not to disturb other drivers by
changing the BARs of their devices.
If reprogramming the bridge BAR fails the old status is restored and -ENOSPC
returned to the calling device driver.
v2: rebase on changes in rbar support
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/pci/setup-bus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
drivers/pci/setup-res.c | 51 +++++++++++++++++++++++++++++++++++++++++
include/linux/pci.h | 2 ++
3 files changed, 114 insertions(+)
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f30ca75..cfab2c7 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1923,6 +1923,67 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
}
EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
+int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
+{
+ const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
+ IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
+
+ struct resource saved;
+ LIST_HEAD(add_list);
+ LIST_HEAD(fail_head);
+ struct pci_dev_resource *fail_res;
+ unsigned i;
+ int ret = 0;
+
+ /* Release all children from the matching bridge resource */
+ for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
+ struct resource *res = &bridge->resource[i];
+
+ if ((res->flags & type_mask) != (type & type_mask))
+ continue;
+
+ saved = *res;
+ if (res->parent) {
+ release_child_resources(res);
+ release_resource(res);
+ }
+ res->start = 0;
+ res->end = 0;
+ break;
+ }
+
+ if (i == PCI_BRIDGE_RESOURCE_END)
+ return -ENOENT;
+
+ __pci_bus_size_bridges(bridge->subordinate, &add_list);
+ __pci_bridge_assign_resources(bridge, &add_list, &fail_head);
+ BUG_ON(!list_empty(&add_list));
+
+ /* restore size and flags */
+ list_for_each_entry(fail_res, &fail_head, list) {
+ struct resource *res = fail_res->res;
+
+ res->start = fail_res->start;
+ res->end = fail_res->end;
+ res->flags = fail_res->flags;
+ }
+
+ /* Revert to the old configuration */
+ if (!list_empty(&fail_head)) {
+ struct resource *res = &bridge->resource[i];
+
+ res->start = saved.start;
+ res->end = saved.end;
+ res->flags = saved.flags;
+
+ pci_claim_resource(bridge, i);
+ ret = -ENOSPC;
+ }
+
+ free_list(&fail_head);
+ return ret;
+}
+
void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
{
struct pci_dev *dev;
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 9526e34..3bb1e29 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -363,6 +363,57 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz
return 0;
}
+int pci_resize_resource(struct pci_dev *dev, int resno, int size)
+{
+ struct resource *res = dev->resource + resno;
+ u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
+ int old = pci_rbar_get_current_size(dev, resno);
+ u64 bytes = 1ULL << (size + 20);
+ int ret = 0;
+
+ if (!sizes)
+ return -ENOTSUPP;
+
+ if (!(sizes & (1 << size)))
+ return -EINVAL;
+
+ if (old < 0)
+ return old;
+
+ /* Make sure the resource isn't assigned before making it larger. */
+ if (resource_size(res) < bytes && res->parent) {
+ release_resource(res);
+ res->end = resource_size(res) - 1;
+ res->start = 0;
+ if (resno < PCI_BRIDGE_RESOURCES)
+ pci_update_resource(dev, resno);
+ }
+
+ ret = pci_rbar_set_size(dev, resno, size);
+ if (ret)
+ goto error_reassign;
+
+ res->end = res->start + bytes - 1;
+
+ ret = pci_reassign_bridge_resources(dev->bus->self, res->flags);
+ if (ret)
+ goto error_resize;
+
+ pci_reenable_device(dev->bus->self);
+ return 0;
+
+error_resize:
+ pci_rbar_set_size(dev, resno, old);
+ res->end = res->start + (1ULL << (old + 20)) - 1;
+
+error_reassign:
+ pci_assign_unassigned_bus_resources(dev->bus);
+ pci_setup_bridge(dev->bus);
+ pci_reenable_device(dev->bus->self);
+ return ret;
+}
+EXPORT_SYMBOL(pci_resize_resource);
+
int pci_enable_resources(struct pci_dev *dev, int mask)
{
u16 cmd, old_cmd;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 6954e50..8eaebb4 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1055,6 +1055,7 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev);
void pci_update_resource(struct pci_dev *dev, int resno);
int __must_check pci_assign_resource(struct pci_dev *dev, int i);
int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align);
+int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size);
int pci_select_bars(struct pci_dev *dev, unsigned long flags);
bool pci_device_is_present(struct pci_dev *pdev);
void pci_ignore_hotplug(struct pci_dev *dev);
@@ -1135,6 +1136,7 @@ void pci_assign_unassigned_resources(void);
void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge);
void pci_assign_unassigned_bus_resources(struct pci_bus *bus);
void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus);
+int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type);
void pdev_enable_device(struct pci_dev *);
int pci_enable_resources(struct pci_dev *, int mask);
void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *),
--
2.7.4
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 2/4] PCI: add functionality for resizing resources v2
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
From: Christian König <christian.koenig@amd.com>
This allows device drivers to request resizing their BARs.
The function only tries to reprogram the windows of the bridge directly above
the requesting device and only the BAR of the same type (usually mem, 64bit,
prefetchable). This is done to make sure not to disturb other drivers by
changing the BARs of their devices.
If reprogramming the bridge BAR fails the old status is restored and -ENOSPC
returned to the calling device driver.
v2: rebase on changes in rbar support
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/pci/setup-bus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
drivers/pci/setup-res.c | 51 +++++++++++++++++++++++++++++++++++++++++
include/linux/pci.h | 2 ++
3 files changed, 114 insertions(+)
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index f30ca75..cfab2c7 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1923,6 +1923,67 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
}
EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
+int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
+{
+ const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
+ IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
+
+ struct resource saved;
+ LIST_HEAD(add_list);
+ LIST_HEAD(fail_head);
+ struct pci_dev_resource *fail_res;
+ unsigned i;
+ int ret = 0;
+
+ /* Release all children from the matching bridge resource */
+ for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
+ struct resource *res = &bridge->resource[i];
+
+ if ((res->flags & type_mask) != (type & type_mask))
+ continue;
+
+ saved = *res;
+ if (res->parent) {
+ release_child_resources(res);
+ release_resource(res);
+ }
+ res->start = 0;
+ res->end = 0;
+ break;
+ }
+
+ if (i == PCI_BRIDGE_RESOURCE_END)
+ return -ENOENT;
+
+ __pci_bus_size_bridges(bridge->subordinate, &add_list);
+ __pci_bridge_assign_resources(bridge, &add_list, &fail_head);
+ BUG_ON(!list_empty(&add_list));
+
+ /* restore size and flags */
+ list_for_each_entry(fail_res, &fail_head, list) {
+ struct resource *res = fail_res->res;
+
+ res->start = fail_res->start;
+ res->end = fail_res->end;
+ res->flags = fail_res->flags;
+ }
+
+ /* Revert to the old configuration */
+ if (!list_empty(&fail_head)) {
+ struct resource *res = &bridge->resource[i];
+
+ res->start = saved.start;
+ res->end = saved.end;
+ res->flags = saved.flags;
+
+ pci_claim_resource(bridge, i);
+ ret = -ENOSPC;
+ }
+
+ free_list(&fail_head);
+ return ret;
+}
+
void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
{
struct pci_dev *dev;
diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
index 9526e34..3bb1e29 100644
--- a/drivers/pci/setup-res.c
+++ b/drivers/pci/setup-res.c
@@ -363,6 +363,57 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz
return 0;
}
+int pci_resize_resource(struct pci_dev *dev, int resno, int size)
+{
+ struct resource *res = dev->resource + resno;
+ u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
+ int old = pci_rbar_get_current_size(dev, resno);
+ u64 bytes = 1ULL << (size + 20);
+ int ret = 0;
+
+ if (!sizes)
+ return -ENOTSUPP;
+
+ if (!(sizes & (1 << size)))
+ return -EINVAL;
+
+ if (old < 0)
+ return old;
+
+ /* Make sure the resource isn't assigned before making it larger. */
+ if (resource_size(res) < bytes && res->parent) {
+ release_resource(res);
+ res->end = resource_size(res) - 1;
+ res->start = 0;
+ if (resno < PCI_BRIDGE_RESOURCES)
+ pci_update_resource(dev, resno);
+ }
+
+ ret = pci_rbar_set_size(dev, resno, size);
+ if (ret)
+ goto error_reassign;
+
+ res->end = res->start + bytes - 1;
+
+ ret = pci_reassign_bridge_resources(dev->bus->self, res->flags);
+ if (ret)
+ goto error_resize;
+
+ pci_reenable_device(dev->bus->self);
+ return 0;
+
+error_resize:
+ pci_rbar_set_size(dev, resno, old);
+ res->end = res->start + (1ULL << (old + 20)) - 1;
+
+error_reassign:
+ pci_assign_unassigned_bus_resources(dev->bus);
+ pci_setup_bridge(dev->bus);
+ pci_reenable_device(dev->bus->self);
+ return ret;
+}
+EXPORT_SYMBOL(pci_resize_resource);
+
int pci_enable_resources(struct pci_dev *dev, int mask)
{
u16 cmd, old_cmd;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 6954e50..8eaebb4 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1055,6 +1055,7 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev);
void pci_update_resource(struct pci_dev *dev, int resno);
int __must_check pci_assign_resource(struct pci_dev *dev, int i);
int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align);
+int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size);
int pci_select_bars(struct pci_dev *dev, unsigned long flags);
bool pci_device_is_present(struct pci_dev *pdev);
void pci_ignore_hotplug(struct pci_dev *dev);
@@ -1135,6 +1136,7 @@ void pci_assign_unassigned_resources(void);
void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge);
void pci_assign_unassigned_bus_resources(struct pci_bus *bus);
void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus);
+int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type);
void pdev_enable_device(struct pci_dev *);
int pci_enable_resources(struct pci_dev *, int mask);
void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *),
--
2.7.4
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
From: Christian König <christian.koenig@amd.com>
Most BIOS don't enable this because of compatibility reasons.
Manually enable a 64bit BAR of 64GB size so that we have
enough room for PCI devices.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
index 6d52b94..bff5242 100644
--- a/arch/x86/pci/fixup.c
+++ b/arch/x86/pci/fixup.c
@@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar);
+
+static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
+{
+ const uint64_t size = 64ULL * 1024 * 1024 * 1024;
+ uint32_t base, limit, high;
+ struct resource *res;
+ unsigned i;
+ int r;
+
+ for (i = 0; i < 8; ++i) {
+
+ pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
+ pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
+
+ /* Is this slot free? */
+ if ((base & 0x3) == 0x0)
+ break;
+
+ base >>= 8;
+ base |= high << 24;
+
+ /* Abort if a slot already configures a 64bit BAR. */
+ if (base > 0x10000)
+ return;
+
+ }
+
+ if (i == 8)
+ return;
+
+ res = kzalloc(sizeof(*res), GFP_KERNEL);
+ res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
+ IORESOURCE_WINDOW;
+ res->name = dev->bus->name;
+ r = allocate_resource(&iomem_resource, res, size, 0x100000000,
+ 0xfd00000000, size, NULL, NULL);
+ if (r) {
+ kfree(res);
+ return;
+ }
+
+ base = ((res->start >> 8) & 0xffffff00) | 0x3;
+ limit = ((res->end + 1) >> 8) & 0xffffff00;
+ high = ((res->start >> 40) & 0xff) |
+ ((((res->end + 1) >> 40) & 0xff) << 16);
+
+ pci_write_config_dword(dev, 0x180 + i * 0x4, high);
+ pci_write_config_dword(dev, 0x84 + i * 0x8, limit);
+ pci_write_config_dword(dev, 0x80 + i * 0x8, base);
+
+ pci_bus_add_resource(dev->bus, res, 0);
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar);
--
2.7.4
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas-DgEjT+Ai2ygdnm+yROfE0A, linux-pci-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
From: Christian König <christian.koenig@amd.com>
Most BIOS don't enable this because of compatibility reasons.
Manually enable a 64bit BAR of 64GB size so that we have
enough room for PCI devices.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
index 6d52b94..bff5242 100644
--- a/arch/x86/pci/fixup.c
+++ b/arch/x86/pci/fixup.c
@@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar);
DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar);
+
+static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
+{
+ const uint64_t size = 64ULL * 1024 * 1024 * 1024;
+ uint32_t base, limit, high;
+ struct resource *res;
+ unsigned i;
+ int r;
+
+ for (i = 0; i < 8; ++i) {
+
+ pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
+ pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
+
+ /* Is this slot free? */
+ if ((base & 0x3) == 0x0)
+ break;
+
+ base >>= 8;
+ base |= high << 24;
+
+ /* Abort if a slot already configures a 64bit BAR. */
+ if (base > 0x10000)
+ return;
+
+ }
+
+ if (i == 8)
+ return;
+
+ res = kzalloc(sizeof(*res), GFP_KERNEL);
+ res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
+ IORESOURCE_WINDOW;
+ res->name = dev->bus->name;
+ r = allocate_resource(&iomem_resource, res, size, 0x100000000,
+ 0xfd00000000, size, NULL, NULL);
+ if (r) {
+ kfree(res);
+ return;
+ }
+
+ base = ((res->start >> 8) & 0xffffff00) | 0x3;
+ limit = ((res->end + 1) >> 8) & 0xffffff00;
+ high = ((res->start >> 40) & 0xff) |
+ ((((res->end + 1) >> 40) & 0xff) << 16);
+
+ pci_write_config_dword(dev, 0x180 + i * 0x4, high);
+ pci_write_config_dword(dev, 0x84 + i * 0x8, limit);
+ pci_write_config_dword(dev, 0x80 + i * 0x8, base);
+
+ pci_bus_add_resource(dev->bus, res, 0);
+}
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar);
--
2.7.4
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-13 12:41 ` Christian König
@ 2017-03-13 12:41 ` Christian König
-1 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
From: Christian König <christian.koenig@amd.com>
Try to resize BAR0 to let CPU access all of VRAM.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
4 files changed, 40 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 3b81ded..905ded9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
struct ttm_mem_reg *mem);
void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
+void amdgpu_resize_bar0(struct amdgpu_device *adev);
void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
int amdgpu_ttm_init(struct amdgpu_device *adev);
void amdgpu_ttm_fini(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 118f4e6..92955fe 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
}
+/**
+ * amdgpu_resize_bar0 - try to resize BAR0
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Try to resize BAR0 to make all VRAM CPU accessible.
+ */
+void amdgpu_resize_bar0(struct amdgpu_device *adev)
+{
+ u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
+ int r;
+
+ r = pci_resize_resource(adev->pdev, 0, size);
+
+ if (r == -ENOTSUPP) {
+ /* The hardware don't support the extension. */
+ return;
+
+ } else if (r == -ENOSPC) {
+ DRM_INFO("Not enoigh PCI address space for a large BAR.");
+ } else if (r) {
+ DRM_ERROR("Problem resizing BAR0 (%d).", r);
+ }
+
+ /* Reinit the doorbell mapping, it is most likely moved as well */
+ amdgpu_doorbell_fini(adev);
+ BUG_ON(amdgpu_doorbell_init(adev));
+}
+
/*
* GPU helpers function.
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index dc9b6d6..36a7aa5 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
break;
}
adev->mc.vram_width = numchan * chansize;
- /* Could aper size report 0 ? */
- adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
- adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
/* size in MB on si */
adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
+ if (!(adev->flags & AMD_IS_APU))
+ amdgpu_resize_bar0(adev);
+ adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
+ adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
+
#ifdef CONFIG_X86_64
if (adev->flags & AMD_IS_APU) {
adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index c087b00..7761ad3 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
break;
}
adev->mc.vram_width = numchan * chansize;
- /* Could aper size report 0 ? */
- adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
- adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
/* size in MB on si */
adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
+ if (!(adev->flags & AMD_IS_APU))
+ amdgpu_resize_bar0(adev);
+ adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
+ adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
+
#ifdef CONFIG_X86_64
if (adev->flags & AMD_IS_APU) {
adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
--
2.7.4
^ permalink raw reply related [flat|nested] 71+ messages in thread
* [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-13 12:41 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-13 12:41 UTC (permalink / raw)
To: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
From: Christian König <christian.koenig@amd.com>
Try to resize BAR0 to let CPU access all of VRAM.
Signed-off-by: Christian König <christian.koenig@amd.com>
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
4 files changed, 40 insertions(+), 6 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 3b81ded..905ded9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
struct ttm_mem_reg *mem);
void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
+void amdgpu_resize_bar0(struct amdgpu_device *adev);
void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
int amdgpu_ttm_init(struct amdgpu_device *adev);
void amdgpu_ttm_fini(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 118f4e6..92955fe 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
}
+/**
+ * amdgpu_resize_bar0 - try to resize BAR0
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Try to resize BAR0 to make all VRAM CPU accessible.
+ */
+void amdgpu_resize_bar0(struct amdgpu_device *adev)
+{
+ u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
+ int r;
+
+ r = pci_resize_resource(adev->pdev, 0, size);
+
+ if (r == -ENOTSUPP) {
+ /* The hardware don't support the extension. */
+ return;
+
+ } else if (r == -ENOSPC) {
+ DRM_INFO("Not enoigh PCI address space for a large BAR.");
+ } else if (r) {
+ DRM_ERROR("Problem resizing BAR0 (%d).", r);
+ }
+
+ /* Reinit the doorbell mapping, it is most likely moved as well */
+ amdgpu_doorbell_fini(adev);
+ BUG_ON(amdgpu_doorbell_init(adev));
+}
+
/*
* GPU helpers function.
*/
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index dc9b6d6..36a7aa5 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
break;
}
adev->mc.vram_width = numchan * chansize;
- /* Could aper size report 0 ? */
- adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
- adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
/* size in MB on si */
adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
+ if (!(adev->flags & AMD_IS_APU))
+ amdgpu_resize_bar0(adev);
+ adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
+ adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
+
#ifdef CONFIG_X86_64
if (adev->flags & AMD_IS_APU) {
adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index c087b00..7761ad3 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
break;
}
adev->mc.vram_width = numchan * chansize;
- /* Could aper size report 0 ? */
- adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
- adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
/* size in MB on si */
adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
+ if (!(adev->flags & AMD_IS_APU))
+ amdgpu_resize_bar0(adev);
+ adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
+ adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
+
#ifdef CONFIG_X86_64
if (adev->flags & AMD_IS_APU) {
adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
--
2.7.4
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
2017-03-13 12:41 ` Christian König
(?)
@ 2017-03-13 16:43 ` Andy Shevchenko
-1 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:43 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 2:41 PM, Christian König
<deathsimple@vodafone.de> wrote:
> From: Christian König <christian.koenig@amd.com>
>
> This allows device drivers to request resizing their BARs.
>
> The function only tries to reprogram the windows of the bridge directly above
> the requesting device and only the BAR of the same type (usually mem, 64bit,
> prefetchable). This is done to make sure not to disturb other drivers by
> changing the BARs of their devices.
>
> If reprogramming the bridge BAR fails the old status is restored and -ENOSPC
> returned to the calling device driver.
Some style comments.
> v2: rebase on changes in rbar support
This kind of comments usually goes after --- delimiter below.
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
...here.
> +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
> +{
> + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
> + IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
> +
> + struct resource saved;
> + LIST_HEAD(add_list);
> + LIST_HEAD(fail_head);
> + struct pci_dev_resource *fail_res;
Perhaps move before definition of 'saved'?
> + unsigned i;
> + int ret = 0;
> +
> + /* Release all children from the matching bridge resource */
> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
> + struct resource *res = &bridge->resource[i];
> +
> + if ((res->flags & type_mask) != (type & type_mask))
IIUC it would be
if ((res->flags ^ type) & type_mask)
(I consider 'diff' as XOR operation is more understandable, but it's up to you)
> + continue;
> + /* restore size and flags */
> + list_for_each_entry(fail_res, &fail_head, list) {
> + struct resource *res = fail_res->res;
> +
> + res->start = fail_res->start;
> + res->end = fail_res->end;
> + res->flags = fail_res->flags;
> + }
> +
> + /* Revert to the old configuration */
> + if (!list_empty(&fail_head)) {
> + struct resource *res = &bridge->resource[i];
> +
> + res->start = saved.start;
> + res->end = saved.end;
> + res->flags = saved.flags;
Would
*res = saved;
work?
> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
> +{
> + struct resource *res = dev->resource + resno;
> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
> + int old = pci_rbar_get_current_size(dev, resno);
> + u64 bytes = 1ULL << (size + 20);
> + int ret = 0;
> +
I would put
sizes = pci_rbar_get_possible_sizes(dev, resno);
here
> + if (!sizes)
> + return -ENOTSUPP;
> +
> + if (!(sizes & (1 << size)))
BIT(size) ?
> + return -EINVAL;
> +
and
old = pci_rbar_get_current_size(dev, resno);
here
> + if (old < 0)
> + return old;
> +error_resize:
> + pci_rbar_set_size(dev, resno, old);
> + res->end = res->start + (1ULL << (old + 20)) - 1;
BIT_ULL(old + 20) ?
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
@ 2017-03-13 16:43 ` Andy Shevchenko
0 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:43 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, linux-kernel, amd-gfx, Platform Driver, helgaas, dri-devel
On Mon, Mar 13, 2017 at 2:41 PM, Christian König
<deathsimple@vodafone.de> wrote:
> From: Christian König <christian.koenig@amd.com>
>
> This allows device drivers to request resizing their BARs.
>
> The function only tries to reprogram the windows of the bridge directly above
> the requesting device and only the BAR of the same type (usually mem, 64bit,
> prefetchable). This is done to make sure not to disturb other drivers by
> changing the BARs of their devices.
>
> If reprogramming the bridge BAR fails the old status is restored and -ENOSPC
> returned to the calling device driver.
Some style comments.
> v2: rebase on changes in rbar support
This kind of comments usually goes after --- delimiter below.
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
...here.
> +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
> +{
> + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
> + IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
> +
> + struct resource saved;
> + LIST_HEAD(add_list);
> + LIST_HEAD(fail_head);
> + struct pci_dev_resource *fail_res;
Perhaps move before definition of 'saved'?
> + unsigned i;
> + int ret = 0;
> +
> + /* Release all children from the matching bridge resource */
> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
> + struct resource *res = &bridge->resource[i];
> +
> + if ((res->flags & type_mask) != (type & type_mask))
IIUC it would be
if ((res->flags ^ type) & type_mask)
(I consider 'diff' as XOR operation is more understandable, but it's up to you)
> + continue;
> + /* restore size and flags */
> + list_for_each_entry(fail_res, &fail_head, list) {
> + struct resource *res = fail_res->res;
> +
> + res->start = fail_res->start;
> + res->end = fail_res->end;
> + res->flags = fail_res->flags;
> + }
> +
> + /* Revert to the old configuration */
> + if (!list_empty(&fail_head)) {
> + struct resource *res = &bridge->resource[i];
> +
> + res->start = saved.start;
> + res->end = saved.end;
> + res->flags = saved.flags;
Would
*res = saved;
work?
> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
> +{
> + struct resource *res = dev->resource + resno;
> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
> + int old = pci_rbar_get_current_size(dev, resno);
> + u64 bytes = 1ULL << (size + 20);
> + int ret = 0;
> +
I would put
sizes = pci_rbar_get_possible_sizes(dev, resno);
here
> + if (!sizes)
> + return -ENOTSUPP;
> +
> + if (!(sizes & (1 << size)))
BIT(size) ?
> + return -EINVAL;
> +
and
old = pci_rbar_get_current_size(dev, resno);
here
> + if (old < 0)
> + return old;
> +error_resize:
> + pci_rbar_set_size(dev, resno, old);
> + res->end = res->start + (1ULL << (old + 20)) - 1;
BIT_ULL(old + 20) ?
--
With Best Regards,
Andy Shevchenko
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
@ 2017-03-13 16:43 ` Andy Shevchenko
0 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:43 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 2:41 PM, Christian K=C3=B6nig
<deathsimple@vodafone.de> wrote:
> From: Christian K=C3=B6nig <christian.koenig@amd.com>
>
> This allows device drivers to request resizing their BARs.
>
> The function only tries to reprogram the windows of the bridge directly a=
bove
> the requesting device and only the BAR of the same type (usually mem, 64b=
it,
> prefetchable). This is done to make sure not to disturb other drivers by
> changing the BARs of their devices.
>
> If reprogramming the bridge BAR fails the old status is restored and -ENO=
SPC
> returned to the calling device driver.
Some style comments.
> v2: rebase on changes in rbar support
This kind of comments usually goes after --- delimiter below.
> Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com>
> ---
...here.
> +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long =
type)
> +{
> + const unsigned long type_mask =3D IORESOURCE_IO | IORESOURCE_MEM =
|
> + IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
> +
> + struct resource saved;
> + LIST_HEAD(add_list);
> + LIST_HEAD(fail_head);
> + struct pci_dev_resource *fail_res;
Perhaps move before definition of 'saved'?
> + unsigned i;
> + int ret =3D 0;
> +
> + /* Release all children from the matching bridge resource */
> + for (i =3D PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i=
) {
> + struct resource *res =3D &bridge->resource[i];
> +
> + if ((res->flags & type_mask) !=3D (type & type_mask))
IIUC it would be
if ((res->flags ^ type) & type_mask)
(I consider 'diff' as XOR operation is more understandable, but it's up to =
you)
> + continue;
> + /* restore size and flags */
> + list_for_each_entry(fail_res, &fail_head, list) {
> + struct resource *res =3D fail_res->res;
> +
> + res->start =3D fail_res->start;
> + res->end =3D fail_res->end;
> + res->flags =3D fail_res->flags;
> + }
> +
> + /* Revert to the old configuration */
> + if (!list_empty(&fail_head)) {
> + struct resource *res =3D &bridge->resource[i];
> +
> + res->start =3D saved.start;
> + res->end =3D saved.end;
> + res->flags =3D saved.flags;
Would
*res =3D saved;
work?
> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
> +{
> + struct resource *res =3D dev->resource + resno;
> + u32 sizes =3D pci_rbar_get_possible_sizes(dev, resno);
> + int old =3D pci_rbar_get_current_size(dev, resno);
> + u64 bytes =3D 1ULL << (size + 20);
> + int ret =3D 0;
> +
I would put
sizes =3D pci_rbar_get_possible_sizes(dev, resno);
here
> + if (!sizes)
> + return -ENOTSUPP;
> +
> + if (!(sizes & (1 << size)))
BIT(size) ?
> + return -EINVAL;
> +
and
old =3D pci_rbar_get_current_size(dev, resno);
here
> + if (old < 0)
> + return old;
> +error_resize:
> + pci_rbar_set_size(dev, resno, old);
> + res->end =3D res->start + (1ULL << (old + 20)) - 1;
BIT_ULL(old + 20) ?
--=20
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
2017-03-13 12:41 ` Christian König
(?)
@ 2017-03-13 16:49 ` Andy Shevchenko
-1 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:49 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 2:41 PM, Christian König
<deathsimple@vodafone.de> wrote:
> Most BIOS don't enable this because of compatibility reasons.
>
> Manually enable a 64bit BAR of 64GB size so that we have
> enough room for PCI devices.
> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
> +{
> + const uint64_t size = 64ULL * 1024 * 1024 * 1024;
Perhaps extend <linux/sizes.h> and use SZ_64G here?
It would be nice to do, since some of the drivers already are using
sizes like 4GB and alike.
> + uint32_t base, limit, high;
> + struct resource *res;
> + unsigned i;
> + int r;
> +
> + for (i = 0; i < 8; ++i) {
> +
Redundant empty line.
> + pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
> + pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
> +
> + /* Is this slot free? */
> + if ((base & 0x3) == 0x0)
> + break;
> +
> + base >>= 8;
> + base |= high << 24;
> +
> + /* Abort if a slot already configures a 64bit BAR. */
> + if (base > 0x10000)
> + return;
> +
Ditto.
> + }
> +
Ditto.
> + if (i == 8)
> + return;
> +
> + res = kzalloc(sizeof(*res), GFP_KERNEL);
> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
> + IORESOURCE_WINDOW;
> + res->name = dev->bus->name;
> + r = allocate_resource(&iomem_resource, res, size, 0x100000000,
> + 0xfd00000000, size, NULL, NULL);
> + if (r) {
> + kfree(res);
> + return;
> + }
> +
> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
> + limit = ((res->end + 1) >> 8) & 0xffffff00;
> + high = ((res->start >> 40) & 0xff) |
> + ((((res->end + 1) >> 40) & 0xff) << 16);
Perhaps some of constants can be replaced by defines (I think some of
them are already defined in ioport.h or somewhere else).
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-03-13 16:49 ` Andy Shevchenko
0 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:49 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, linux-kernel, amd-gfx, Platform Driver, helgaas, dri-devel
On Mon, Mar 13, 2017 at 2:41 PM, Christian König
<deathsimple@vodafone.de> wrote:
> Most BIOS don't enable this because of compatibility reasons.
>
> Manually enable a 64bit BAR of 64GB size so that we have
> enough room for PCI devices.
> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
> +{
> + const uint64_t size = 64ULL * 1024 * 1024 * 1024;
Perhaps extend <linux/sizes.h> and use SZ_64G here?
It would be nice to do, since some of the drivers already are using
sizes like 4GB and alike.
> + uint32_t base, limit, high;
> + struct resource *res;
> + unsigned i;
> + int r;
> +
> + for (i = 0; i < 8; ++i) {
> +
Redundant empty line.
> + pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
> + pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
> +
> + /* Is this slot free? */
> + if ((base & 0x3) == 0x0)
> + break;
> +
> + base >>= 8;
> + base |= high << 24;
> +
> + /* Abort if a slot already configures a 64bit BAR. */
> + if (base > 0x10000)
> + return;
> +
Ditto.
> + }
> +
Ditto.
> + if (i == 8)
> + return;
> +
> + res = kzalloc(sizeof(*res), GFP_KERNEL);
> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
> + IORESOURCE_WINDOW;
> + res->name = dev->bus->name;
> + r = allocate_resource(&iomem_resource, res, size, 0x100000000,
> + 0xfd00000000, size, NULL, NULL);
> + if (r) {
> + kfree(res);
> + return;
> + }
> +
> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
> + limit = ((res->end + 1) >> 8) & 0xffffff00;
> + high = ((res->start >> 40) & 0xff) |
> + ((((res->end + 1) >> 40) & 0xff) << 16);
Perhaps some of constants can be replaced by defines (I think some of
them are already defined in ioport.h or somewhere else).
--
With Best Regards,
Andy Shevchenko
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-03-13 16:49 ` Andy Shevchenko
0 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:49 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 2:41 PM, Christian K=C3=B6nig
<deathsimple@vodafone.de> wrote:
> Most BIOS don't enable this because of compatibility reasons.
>
> Manually enable a 64bit BAR of 64GB size so that we have
> enough room for PCI devices.
> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
> +{
> + const uint64_t size =3D 64ULL * 1024 * 1024 * 1024;
Perhaps extend <linux/sizes.h> and use SZ_64G here?
It would be nice to do, since some of the drivers already are using
sizes like 4GB and alike.
> + uint32_t base, limit, high;
> + struct resource *res;
> + unsigned i;
> + int r;
> +
> + for (i =3D 0; i < 8; ++i) {
> +
Redundant empty line.
> + pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
> + pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
> +
> + /* Is this slot free? */
> + if ((base & 0x3) =3D=3D 0x0)
> + break;
> +
> + base >>=3D 8;
> + base |=3D high << 24;
> +
> + /* Abort if a slot already configures a 64bit BAR. */
> + if (base > 0x10000)
> + return;
> +
Ditto.
> + }
> +
Ditto.
> + if (i =3D=3D 8)
> + return;
> +
> + res =3D kzalloc(sizeof(*res), GFP_KERNEL);
> + res->flags =3D IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_=
MEM_64 |
> + IORESOURCE_WINDOW;
> + res->name =3D dev->bus->name;
> + r =3D allocate_resource(&iomem_resource, res, size, 0x100000000,
> + 0xfd00000000, size, NULL, NULL);
> + if (r) {
> + kfree(res);
> + return;
> + }
> +
> + base =3D ((res->start >> 8) & 0xffffff00) | 0x3;
> + limit =3D ((res->end + 1) >> 8) & 0xffffff00;
> + high =3D ((res->start >> 40) & 0xff) |
> + ((((res->end + 1) >> 40) & 0xff) << 16);
Perhaps some of constants can be replaced by defines (I think some of
them are already defined in ioport.h or somewhere else).
--=20
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-13 12:41 ` Christian König
@ 2017-03-13 16:51 ` Andy Shevchenko
-1 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:51 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 2:41 PM, Christian König
<deathsimple@vodafone.de> wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Try to resize BAR0 to let CPU access all of VRAM.
> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> +{
> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> + int r;
> +
> + r = pci_resize_resource(adev->pdev, 0, size);
> +
Redundant.
> + if (r == -ENOTSUPP) {
> + /* The hardware don't support the extension. */
> + return;
> +
Ditto.
> + } else if (r == -ENOSPC) {
Useless use of else. And thus of curly braces.
> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> + } else if (r) {
> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> + }
> +
> + /* Reinit the doorbell mapping, it is most likely moved as well */
> + amdgpu_doorbell_fini(adev);
> + BUG_ON(amdgpu_doorbell_init(adev));
Comment why it's used here.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-13 16:51 ` Andy Shevchenko
0 siblings, 0 replies; 71+ messages in thread
From: Andy Shevchenko @ 2017-03-13 16:51 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 2:41 PM, Christian K=C3=B6nig
<deathsimple@vodafone.de> wrote:
> From: Christian K=C3=B6nig <christian.koenig@amd.com>
>
> Try to resize BAR0 to let CPU access all of VRAM.
> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> +{
> + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20=
;
> + int r;
> +
> + r =3D pci_resize_resource(adev->pdev, 0, size);
> +
Redundant.
> + if (r =3D=3D -ENOTSUPP) {
> + /* The hardware don't support the extension. */
> + return;
> +
Ditto.
> + } else if (r =3D=3D -ENOSPC) {
Useless use of else. And thus of curly braces.
> + DRM_INFO("Not enoigh PCI address space for a large BAR.")=
;
> + } else if (r) {
> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> + }
> +
> + /* Reinit the doorbell mapping, it is most likely moved as well *=
/
> + amdgpu_doorbell_fini(adev);
> + BUG_ON(amdgpu_doorbell_init(adev));
Comment why it's used here.
--=20
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
2017-03-13 12:41 ` Christian König
(?)
(?)
@ 2017-03-14 9:01 ` kbuild test robot
-1 siblings, 0 replies; 71+ messages in thread
From: kbuild test robot @ 2017-03-14 9:01 UTC (permalink / raw)
To: Christian König
Cc: kbuild-all, helgaas, linux-pci, dri-devel, platform-driver-x86,
amd-gfx, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2131 bytes --]
Hi Christian,
[auto build test WARNING on pci/next]
[also build test WARNING on v4.11-rc2 next-20170310]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334
base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: i386-randconfig-x077-201711 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All warnings (new ones prefixed by >>):
drivers/pci/setup-res.c: In function 'pci_resize_resource':
>> drivers/pci/setup-res.c:422:2: warning: ignoring return value of 'pci_reenable_device', declared with attribute warn_unused_result [-Wunused-result]
pci_reenable_device(dev->bus->self);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/pci/setup-res.c:432:2: warning: ignoring return value of 'pci_reenable_device', declared with attribute warn_unused_result [-Wunused-result]
pci_reenable_device(dev->bus->self);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
vim +/pci_reenable_device +422 drivers/pci/setup-res.c
406 res->end = resource_size(res) - 1;
407 res->start = 0;
408 if (resno < PCI_BRIDGE_RESOURCES)
409 pci_update_resource(dev, resno);
410 }
411
412 ret = pci_rbar_set_size(dev, resno, size);
413 if (ret)
414 goto error_reassign;
415
416 res->end = res->start + bytes - 1;
417
418 ret = pci_reassign_bridge_resources(dev->bus->self, res->flags);
419 if (ret)
420 goto error_resize;
421
> 422 pci_reenable_device(dev->bus->self);
423 return 0;
424
425 error_resize:
426 pci_rbar_set_size(dev, resno, old);
427 res->end = res->start + (1ULL << (old + 20)) - 1;
428
429 error_reassign:
430 pci_assign_unassigned_bus_resources(dev->bus);
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 24132 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
2017-03-13 12:41 ` Christian König
@ 2017-03-14 9:25 ` kbuild test robot
-1 siblings, 0 replies; 71+ messages in thread
From: kbuild test robot @ 2017-03-14 9:25 UTC (permalink / raw)
To: Christian König
Cc: kbuild-all, helgaas, linux-pci, dri-devel, platform-driver-x86,
amd-gfx, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 2346 bytes --]
Hi Christian,
[auto build test WARNING on pci/next]
[also build test WARNING on v4.11-rc2 next-20170310]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334
base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: i386-randconfig-s0-201711 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All warnings (new ones prefixed by >>):
arch/x86/pci/fixup.c: In function 'pci_amd_enable_64bit_bar':
>> arch/x86/pci/fixup.c:608:52: warning: large integer implicitly truncated to unsigned type [-Woverflow]
r = allocate_resource(&iomem_resource, res, size, 0x100000000,
^~~~~~~~~~~
arch/x86/pci/fixup.c:609:10: warning: large integer implicitly truncated to unsigned type [-Woverflow]
0xfd00000000, size, NULL, NULL);
^~~~~~~~~~~~
>> arch/x86/pci/fixup.c:617:22: warning: right shift count >= width of type [-Wshift-count-overflow]
high = ((res->start >> 40) & 0xff) |
^~
arch/x86/pci/fixup.c:618:21: warning: right shift count >= width of type [-Wshift-count-overflow]
((((res->end + 1) >> 40) & 0xff) << 16);
^~
vim +608 arch/x86/pci/fixup.c
602 return;
603
604 res = kzalloc(sizeof(*res), GFP_KERNEL);
605 res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
606 IORESOURCE_WINDOW;
607 res->name = dev->bus->name;
> 608 r = allocate_resource(&iomem_resource, res, size, 0x100000000,
609 0xfd00000000, size, NULL, NULL);
610 if (r) {
611 kfree(res);
612 return;
613 }
614
615 base = ((res->start >> 8) & 0xffffff00) | 0x3;
616 limit = ((res->end + 1) >> 8) & 0xffffff00;
> 617 high = ((res->start >> 40) & 0xff) |
618 ((((res->end + 1) >> 40) & 0xff) << 16);
619
620 pci_write_config_dword(dev, 0x180 + i * 0x4, high);
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 27265 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-03-14 9:25 ` kbuild test robot
0 siblings, 0 replies; 71+ messages in thread
From: kbuild test robot @ 2017-03-14 9:25 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, linux-kernel, amd-gfx, platform-driver-x86,
helgaas, kbuild-all
[-- Attachment #1: Type: text/plain, Size: 2346 bytes --]
Hi Christian,
[auto build test WARNING on pci/next]
[also build test WARNING on v4.11-rc2 next-20170310]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334
base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: i386-randconfig-s0-201711 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All warnings (new ones prefixed by >>):
arch/x86/pci/fixup.c: In function 'pci_amd_enable_64bit_bar':
>> arch/x86/pci/fixup.c:608:52: warning: large integer implicitly truncated to unsigned type [-Woverflow]
r = allocate_resource(&iomem_resource, res, size, 0x100000000,
^~~~~~~~~~~
arch/x86/pci/fixup.c:609:10: warning: large integer implicitly truncated to unsigned type [-Woverflow]
0xfd00000000, size, NULL, NULL);
^~~~~~~~~~~~
>> arch/x86/pci/fixup.c:617:22: warning: right shift count >= width of type [-Wshift-count-overflow]
high = ((res->start >> 40) & 0xff) |
^~
arch/x86/pci/fixup.c:618:21: warning: right shift count >= width of type [-Wshift-count-overflow]
((((res->end + 1) >> 40) & 0xff) << 16);
^~
vim +608 arch/x86/pci/fixup.c
602 return;
603
604 res = kzalloc(sizeof(*res), GFP_KERNEL);
605 res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
606 IORESOURCE_WINDOW;
607 res->name = dev->bus->name;
> 608 r = allocate_resource(&iomem_resource, res, size, 0x100000000,
609 0xfd00000000, size, NULL, NULL);
610 if (r) {
611 kfree(res);
612 return;
613 }
614
615 base = ((res->start >> 8) & 0xffffff00) | 0x3;
616 limit = ((res->end + 1) >> 8) & 0xffffff00;
> 617 high = ((res->start >> 40) & 0xff) |
618 ((((res->end + 1) >> 40) & 0xff) << 16);
619
620 pci_write_config_dword(dev, 0x180 + i * 0x4, high);
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 27265 bytes --]
[-- Attachment #3: Type: text/plain, Size: 160 bytes --]
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 1/4] PCI: add resizeable BAR infrastructure v3
2017-03-13 12:41 ` Christian König
@ 2017-03-14 13:09 ` kbuild test robot
-1 siblings, 0 replies; 71+ messages in thread
From: kbuild test robot @ 2017-03-14 13:09 UTC (permalink / raw)
To: Christian König
Cc: kbuild-all, helgaas, linux-pci, dri-devel, platform-driver-x86,
amd-gfx, linux-kernel
[-- Attachment #1: Type: text/plain, Size: 4455 bytes --]
Hi Christian,
[auto build test WARNING on pci/next]
[also build test WARNING on v4.11-rc2 next-20170310]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334
base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
reproduce: make htmldocs
All warnings (new ones prefixed by >>):
lib/crc32.c:148: warning: No description found for parameter 'tab)[256]'
lib/crc32.c:148: warning: Excess function parameter 'tab' description in 'crc32_le_generic'
lib/crc32.c:293: warning: No description found for parameter 'tab)[256]'
lib/crc32.c:293: warning: Excess function parameter 'tab' description in 'crc32_be_generic'
lib/crc32.c:1: warning: no structured comments found
>> drivers/pci/pci.c:2951: warning: No description found for parameter 'pdev'
>> drivers/pci/pci.c:2951: warning: Excess function parameter 'dev' description in 'pci_rbar_get_possible_sizes'
drivers/pci/pci.c:2989: warning: No description found for parameter 'pdev'
>> drivers/pci/pci.c:2989: warning: Excess function parameter 'dev' description in 'pci_rbar_get_current_size'
drivers/pci/pci.c:3027: warning: No description found for parameter 'pdev'
>> drivers/pci/pci.c:3027: warning: Excess function parameter 'dev' description in 'pci_rbar_set_size'
vim +/pdev +2951 drivers/pci/pci.c
2945 * @bar: BAR to query
2946 *
2947 * Get the possible sizes of a resizeable BAR as bitmask defined in the spec
2948 * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable.
2949 */
2950 u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar)
> 2951 {
2952 unsigned pos, nbars;
2953 u32 ctrl, cap;
2954 unsigned i;
2955
2956 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
2957 if (!pos)
2958 return 0;
2959
2960 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
2961 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
2962
2963 for (i = 0; i < nbars; ++i, pos += 8) {
2964 int bar_idx;
2965
2966 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
2967 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
2968 PCI_REBAR_CTRL_BAR_IDX_SHIFT;
2969 if (bar_idx != bar)
2970 continue;
2971
2972 pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
2973 return (cap & PCI_REBAR_CTRL_SIZES_MASK) >>
2974 PCI_REBAR_CTRL_SIZES_SHIFT;
2975 }
2976
2977 return 0;
2978 }
2979
2980 /**
2981 * pci_rbar_get_current_size - get the current size of a BAR
2982 * @dev: PCI device
2983 * @bar: BAR to set size to
2984 *
2985 * Read the size of a BAR from the resizeable BAR config.
2986 * Returns size if found or negativ error code.
2987 */
2988 int pci_rbar_get_current_size(struct pci_dev *pdev, int bar)
> 2989 {
2990 unsigned pos, nbars;
2991 u32 ctrl;
2992 unsigned i;
2993
2994 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
2995 if (!pos)
2996 return -ENOTSUPP;
2997
2998 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
2999 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
3000
3001 for (i = 0; i < nbars; ++i, pos += 8) {
3002 int bar_idx;
3003
3004 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
3005 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
3006 PCI_REBAR_CTRL_BAR_IDX_SHIFT;
3007 if (bar_idx != bar)
3008 continue;
3009
3010 return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >>
3011 PCI_REBAR_CTRL_BAR_SIZE_SHIFT;
3012 }
3013
3014 return -ENOENT;
3015 }
3016
3017 /**
3018 * pci_rbar_set_size - set a new size for a BAR
3019 * @dev: PCI device
3020 * @bar: BAR to set size to
3021 * @size: new size as defined in the spec (log2(size in bytes) - 20)
3022 *
3023 * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB).
3024 * Returns true if resizing was successful, false otherwise.
3025 */
3026 int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size)
> 3027 {
3028 unsigned pos, nbars;
3029 u32 ctrl;
3030 unsigned i;
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 6576 bytes --]
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 1/4] PCI: add resizeable BAR infrastructure v3
@ 2017-03-14 13:09 ` kbuild test robot
0 siblings, 0 replies; 71+ messages in thread
From: kbuild test robot @ 2017-03-14 13:09 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, linux-kernel, amd-gfx, platform-driver-x86,
helgaas, kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4455 bytes --]
Hi Christian,
[auto build test WARNING on pci/next]
[also build test WARNING on v4.11-rc2 next-20170310]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Christian-K-nig/PCI-add-resizeable-BAR-infrastructure-v3/20170314-163334
base: https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
reproduce: make htmldocs
All warnings (new ones prefixed by >>):
lib/crc32.c:148: warning: No description found for parameter 'tab)[256]'
lib/crc32.c:148: warning: Excess function parameter 'tab' description in 'crc32_le_generic'
lib/crc32.c:293: warning: No description found for parameter 'tab)[256]'
lib/crc32.c:293: warning: Excess function parameter 'tab' description in 'crc32_be_generic'
lib/crc32.c:1: warning: no structured comments found
>> drivers/pci/pci.c:2951: warning: No description found for parameter 'pdev'
>> drivers/pci/pci.c:2951: warning: Excess function parameter 'dev' description in 'pci_rbar_get_possible_sizes'
drivers/pci/pci.c:2989: warning: No description found for parameter 'pdev'
>> drivers/pci/pci.c:2989: warning: Excess function parameter 'dev' description in 'pci_rbar_get_current_size'
drivers/pci/pci.c:3027: warning: No description found for parameter 'pdev'
>> drivers/pci/pci.c:3027: warning: Excess function parameter 'dev' description in 'pci_rbar_set_size'
vim +/pdev +2951 drivers/pci/pci.c
2945 * @bar: BAR to query
2946 *
2947 * Get the possible sizes of a resizeable BAR as bitmask defined in the spec
2948 * (bit 0=1MB, bit 19=512GB). Returns 0 if BAR isn't resizeable.
2949 */
2950 u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar)
> 2951 {
2952 unsigned pos, nbars;
2953 u32 ctrl, cap;
2954 unsigned i;
2955
2956 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
2957 if (!pos)
2958 return 0;
2959
2960 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
2961 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
2962
2963 for (i = 0; i < nbars; ++i, pos += 8) {
2964 int bar_idx;
2965
2966 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
2967 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
2968 PCI_REBAR_CTRL_BAR_IDX_SHIFT;
2969 if (bar_idx != bar)
2970 continue;
2971
2972 pci_read_config_dword(pdev, pos + PCI_REBAR_CAP, &cap);
2973 return (cap & PCI_REBAR_CTRL_SIZES_MASK) >>
2974 PCI_REBAR_CTRL_SIZES_SHIFT;
2975 }
2976
2977 return 0;
2978 }
2979
2980 /**
2981 * pci_rbar_get_current_size - get the current size of a BAR
2982 * @dev: PCI device
2983 * @bar: BAR to set size to
2984 *
2985 * Read the size of a BAR from the resizeable BAR config.
2986 * Returns size if found or negativ error code.
2987 */
2988 int pci_rbar_get_current_size(struct pci_dev *pdev, int bar)
> 2989 {
2990 unsigned pos, nbars;
2991 u32 ctrl;
2992 unsigned i;
2993
2994 pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_REBAR);
2995 if (!pos)
2996 return -ENOTSUPP;
2997
2998 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
2999 nbars = (ctrl & PCI_REBAR_CTRL_NBAR_MASK) >> PCI_REBAR_CTRL_NBAR_SHIFT;
3000
3001 for (i = 0; i < nbars; ++i, pos += 8) {
3002 int bar_idx;
3003
3004 pci_read_config_dword(pdev, pos + PCI_REBAR_CTRL, &ctrl);
3005 bar_idx = (ctrl & PCI_REBAR_CTRL_BAR_IDX_MASK) >>
3006 PCI_REBAR_CTRL_BAR_IDX_SHIFT;
3007 if (bar_idx != bar)
3008 continue;
3009
3010 return (ctrl & PCI_REBAR_CTRL_BAR_SIZE_MASK) >>
3011 PCI_REBAR_CTRL_BAR_SIZE_SHIFT;
3012 }
3013
3014 return -ENOENT;
3015 }
3016
3017 /**
3018 * pci_rbar_set_size - set a new size for a BAR
3019 * @dev: PCI device
3020 * @bar: BAR to set size to
3021 * @size: new size as defined in the spec (log2(size in bytes) - 20)
3022 *
3023 * Set the new size of a BAR as defined in the spec (0=1MB, 19=512GB).
3024 * Returns true if resizing was successful, false otherwise.
3025 */
3026 int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size)
> 3027 {
3028 unsigned pos, nbars;
3029 u32 ctrl;
3030 unsigned i;
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 6576 bytes --]
[-- Attachment #3: Type: text/plain, Size: 160 bytes --]
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-13 12:41 ` Christian König
(?)
@ 2017-03-15 7:23 ` Ayyappa Ch
-1 siblings, 0 replies; 71+ messages in thread
From: Ayyappa Ch @ 2017-03-15 7:23 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
Is it possible on Carrizo asics? Or only supports on newer asics?
On Mon, Mar 13, 2017 at 6:11 PM, Christian König
<deathsimple@vodafone.de> wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Try to resize BAR0 to let CPU access all of VRAM.
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> 4 files changed, 40 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 3b81ded..905ded9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
> struct ttm_mem_reg *mem);
> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
> int amdgpu_ttm_init(struct amdgpu_device *adev);
> void amdgpu_ttm_fini(struct amdgpu_device *adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 118f4e6..92955fe 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> }
>
> +/**
> + * amdgpu_resize_bar0 - try to resize BAR0
> + *
> + * @adev: amdgpu_device pointer
> + *
> + * Try to resize BAR0 to make all VRAM CPU accessible.
> + */
> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> +{
> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> + int r;
> +
> + r = pci_resize_resource(adev->pdev, 0, size);
> +
> + if (r == -ENOTSUPP) {
> + /* The hardware don't support the extension. */
> + return;
> +
> + } else if (r == -ENOSPC) {
> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> + } else if (r) {
> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> + }
> +
> + /* Reinit the doorbell mapping, it is most likely moved as well */
> + amdgpu_doorbell_fini(adev);
> + BUG_ON(amdgpu_doorbell_init(adev));
> +}
> +
> /*
> * GPU helpers function.
> */
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> index dc9b6d6..36a7aa5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
> break;
> }
> adev->mc.vram_width = numchan * chansize;
> - /* Could aper size report 0 ? */
> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> /* size in MB on si */
> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>
> + if (!(adev->flags & AMD_IS_APU))
> + amdgpu_resize_bar0(adev);
> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> +
> #ifdef CONFIG_X86_64
> if (adev->flags & AMD_IS_APU) {
> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> index c087b00..7761ad3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
> break;
> }
> adev->mc.vram_width = numchan * chansize;
> - /* Could aper size report 0 ? */
> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> /* size in MB on si */
> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>
> + if (!(adev->flags & AMD_IS_APU))
> + amdgpu_resize_bar0(adev);
> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> +
> #ifdef CONFIG_X86_64
> if (adev->flags & AMD_IS_APU) {
> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> --
> 2.7.4
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 7:23 ` Ayyappa Ch
0 siblings, 0 replies; 71+ messages in thread
From: Ayyappa Ch @ 2017-03-15 7:23 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
Is it possible on Carrizo asics? Or only supports on newer asics?
On Mon, Mar 13, 2017 at 6:11 PM, Christian König
<deathsimple@vodafone.de> wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Try to resize BAR0 to let CPU access all of VRAM.
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> 4 files changed, 40 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 3b81ded..905ded9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
> struct ttm_mem_reg *mem);
> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
> int amdgpu_ttm_init(struct amdgpu_device *adev);
> void amdgpu_ttm_fini(struct amdgpu_device *adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 118f4e6..92955fe 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> }
>
> +/**
> + * amdgpu_resize_bar0 - try to resize BAR0
> + *
> + * @adev: amdgpu_device pointer
> + *
> + * Try to resize BAR0 to make all VRAM CPU accessible.
> + */
> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> +{
> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> + int r;
> +
> + r = pci_resize_resource(adev->pdev, 0, size);
> +
> + if (r == -ENOTSUPP) {
> + /* The hardware don't support the extension. */
> + return;
> +
> + } else if (r == -ENOSPC) {
> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> + } else if (r) {
> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> + }
> +
> + /* Reinit the doorbell mapping, it is most likely moved as well */
> + amdgpu_doorbell_fini(adev);
> + BUG_ON(amdgpu_doorbell_init(adev));
> +}
> +
> /*
> * GPU helpers function.
> */
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> index dc9b6d6..36a7aa5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
> break;
> }
> adev->mc.vram_width = numchan * chansize;
> - /* Could aper size report 0 ? */
> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> /* size in MB on si */
> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>
> + if (!(adev->flags & AMD_IS_APU))
> + amdgpu_resize_bar0(adev);
> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> +
> #ifdef CONFIG_X86_64
> if (adev->flags & AMD_IS_APU) {
> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> index c087b00..7761ad3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
> break;
> }
> adev->mc.vram_width = numchan * chansize;
> - /* Could aper size report 0 ? */
> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> /* size in MB on si */
> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>
> + if (!(adev->flags & AMD_IS_APU))
> + amdgpu_resize_bar0(adev);
> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> +
> #ifdef CONFIG_X86_64
> if (adev->flags & AMD_IS_APU) {
> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> --
> 2.7.4
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 7:23 ` Ayyappa Ch
0 siblings, 0 replies; 71+ messages in thread
From: Ayyappa Ch @ 2017-03-15 7:23 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
Is it possible on Carrizo asics? Or only supports on newer asics?
On Mon, Mar 13, 2017 at 6:11 PM, Christian K=C3=B6nig
<deathsimple@vodafone.de> wrote:
> From: Christian K=C3=B6nig <christian.koenig@amd.com>
>
> Try to resize BAR0 to let CPU access all of VRAM.
>
> Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 ++++++++++++++++++++++++=
+++++
> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> 4 files changed, 40 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/am=
dgpu/amdgpu.h
> index 3b81ded..905ded9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_devi=
ce *adev, struct ttm_tt *ttm,
> struct ttm_mem_reg *mem);
> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *=
mc, u64 base);
> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *m=
c);
> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 siz=
e);
> int amdgpu_ttm_init(struct amdgpu_device *adev);
> void amdgpu_ttm_fini(struct amdgpu_device *adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm=
/amd/amdgpu/amdgpu_device.c
> index 118f4e6..92955fe 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev,=
struct amdgpu_mc *mc)
> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> }
>
> +/**
> + * amdgpu_resize_bar0 - try to resize BAR0
> + *
> + * @adev: amdgpu_device pointer
> + *
> + * Try to resize BAR0 to make all VRAM CPU accessible.
> + */
> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> +{
> + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20=
;
> + int r;
> +
> + r =3D pci_resize_resource(adev->pdev, 0, size);
> +
> + if (r =3D=3D -ENOTSUPP) {
> + /* The hardware don't support the extension. */
> + return;
> +
> + } else if (r =3D=3D -ENOSPC) {
> + DRM_INFO("Not enoigh PCI address space for a large BAR.")=
;
> + } else if (r) {
> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> + }
> +
> + /* Reinit the doorbell mapping, it is most likely moved as well *=
/
> + amdgpu_doorbell_fini(adev);
> + BUG_ON(amdgpu_doorbell_init(adev));
> +}
> +
> /*
> * GPU helpers function.
> */
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/=
amdgpu/gmc_v7_0.c
> index dc9b6d6..36a7aa5 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *a=
dev)
> break;
> }
> adev->mc.vram_width =3D numchan * chansize;
> - /* Could aper size report 0 ? */
> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
> /* size in MB on si */
> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 10=
24ULL;
> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * =
1024ULL;
>
> + if (!(adev->flags & AMD_IS_APU))
> + amdgpu_resize_bar0(adev);
> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
> +
> #ifdef CONFIG_X86_64
> if (adev->flags & AMD_IS_APU) {
> adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)) <=
< 22;
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/=
amdgpu/gmc_v8_0.c
> index c087b00..7761ad3 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *a=
dev)
> break;
> }
> adev->mc.vram_width =3D numchan * chansize;
> - /* Could aper size report 0 ? */
> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
> /* size in MB on si */
> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 10=
24ULL;
> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL * =
1024ULL;
>
> + if (!(adev->flags & AMD_IS_APU))
> + amdgpu_resize_bar0(adev);
> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
> +
> #ifdef CONFIG_X86_64
> if (adev->flags & AMD_IS_APU) {
> adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)) <=
< 22;
> --
> 2.7.4
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-15 7:23 ` Ayyappa Ch
@ 2017-03-15 7:37 ` Christian König
-1 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 7:37 UTC (permalink / raw)
To: Ayyappa Ch
Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
Carizzo is an APU and resizing BARs isn't needed nor supported there.
The CPU can access the full stolen VRAM directly on that hardware.
As far as I know ASICs with support for this are Tonga, Fiji and all
Polaris variants.
Christian.
Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> Is it possible on Carrizo asics? Or only supports on newer asics?
>
> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>> From: Christian König <christian.koenig@amd.com>
>>
>> Try to resize BAR0 to let CPU access all of VRAM.
>>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index 3b81ded..905ded9 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>> struct ttm_mem_reg *mem);
>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 118f4e6..92955fe 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>> }
>>
>> +/**
>> + * amdgpu_resize_bar0 - try to resize BAR0
>> + *
>> + * @adev: amdgpu_device pointer
>> + *
>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> + */
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>> +{
>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>> + int r;
>> +
>> + r = pci_resize_resource(adev->pdev, 0, size);
>> +
>> + if (r == -ENOTSUPP) {
>> + /* The hardware don't support the extension. */
>> + return;
>> +
>> + } else if (r == -ENOSPC) {
>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>> + } else if (r) {
>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> + }
>> +
>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>> + amdgpu_doorbell_fini(adev);
>> + BUG_ON(amdgpu_doorbell_init(adev));
>> +}
>> +
>> /*
>> * GPU helpers function.
>> */
>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> index dc9b6d6..36a7aa5 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> index c087b00..7761ad3 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> --
>> 2.7.4
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 7:37 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 7:37 UTC (permalink / raw)
To: Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
Carizzo is an APU and resizing BARs isn't needed nor supported there.
The CPU can access the full stolen VRAM directly on that hardware.
As far as I know ASICs with support for this are Tonga, Fiji and all
Polaris variants.
Christian.
Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> Is it possible on Carrizo asics? Or only supports on newer asics?
>
> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>> From: Christian König <christian.koenig@amd.com>
>>
>> Try to resize BAR0 to let CPU access all of VRAM.
>>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index 3b81ded..905ded9 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>> struct ttm_mem_reg *mem);
>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 118f4e6..92955fe 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>> }
>>
>> +/**
>> + * amdgpu_resize_bar0 - try to resize BAR0
>> + *
>> + * @adev: amdgpu_device pointer
>> + *
>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> + */
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>> +{
>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>> + int r;
>> +
>> + r = pci_resize_resource(adev->pdev, 0, size);
>> +
>> + if (r == -ENOTSUPP) {
>> + /* The hardware don't support the extension. */
>> + return;
>> +
>> + } else if (r == -ENOSPC) {
>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>> + } else if (r) {
>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> + }
>> +
>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>> + amdgpu_doorbell_fini(adev);
>> + BUG_ON(amdgpu_doorbell_init(adev));
>> +}
>> +
>> /*
>> * GPU helpers function.
>> */
>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> index dc9b6d6..36a7aa5 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> index c087b00..7761ad3 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> --
>> 2.7.4
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 8:25 ` Zhou, David(ChunMing)
0 siblings, 0 replies; 71+ messages in thread
From: Zhou, David(ChunMing) @ 2017-03-15 8:25 UTC (permalink / raw)
To: Christian König, Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
Does that means we don't need invisible vram later?
David
-----Original Message-----
From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig
Sent: Wednesday, March 15, 2017 3:38 PM
To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org
Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
Carizzo is an APU and resizing BARs isn't needed nor supported there.
The CPU can access the full stolen VRAM directly on that hardware.
As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
Christian.
Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> Is it possible on Carrizo asics? Or only supports on newer asics?
>
> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>> From: Christian König <christian.koenig@amd.com>
>>
>> Try to resize BAR0 to let CPU access all of VRAM.
>>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index 3b81ded..905ded9 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>> struct ttm_mem_reg *mem);
>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>> amdgpu_mc *mc);
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 118f4e6..92955fe 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>> }
>>
>> +/**
>> + * amdgpu_resize_bar0 - try to resize BAR0
>> + *
>> + * @adev: amdgpu_device pointer
>> + *
>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> + */
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>> + int r;
>> +
>> + r = pci_resize_resource(adev->pdev, 0, size);
>> +
>> + if (r == -ENOTSUPP) {
>> + /* The hardware don't support the extension. */
>> + return;
>> +
>> + } else if (r == -ENOSPC) {
>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>> + } else if (r) {
>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> + }
>> +
>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>> + amdgpu_doorbell_fini(adev);
>> + BUG_ON(amdgpu_doorbell_init(adev));
>> +}
>> +
>> /*
>> * GPU helpers function.
>> */
>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> index dc9b6d6..36a7aa5 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>> * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base =
>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> index c087b00..7761ad3 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>> * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base =
>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> --
>> 2.7.4
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 8:25 ` Zhou, David(ChunMing)
0 siblings, 0 replies; 71+ messages in thread
From: Zhou, David(ChunMing) @ 2017-03-15 8:25 UTC (permalink / raw)
To: Christian König, Ayyappa Ch
Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA,
helgaas-DgEjT+Ai2ygdnm+yROfE0A,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
Does that means we don't need invisible vram later?
David
-----Original Message-----
From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig
Sent: Wednesday, March 15, 2017 3:38 PM
To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org
Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
Carizzo is an APU and resizing BARs isn't needed nor supported there.
The CPU can access the full stolen VRAM directly on that hardware.
As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
Christian.
Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> Is it possible on Carrizo asics? Or only supports on newer asics?
>
> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>> From: Christian König <christian.koenig@amd.com>
>>
>> Try to resize BAR0 to let CPU access all of VRAM.
>>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> ---
>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index 3b81ded..905ded9 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>> struct ttm_mem_reg *mem);
>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>> amdgpu_mc *mc);
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 118f4e6..92955fe 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>> }
>>
>> +/**
>> + * amdgpu_resize_bar0 - try to resize BAR0
>> + *
>> + * @adev: amdgpu_device pointer
>> + *
>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> + */
>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>> + int r;
>> +
>> + r = pci_resize_resource(adev->pdev, 0, size);
>> +
>> + if (r == -ENOTSUPP) {
>> + /* The hardware don't support the extension. */
>> + return;
>> +
>> + } else if (r == -ENOSPC) {
>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>> + } else if (r) {
>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> + }
>> +
>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>> + amdgpu_doorbell_fini(adev);
>> + BUG_ON(amdgpu_doorbell_init(adev));
>> +}
>> +
>> /*
>> * GPU helpers function.
>> */
>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> index dc9b6d6..36a7aa5 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>> * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base =
>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> index c087b00..7761ad3 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>> break;
>> }
>> adev->mc.vram_width = numchan * chansize;
>> - /* Could aper size report 0 ? */
>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> /* size in MB on si */
>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>> * 1024ULL;
>>
>> + if (!(adev->flags & AMD_IS_APU))
>> + amdgpu_resize_bar0(adev);
>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> +
>> #ifdef CONFIG_X86_64
>> if (adev->flags & AMD_IS_APU) {
>> adev->mc.aper_base =
>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> --
>> 2.7.4
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 8:25 ` Zhou, David(ChunMing)
0 siblings, 0 replies; 71+ messages in thread
From: Zhou, David(ChunMing) @ 2017-03-15 8:25 UTC (permalink / raw)
To: Christian König, Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
RG9lcyB0aGF0IG1lYW5zIHdlIGRvbid0IG5lZWQgaW52aXNpYmxlIHZyYW0gbGF0ZXI/DQoNCkRh
dmlkDQoNCi0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQpGcm9tOiBkcmktZGV2ZWwgW21haWx0
bzpkcmktZGV2ZWwtYm91bmNlc0BsaXN0cy5mcmVlZGVza3RvcC5vcmddIE9uIEJlaGFsZiBPZiBD
aHJpc3RpYW4gSz9uaWcNClNlbnQ6IFdlZG5lc2RheSwgTWFyY2ggMTUsIDIwMTcgMzozOCBQTQ0K
VG86IEF5eWFwcGEgQ2ggPGF5eWFwcGEuY2gubGludXhAZ21haWwuY29tPg0KQ2M6IGxpbnV4LXBj
aUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7IGFtZC1nZnhA
bGlzdHMuZnJlZWRlc2t0b3Aub3JnOyBwbGF0Zm9ybS1kcml2ZXIteDg2QHZnZXIua2VybmVsLm9y
ZzsgaGVsZ2Fhc0BrZXJuZWwub3JnOyBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQpT
dWJqZWN0OiBSZTogW1BBVENIIDQvNF0gZHJtL2FtZGdwdTogcmVzaXplIFZSQU0gQkFSIGZvciBD
UFUgYWNjZXNzDQoNCkNhcml6em8gaXMgYW4gQVBVIGFuZCByZXNpemluZyBCQVJzIGlzbid0IG5l
ZWRlZCBub3Igc3VwcG9ydGVkIHRoZXJlLiANClRoZSBDUFUgY2FuIGFjY2VzcyB0aGUgZnVsbCBz
dG9sZW4gVlJBTSBkaXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KDQpBcyBmYXIgYXMgSSBrbm93
IEFTSUNzIHdpdGggc3VwcG9ydCBmb3IgdGhpcyBhcmUgVG9uZ2EsIEZpamkgYW5kIGFsbCBQb2xh
cmlzIHZhcmlhbnRzLg0KDQpDaHJpc3RpYW4uDQoNCkFtIDE1LjAzLjIwMTcgdW0gMDg6MjMgc2No
cmllYiBBeXlhcHBhIENoOg0KPiBJcyBpdCBwb3NzaWJsZSBvbiBDYXJyaXpvIGFzaWNzPyBPciBv
bmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNzPw0KPg0KPiBPbiBNb24sIE1hciAxMywgMjAxNyBh
dCA2OjExIFBNLCBDaHJpc3RpYW4gS8O2bmlnIA0KPiA8ZGVhdGhzaW1wbGVAdm9kYWZvbmUuZGU+
IHdyb3RlOg0KPj4gRnJvbTogQ2hyaXN0aWFuIEvDtm5pZyA8Y2hyaXN0aWFuLmtvZW5pZ0BhbWQu
Y29tPg0KPj4NCj4+IFRyeSB0byByZXNpemUgQkFSMCB0byBsZXQgQ1BVIGFjY2VzcyBhbGwgb2Yg
VlJBTS4NCj4+DQo+PiBTaWduZWQtb2ZmLWJ5OiBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4u
a29lbmlnQGFtZC5jb20+DQo+PiAtLS0NCj4+ICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv
YW1kZ3B1LmggICAgICAgIHwgIDEgKw0KPj4gICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9h
bWRncHVfZGV2aWNlLmMgfCAyOSArKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPj4gICBk
cml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjdfMC5jICAgICAgfCAgOCArKysrKy0tLQ0K
Pj4gICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jICAgICAgfCAgOCArKysr
Ky0tLQ0KPj4gICA0IGZpbGVzIGNoYW5nZWQsIDQwIGluc2VydGlvbnMoKyksIDYgZGVsZXRpb25z
KC0pDQo+Pg0KPj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdw
dS5oIA0KPj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPj4gaW5kZXgg
M2I4MWRlZC4uOTA1ZGVkOSAxMDA2NDQNCj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k
Z3B1L2FtZGdwdS5oDQo+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUu
aA0KPj4gQEAgLTE3MTksNiArMTcxOSw3IEBAIHVpbnQ2NF90IGFtZGdwdV90dG1fdHRfcHRlX2Zs
YWdzKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LCBzdHJ1Y3QgdHRtX3R0ICp0dG0sDQo+PiAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0IHR0bV9tZW1fcmVnICptZW0p
Ow0KPj4gICB2b2lkIGFtZGdwdV92cmFtX2xvY2F0aW9uKHN0cnVjdCBhbWRncHVfZGV2aWNlICph
ZGV2LCBzdHJ1Y3QgYW1kZ3B1X21jICptYywgdTY0IGJhc2UpOw0KPj4gICB2b2lkIGFtZGdwdV9n
dHRfbG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdCANCj4+IGFtZGdw
dV9tYyAqbWMpOw0KPj4gK3ZvaWQgYW1kZ3B1X3Jlc2l6ZV9iYXIwKHN0cnVjdCBhbWRncHVfZGV2
aWNlICphZGV2KTsNCj4+ICAgdm9pZCBhbWRncHVfdHRtX3NldF9hY3RpdmVfdnJhbV9zaXplKHN0
cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LCB1NjQgc2l6ZSk7DQo+PiAgIGludCBhbWRncHVfdHRt
X2luaXQoc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYpOw0KPj4gICB2b2lkIGFtZGdwdV90dG1f
ZmluaShzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldik7IGRpZmYgLS1naXQgDQo+PiBhL2RyaXZl
cnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9kZXZpY2UuYyANCj4+IGIvZHJpdmVycy9ncHUv
ZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+PiBpbmRleCAxMThmNGU2Li45Mjk1NWZl
IDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2Rldmlj
ZS5jDQo+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMN
Cj4+IEBAIC02OTIsNiArNjkyLDM1IEBAIHZvaWQgYW1kZ3B1X2d0dF9sb2NhdGlvbihzdHJ1Y3Qg
YW1kZ3B1X2RldmljZSAqYWRldiwgc3RydWN0IGFtZGdwdV9tYyAqbWMpDQo+PiAgICAgICAgICAg
ICAgICAgICAgICAgICAgbWMtPmd0dF9zaXplID4+IDIwLCBtYy0+Z3R0X3N0YXJ0LCBtYy0+Z3R0
X2VuZCk7DQo+PiAgIH0NCj4+DQo+PiArLyoqDQo+PiArICogYW1kZ3B1X3Jlc2l6ZV9iYXIwIC0g
dHJ5IHRvIHJlc2l6ZSBCQVIwDQo+PiArICoNCj4+ICsgKiBAYWRldjogYW1kZ3B1X2RldmljZSBw
b2ludGVyDQo+PiArICoNCj4+ICsgKiBUcnkgdG8gcmVzaXplIEJBUjAgdG8gbWFrZSBhbGwgVlJB
TSBDUFUgYWNjZXNzaWJsZS4NCj4+ICsgKi8NCj4+ICt2b2lkIGFtZGdwdV9yZXNpemVfYmFyMChz
dHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldikgew0KPj4gKyAgICAgICB1MzIgc2l6ZSA9IG1heChp
bG9nMihhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSAtIDEpICsgMSwgMjApIC0gMjA7DQo+PiArICAg
ICAgIGludCByOw0KPj4gKw0KPj4gKyAgICAgICByID0gcGNpX3Jlc2l6ZV9yZXNvdXJjZShhZGV2
LT5wZGV2LCAwLCBzaXplKTsNCj4+ICsNCj4+ICsgICAgICAgaWYgKHIgPT0gLUVOT1RTVVBQKSB7
DQo+PiArICAgICAgICAgICAgICAgLyogVGhlIGhhcmR3YXJlIGRvbid0IHN1cHBvcnQgdGhlIGV4
dGVuc2lvbi4gKi8NCj4+ICsgICAgICAgICAgICAgICByZXR1cm47DQo+PiArDQo+PiArICAgICAg
IH0gZWxzZSBpZiAociA9PSAtRU5PU1BDKSB7DQo+PiArICAgICAgICAgICAgICAgRFJNX0lORk8o
Ik5vdCBlbm9pZ2ggUENJIGFkZHJlc3Mgc3BhY2UgZm9yIGEgbGFyZ2UgQkFSLiIpOw0KPj4gKyAg
ICAgICB9IGVsc2UgaWYgKHIpIHsNCj4+ICsgICAgICAgICAgICAgICBEUk1fRVJST1IoIlByb2Js
ZW0gcmVzaXppbmcgQkFSMCAoJWQpLiIsIHIpOw0KPj4gKyAgICAgICB9DQo+PiArDQo+PiArICAg
ICAgIC8qIFJlaW5pdCB0aGUgZG9vcmJlbGwgbWFwcGluZywgaXQgaXMgbW9zdCBsaWtlbHkgbW92
ZWQgYXMgd2VsbCAqLw0KPj4gKyAgICAgICBhbWRncHVfZG9vcmJlbGxfZmluaShhZGV2KTsNCj4+
ICsgICAgICAgQlVHX09OKGFtZGdwdV9kb29yYmVsbF9pbml0KGFkZXYpKTsNCj4+ICt9DQo+PiAr
DQo+PiAgIC8qDQo+PiAgICAqIEdQVSBoZWxwZXJzIGZ1bmN0aW9uLg0KPj4gICAgKi8NCj4+IGRp
ZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjdfMC5jIA0KPj4gYi9k
cml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjdfMC5jDQo+PiBpbmRleCBkYzliNmQ2Li4z
NmE3YWE1IDEwMDY0NA0KPj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3
XzAuYw0KPj4gKysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0KPj4g
QEAgLTM2NywxMyArMzY3LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y3XzBfbWNfaW5pdChzdHJ1Y3Qg
YW1kZ3B1X2RldmljZSAqYWRldikNCj4+ICAgICAgICAgICAgICAgICAgYnJlYWs7DQo+PiAgICAg
ICAgICB9DQo+PiAgICAgICAgICBhZGV2LT5tYy52cmFtX3dpZHRoID0gbnVtY2hhbiAqIGNoYW5z
aXplOw0KPj4gLSAgICAgICAvKiBDb3VsZCBhcGVyIHNpemUgcmVwb3J0IDAgPyAqLw0KPj4gLSAg
ICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwg
MCk7DQo+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRl
di0+cGRldiwgMCk7DQo+PiAgICAgICAgICAvKiBzaXplIGluIE1CIG9uIHNpICovDQo+PiAgICAg
ICAgICBhZGV2LT5tYy5tY192cmFtX3NpemUgPSBSUkVHMzIobW1DT05GSUdfTUVNU0laRSkgKiAx
MDI0VUxMICogMTAyNFVMTDsNCj4+ICAgICAgICAgIGFkZXYtPm1jLnJlYWxfdnJhbV9zaXplID0g
UlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICogMTAyNFVMTCANCj4+ICogMTAyNFVMTDsNCj4+DQo+
PiArICAgICAgIGlmICghKGFkZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkpDQo+PiArICAgICAgICAg
ICAgICAgYW1kZ3B1X3Jlc2l6ZV9iYXIwKGFkZXYpOw0KPj4gKyAgICAgICBhZGV2LT5tYy5hcGVy
X2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCk7DQo+PiArICAgICAgIGFk
ZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+PiAr
DQo+PiAgICNpZmRlZiBDT05GSUdfWDg2XzY0DQo+PiAgICAgICAgICBpZiAoYWRldi0+ZmxhZ3Mg
JiBBTURfSVNfQVBVKSB7DQo+PiAgICAgICAgICAgICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9
IA0KPj4gKCh1NjQpUlJFRzMyKG1tTUNfVk1fRkJfT0ZGU0VUKSkgPDwgMjI7IGRpZmYgLS1naXQg
DQo+PiBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMgDQo+PiBiL2RyaXZl
cnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMNCj4+IGluZGV4IGMwODdiMDAuLjc3NjFh
ZDMgMTAwNjQ0DQo+PiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5j
DQo+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+PiBAQCAt
NDU5LDEzICs0NTksMTUgQEAgc3RhdGljIGludCBnbWNfdjhfMF9tY19pbml0KHN0cnVjdCBhbWRn
cHVfZGV2aWNlICphZGV2KQ0KPj4gICAgICAgICAgICAgICAgICBicmVhazsNCj4+ICAgICAgICAg
IH0NCj4+ICAgICAgICAgIGFkZXYtPm1jLnZyYW1fd2lkdGggPSBudW1jaGFuICogY2hhbnNpemU7
DQo+PiAtICAgICAgIC8qIENvdWxkIGFwZXIgc2l6ZSByZXBvcnQgMCA/ICovDQo+PiAtICAgICAg
IGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsN
Cj4+IC0gICAgICAgYWRldi0+bWMuYXBlcl9zaXplID0gcGNpX3Jlc291cmNlX2xlbihhZGV2LT5w
ZGV2LCAwKTsNCj4+ICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24gc2kgKi8NCj4+ICAgICAgICAg
IGFkZXYtPm1jLm1jX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqIDEwMjRV
TEwgKiAxMDI0VUxMOw0KPj4gICAgICAgICAgYWRldi0+bWMucmVhbF92cmFtX3NpemUgPSBSUkVH
MzIobW1DT05GSUdfTUVNU0laRSkgKiAxMDI0VUxMIA0KPj4gKiAxMDI0VUxMOw0KPj4NCj4+ICsg
ICAgICAgaWYgKCEoYWRldi0+ZmxhZ3MgJiBBTURfSVNfQVBVKSkNCj4+ICsgICAgICAgICAgICAg
ICBhbWRncHVfcmVzaXplX2JhcjAoYWRldik7DQo+PiArICAgICAgIGFkZXYtPm1jLmFwZXJfYmFz
ZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsNCj4+ICsgICAgICAgYWRldi0+
bWMuYXBlcl9zaXplID0gcGNpX3Jlc291cmNlX2xlbihhZGV2LT5wZGV2LCAwKTsNCj4+ICsNCj4+
ICAgI2lmZGVmIENPTkZJR19YODZfNjQNCj4+ICAgICAgICAgIGlmIChhZGV2LT5mbGFncyAmIEFN
RF9JU19BUFUpIHsNCj4+ICAgICAgICAgICAgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gDQo+
PiAoKHU2NClSUkVHMzIobW1NQ19WTV9GQl9PRkZTRVQpKSA8PCAyMjsNCj4+IC0tDQo+PiAyLjcu
NA0KPj4NCj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
DQo+PiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+PiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0
b3Aub3JnDQo+PiBodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZv
L2RyaS1kZXZlbA0KDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fDQpkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQpkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0
b3Aub3JnDQpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2Ry
aS1kZXZlbA0K
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-15 8:25 ` Zhou, David(ChunMing)
(?)
@ 2017-03-15 9:29 ` Christian König
-1 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 9:29 UTC (permalink / raw)
To: Zhou, David(ChunMing), Ayyappa Ch
Cc: linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas,
amd-gfx
Yes, exactly that.
Christian.
Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> Does that means we don't need invisible vram later?
>
> David
>
> -----Original Message-----
> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig
> Sent: Wednesday, March 15, 2017 3:38 PM
> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Carizzo is an APU and resizing BARs isn't needed nor supported there.
> The CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
>
> Christian.
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>
>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> <deathsimple@vodafone.de> wrote:
>>> From: Christian König <christian.koenig@amd.com>
>>>
>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>
>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index 3b81ded..905ded9 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>>> struct ttm_mem_reg *mem);
>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>>> amdgpu_mc *mc);
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 118f4e6..92955fe 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>> }
>>>
>>> +/**
>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>> + *
>>> + * @adev: amdgpu_device pointer
>>> + *
>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>> + */
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>> + int r;
>>> +
>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>> +
>>> + if (r == -ENOTSUPP) {
>>> + /* The hardware don't support the extension. */
>>> + return;
>>> +
>>> + } else if (r == -ENOSPC) {
>>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>>> + } else if (r) {
>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>> + }
>>> +
>>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>>> + amdgpu_doorbell_fini(adev);
>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>> +}
>>> +
>>> /*
>>> * GPU helpers function.
>>> */
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> index dc9b6d6..36a7aa5 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>>> * 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =
>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> index c087b00..7761ad3 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>>> * 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =
>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 9:29 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 9:29 UTC (permalink / raw)
To: Zhou, David(ChunMing), Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
Yes, exactly that.
Christian.
Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> Does that means we don't need invisible vram later?
>
> David
>
> -----Original Message-----
> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig
> Sent: Wednesday, March 15, 2017 3:38 PM
> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Carizzo is an APU and resizing BARs isn't needed nor supported there.
> The CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
>
> Christian.
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>
>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> <deathsimple@vodafone.de> wrote:
>>> From: Christian König <christian.koenig@amd.com>
>>>
>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>
>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index 3b81ded..905ded9 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>>> struct ttm_mem_reg *mem);
>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>>> amdgpu_mc *mc);
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 118f4e6..92955fe 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>> }
>>>
>>> +/**
>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>> + *
>>> + * @adev: amdgpu_device pointer
>>> + *
>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>> + */
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>> + int r;
>>> +
>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>> +
>>> + if (r == -ENOTSUPP) {
>>> + /* The hardware don't support the extension. */
>>> + return;
>>> +
>>> + } else if (r == -ENOSPC) {
>>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>>> + } else if (r) {
>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>> + }
>>> +
>>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>>> + amdgpu_doorbell_fini(adev);
>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>> +}
>>> +
>>> /*
>>> * GPU helpers function.
>>> */
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> index dc9b6d6..36a7aa5 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>>> * 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =
>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> index c087b00..7761ad3 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>>> * 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =
>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 9:29 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 9:29 UTC (permalink / raw)
To: Zhou, David(ChunMing), Ayyappa Ch
Cc: linux-pci, linux-kernel, dri-devel, platform-driver-x86, helgaas,
amd-gfx
Yes, exactly that.
Christian.
Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> Does that means we don't need invisible vram later?
>
> David
>
> -----Original Message-----
> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of Christian K?nig
> Sent: Wednesday, March 15, 2017 3:38 PM
> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Carizzo is an APU and resizing BARs isn't needed nor supported there.
> The CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
>
> Christian.
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>
>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> <deathsimple@vodafone.de> wrote:
>>> From: Christian König <christian.koenig@amd.com>
>>>
>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>
>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index 3b81ded..905ded9 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>>> struct ttm_mem_reg *mem);
>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>>> amdgpu_mc *mc);
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 118f4e6..92955fe 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>> }
>>>
>>> +/**
>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>> + *
>>> + * @adev: amdgpu_device pointer
>>> + *
>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>> + */
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>> + int r;
>>> +
>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>> +
>>> + if (r == -ENOTSUPP) {
>>> + /* The hardware don't support the extension. */
>>> + return;
>>> +
>>> + } else if (r == -ENOSPC) {
>>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>>> + } else if (r) {
>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>> + }
>>> +
>>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>>> + amdgpu_doorbell_fini(adev);
>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>> +}
>>> +
>>> /*
>>> * GPU helpers function.
>>> */
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> index dc9b6d6..36a7aa5 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>>> * 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =
>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> index c087b00..7761ad3 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL * 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL
>>> * 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =
>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-15 7:37 ` Christian König
(?)
@ 2017-03-15 10:42 ` Ayyappa Ch
-1 siblings, 0 replies; 71+ messages in thread
From: Ayyappa Ch @ 2017-03-15 10:42 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
It also needs any support from VBIOS side ? I mean PCIe large bar support?
Thanks,
Ayyappa.
On Wed, Mar 15, 2017 at 1:07 PM, Christian König
<deathsimple@vodafone.de> wrote:
> Carizzo is an APU and resizing BARs isn't needed nor supported there. The
> CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
> variants.
>
> Christian.
>
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>
>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>
>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> <deathsimple@vodafone.de> wrote:
>>>
>>> From: Christian König <christian.koenig@amd.com>
>>>
>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>
>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>> +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index 3b81ded..905ded9 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>> struct ttm_mem_reg *mem);
>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc
>>> *mc, u64 base);
>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc
>>> *mc);
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64
>>> size);
>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 118f4e6..92955fe 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev,
>>> struct amdgpu_mc *mc)
>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>> }
>>>
>>> +/**
>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>> + *
>>> + * @adev: amdgpu_device pointer
>>> + *
>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>> + */
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>>> +{
>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>> + int r;
>>> +
>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>> +
>>> + if (r == -ENOTSUPP) {
>>> + /* The hardware don't support the extension. */
>>> + return;
>>> +
>>> + } else if (r == -ENOSPC) {
>>> + DRM_INFO("Not enoigh PCI address space for a large
>>> BAR.");
>>> + } else if (r) {
>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>> + }
>>> +
>>> + /* Reinit the doorbell mapping, it is most likely moved as well
>>> */
>>> + amdgpu_doorbell_fini(adev);
>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>> +}
>>> +
>>> /*
>>> * GPU helpers function.
>>> */
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> index dc9b6d6..36a7aa5 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>>> *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>> 22;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> index c087b00..7761ad3 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>>> *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>> 22;
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
>
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 10:42 ` Ayyappa Ch
0 siblings, 0 replies; 71+ messages in thread
From: Ayyappa Ch @ 2017-03-15 10:42 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
It also needs any support from VBIOS side ? I mean PCIe large bar support?
Thanks,
Ayyappa.
On Wed, Mar 15, 2017 at 1:07 PM, Christian König
<deathsimple@vodafone.de> wrote:
> Carizzo is an APU and resizing BARs isn't needed nor supported there. The
> CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
> variants.
>
> Christian.
>
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>
>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>
>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> <deathsimple@vodafone.de> wrote:
>>>
>>> From: Christian König <christian.koenig@amd.com>
>>>
>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>
>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>> +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index 3b81ded..905ded9 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>> struct ttm_mem_reg *mem);
>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc
>>> *mc, u64 base);
>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc
>>> *mc);
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64
>>> size);
>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 118f4e6..92955fe 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev,
>>> struct amdgpu_mc *mc)
>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>> }
>>>
>>> +/**
>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>> + *
>>> + * @adev: amdgpu_device pointer
>>> + *
>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>> + */
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>>> +{
>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>> + int r;
>>> +
>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>> +
>>> + if (r == -ENOTSUPP) {
>>> + /* The hardware don't support the extension. */
>>> + return;
>>> +
>>> + } else if (r == -ENOSPC) {
>>> + DRM_INFO("Not enoigh PCI address space for a large
>>> BAR.");
>>> + } else if (r) {
>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>> + }
>>> +
>>> + /* Reinit the doorbell mapping, it is most likely moved as well
>>> */
>>> + amdgpu_doorbell_fini(adev);
>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>> +}
>>> +
>>> /*
>>> * GPU helpers function.
>>> */
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> index dc9b6d6..36a7aa5 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>>> *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>> 22;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> index c087b00..7761ad3 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>>> *adev)
>>> break;
>>> }
>>> adev->mc.vram_width = numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>> 22;
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
>
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 10:42 ` Ayyappa Ch
0 siblings, 0 replies; 71+ messages in thread
From: Ayyappa Ch @ 2017-03-15 10:42 UTC (permalink / raw)
To: Christian König
Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
It also needs any support from VBIOS side ? I mean PCIe large bar support?
Thanks,
Ayyappa.
On Wed, Mar 15, 2017 at 1:07 PM, Christian K=C3=B6nig
<deathsimple@vodafone.de> wrote:
> Carizzo is an APU and resizing BARs isn't needed nor supported there. The
> CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all Pola=
ris
> variants.
>
> Christian.
>
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>
>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>
>> On Mon, Mar 13, 2017 at 6:11 PM, Christian K=C3=B6nig
>> <deathsimple@vodafone.de> wrote:
>>>
>>> From: Christian K=C3=B6nig <christian.koenig@amd.com>
>>>
>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>
>>> Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com>
>>> ---
>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>> +++++++++++++++++++++++++++++
>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index 3b81ded..905ded9 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>> struct ttm_mem_reg *mem);
>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_m=
c
>>> *mc, u64 base);
>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc
>>> *mc);
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64
>>> size);
>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 118f4e6..92955fe 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *ade=
v,
>>> struct amdgpu_mc *mc)
>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end=
);
>>> }
>>>
>>> +/**
>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>> + *
>>> + * @adev: amdgpu_device pointer
>>> + *
>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>> + */
>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>>> +{
>>> + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - =
20;
>>> + int r;
>>> +
>>> + r =3D pci_resize_resource(adev->pdev, 0, size);
>>> +
>>> + if (r =3D=3D -ENOTSUPP) {
>>> + /* The hardware don't support the extension. */
>>> + return;
>>> +
>>> + } else if (r =3D=3D -ENOSPC) {
>>> + DRM_INFO("Not enoigh PCI address space for a large
>>> BAR.");
>>> + } else if (r) {
>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>> + }
>>> +
>>> + /* Reinit the doorbell mapping, it is most likely moved as well
>>> */
>>> + amdgpu_doorbell_fini(adev);
>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>> +}
>>> +
>>> /*
>>> * GPU helpers function.
>>> */
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> index dc9b6d6..36a7aa5 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>>> *adev)
>>> break;
>>> }
>>> adev->mc.vram_width =3D numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL=
*
>>> 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)=
) <<
>>> 22;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> index c087b00..7761ad3 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>>> *adev)
>>> break;
>>> }
>>> adev->mc.vram_width =3D numchan * chansize;
>>> - /* Could aper size report 0 ? */
>>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>>> /* size in MB on si */
>>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>> 1024ULL;
>>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024ULL=
*
>>> 1024ULL;
>>>
>>> + if (!(adev->flags & AMD_IS_APU))
>>> + amdgpu_resize_bar0(adev);
>>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>>> +
>>> #ifdef CONFIG_X86_64
>>> if (adev->flags & AMD_IS_APU) {
>>> adev->mc.aper_base =3D ((u64)RREG32(mmMC_VM_FB_OFFSET)=
) <<
>>> 22;
>>> --
>>> 2.7.4
>>>
>>> _______________________________________________
>>> dri-devel mailing list
>>> dri-devel@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
>
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-15 10:42 ` Ayyappa Ch
@ 2017-03-15 11:03 ` Christian König
-1 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 11:03 UTC (permalink / raw)
To: Ayyappa Ch
Cc: helgaas, linux-pci, dri-devel, platform-driver-x86, amd-gfx,
linux-kernel
No, we resize the BAR on the fly during driver load without help from
the BIOS or VBIOS.
Christian.
Am 15.03.2017 um 11:42 schrieb Ayyappa Ch:
> It also needs any support from VBIOS side ? I mean PCIe large bar support?
>
> Thanks,
> Ayyappa.
>
> On Wed, Mar 15, 2017 at 1:07 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>> Carizzo is an APU and resizing BARs isn't needed nor supported there. The
>> CPU can access the full stolen VRAM directly on that hardware.
>>
>> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
>> variants.
>>
>> Christian.
>>
>>
>> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>>
>>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>>> <deathsimple@vodafone.de> wrote:
>>>> From: Christian König <christian.koenig@amd.com>
>>>>
>>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>>
>>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>>> ---
>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>>> +++++++++++++++++++++++++++++
>>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> index 3b81ded..905ded9 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>>> struct ttm_mem_reg *mem);
>>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc
>>>> *mc, u64 base);
>>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc
>>>> *mc);
>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64
>>>> size);
>>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> index 118f4e6..92955fe 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev,
>>>> struct amdgpu_mc *mc)
>>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>>> }
>>>>
>>>> +/**
>>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>>> + *
>>>> + * @adev: amdgpu_device pointer
>>>> + *
>>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>>> + */
>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>>>> +{
>>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>>> + int r;
>>>> +
>>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>>> +
>>>> + if (r == -ENOTSUPP) {
>>>> + /* The hardware don't support the extension. */
>>>> + return;
>>>> +
>>>> + } else if (r == -ENOSPC) {
>>>> + DRM_INFO("Not enoigh PCI address space for a large
>>>> BAR.");
>>>> + } else if (r) {
>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>>> + }
>>>> +
>>>> + /* Reinit the doorbell mapping, it is most likely moved as well
>>>> */
>>>> + amdgpu_doorbell_fini(adev);
>>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>>> +}
>>>> +
>>>> /*
>>>> * GPU helpers function.
>>>> */
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> index dc9b6d6..36a7aa5 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>>>> *adev)
>>>> break;
>>>> }
>>>> adev->mc.vram_width = numchan * chansize;
>>>> - /* Could aper size report 0 ? */
>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> /* size in MB on si */
>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>>
>>>> + if (!(adev->flags & AMD_IS_APU))
>>>> + amdgpu_resize_bar0(adev);
>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>> if (adev->flags & AMD_IS_APU) {
>>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>>> 22;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> index c087b00..7761ad3 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>>>> *adev)
>>>> break;
>>>> }
>>>> adev->mc.vram_width = numchan * chansize;
>>>> - /* Could aper size report 0 ? */
>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> /* size in MB on si */
>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>>
>>>> + if (!(adev->flags & AMD_IS_APU))
>>>> + amdgpu_resize_bar0(adev);
>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>> if (adev->flags & AMD_IS_APU) {
>>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>>> 22;
>>>> --
>>>> 2.7.4
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
>>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 11:03 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-15 11:03 UTC (permalink / raw)
To: Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
No, we resize the BAR on the fly during driver load without help from
the BIOS or VBIOS.
Christian.
Am 15.03.2017 um 11:42 schrieb Ayyappa Ch:
> It also needs any support from VBIOS side ? I mean PCIe large bar support?
>
> Thanks,
> Ayyappa.
>
> On Wed, Mar 15, 2017 at 1:07 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>> Carizzo is an APU and resizing BARs isn't needed nor supported there. The
>> CPU can access the full stolen VRAM directly on that hardware.
>>
>> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
>> variants.
>>
>> Christian.
>>
>>
>> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>>
>>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>>> <deathsimple@vodafone.de> wrote:
>>>> From: Christian König <christian.koenig@amd.com>
>>>>
>>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>>
>>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>>> ---
>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>>> +++++++++++++++++++++++++++++
>>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> index 3b81ded..905ded9 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>>> struct ttm_mem_reg *mem);
>>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc
>>>> *mc, u64 base);
>>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc
>>>> *mc);
>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64
>>>> size);
>>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>>> void amdgpu_ttm_fini(struct amdgpu_device *adev);
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> index 118f4e6..92955fe 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev,
>>>> struct amdgpu_mc *mc)
>>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>>> }
>>>>
>>>> +/**
>>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>>> + *
>>>> + * @adev: amdgpu_device pointer
>>>> + *
>>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>>> + */
>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
>>>> +{
>>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>>> + int r;
>>>> +
>>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>>> +
>>>> + if (r == -ENOTSUPP) {
>>>> + /* The hardware don't support the extension. */
>>>> + return;
>>>> +
>>>> + } else if (r == -ENOSPC) {
>>>> + DRM_INFO("Not enoigh PCI address space for a large
>>>> BAR.");
>>>> + } else if (r) {
>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>>> + }
>>>> +
>>>> + /* Reinit the doorbell mapping, it is most likely moved as well
>>>> */
>>>> + amdgpu_doorbell_fini(adev);
>>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>>> +}
>>>> +
>>>> /*
>>>> * GPU helpers function.
>>>> */
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> index dc9b6d6..36a7aa5 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>>>> *adev)
>>>> break;
>>>> }
>>>> adev->mc.vram_width = numchan * chansize;
>>>> - /* Could aper size report 0 ? */
>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> /* size in MB on si */
>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>>
>>>> + if (!(adev->flags & AMD_IS_APU))
>>>> + amdgpu_resize_bar0(adev);
>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>> if (adev->flags & AMD_IS_APU) {
>>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>>> 22;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> index c087b00..7761ad3 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>>>> *adev)
>>>> break;
>>>> }
>>>> adev->mc.vram_width = numchan * chansize;
>>>> - /* Could aper size report 0 ? */
>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> /* size in MB on si */
>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>>>> 1024ULL;
>>>>
>>>> + if (!(adev->flags & AMD_IS_APU))
>>>> + amdgpu_resize_bar0(adev);
>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>> +
>>>> #ifdef CONFIG_X86_64
>>>> if (adev->flags & AMD_IS_APU) {
>>>> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET)) <<
>>>> 22;
>>>> --
>>>> 2.7.4
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>
>>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 16:08 ` Deucher, Alexander
0 siblings, 0 replies; 71+ messages in thread
From: Deucher, Alexander @ 2017-03-15 16:08 UTC (permalink / raw)
To: 'Christian König', Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
> -----Original Message-----
> From: amd-gfx [mailto:amd-gfx-bounces@lists.freedesktop.org] On Behalf
> Of Christian König
> Sent: Wednesday, March 15, 2017 3:38 AM
> To: Ayyappa Ch
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Carizzo is an APU and resizing BARs isn't needed nor supported there.
> The CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all
> Polaris variants.
I think resizable BARs are supported as far back as evergreen or NI.
Alex
>
> Christian.
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> > Is it possible on Carrizo asics? Or only supports on newer asics?
> >
> > On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> > <deathsimple@vodafone.de> wrote:
> >> From: Christian König <christian.koenig@amd.com>
> >>
> >> Try to resize BAR0 to let CPU access all of VRAM.
> >>
> >> Signed-off-by: Christian König <christian.koenig@amd.com>
> >> ---
> >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> +++++++++++++++++++++++++++++
> >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >> 4 files changed, 40 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> index 3b81ded..905ded9 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> amdgpu_device *adev, struct ttm_tt *ttm,
> >> struct ttm_mem_reg *mem);
> >> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> amdgpu_mc *mc, u64 base);
> >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> amdgpu_mc *mc);
> >> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
> u64 size);
> >> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >> void amdgpu_ttm_fini(struct amdgpu_device *adev);
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> index 118f4e6..92955fe 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
> *adev, struct amdgpu_mc *mc)
> >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >> }
> >>
> >> +/**
> >> + * amdgpu_resize_bar0 - try to resize BAR0
> >> + *
> >> + * @adev: amdgpu_device pointer
> >> + *
> >> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >> + */
> >> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> >> +{
> >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >> + int r;
> >> +
> >> + r = pci_resize_resource(adev->pdev, 0, size);
> >> +
> >> + if (r == -ENOTSUPP) {
> >> + /* The hardware don't support the extension. */
> >> + return;
> >> +
> >> + } else if (r == -ENOSPC) {
> >> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >> + } else if (r) {
> >> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >> + }
> >> +
> >> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >> + amdgpu_doorbell_fini(adev);
> >> + BUG_ON(amdgpu_doorbell_init(adev));
> >> +}
> >> +
> >> /*
> >> * GPU helpers function.
> >> */
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> index dc9b6d6..36a7aa5 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
> amdgpu_device *adev)
> >> break;
> >> }
> >> adev->mc.vram_width = numchan * chansize;
> >> - /* Could aper size report 0 ? */
> >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> /* size in MB on si */
> >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >>
> >> + if (!(adev->flags & AMD_IS_APU))
> >> + amdgpu_resize_bar0(adev);
> >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> +
> >> #ifdef CONFIG_X86_64
> >> if (adev->flags & AMD_IS_APU) {
> >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET))
> << 22;
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> index c087b00..7761ad3 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
> amdgpu_device *adev)
> >> break;
> >> }
> >> adev->mc.vram_width = numchan * chansize;
> >> - /* Could aper size report 0 ? */
> >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> /* size in MB on si */
> >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >>
> >> + if (!(adev->flags & AMD_IS_APU))
> >> + amdgpu_resize_bar0(adev);
> >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> +
> >> #ifdef CONFIG_X86_64
> >> if (adev->flags & AMD_IS_APU) {
> >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET))
> << 22;
> >> --
> >> 2.7.4
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 16:08 ` Deucher, Alexander
0 siblings, 0 replies; 71+ messages in thread
From: Deucher, Alexander @ 2017-03-15 16:08 UTC (permalink / raw)
To: 'Christian König', Ayyappa Ch
Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA,
helgaas-DgEjT+Ai2ygdnm+yROfE0A,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
> -----Original Message-----
> From: amd-gfx [mailto:amd-gfx-bounces@lists.freedesktop.org] On Behalf
> Of Christian König
> Sent: Wednesday, March 15, 2017 3:38 AM
> To: Ayyappa Ch
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Carizzo is an APU and resizing BARs isn't needed nor supported there.
> The CPU can access the full stolen VRAM directly on that hardware.
>
> As far as I know ASICs with support for this are Tonga, Fiji and all
> Polaris variants.
I think resizable BARs are supported as far back as evergreen or NI.
Alex
>
> Christian.
>
> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> > Is it possible on Carrizo asics? Or only supports on newer asics?
> >
> > On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> > <deathsimple@vodafone.de> wrote:
> >> From: Christian König <christian.koenig@amd.com>
> >>
> >> Try to resize BAR0 to let CPU access all of VRAM.
> >>
> >> Signed-off-by: Christian König <christian.koenig@amd.com>
> >> ---
> >> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> +++++++++++++++++++++++++++++
> >> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >> 4 files changed, 40 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> index 3b81ded..905ded9 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> amdgpu_device *adev, struct ttm_tt *ttm,
> >> struct ttm_mem_reg *mem);
> >> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> amdgpu_mc *mc, u64 base);
> >> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> amdgpu_mc *mc);
> >> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
> u64 size);
> >> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >> void amdgpu_ttm_fini(struct amdgpu_device *adev);
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> index 118f4e6..92955fe 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
> *adev, struct amdgpu_mc *mc)
> >> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >> }
> >>
> >> +/**
> >> + * amdgpu_resize_bar0 - try to resize BAR0
> >> + *
> >> + * @adev: amdgpu_device pointer
> >> + *
> >> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >> + */
> >> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> >> +{
> >> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >> + int r;
> >> +
> >> + r = pci_resize_resource(adev->pdev, 0, size);
> >> +
> >> + if (r == -ENOTSUPP) {
> >> + /* The hardware don't support the extension. */
> >> + return;
> >> +
> >> + } else if (r == -ENOSPC) {
> >> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >> + } else if (r) {
> >> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >> + }
> >> +
> >> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >> + amdgpu_doorbell_fini(adev);
> >> + BUG_ON(amdgpu_doorbell_init(adev));
> >> +}
> >> +
> >> /*
> >> * GPU helpers function.
> >> */
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> index dc9b6d6..36a7aa5 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
> amdgpu_device *adev)
> >> break;
> >> }
> >> adev->mc.vram_width = numchan * chansize;
> >> - /* Could aper size report 0 ? */
> >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> /* size in MB on si */
> >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >>
> >> + if (!(adev->flags & AMD_IS_APU))
> >> + amdgpu_resize_bar0(adev);
> >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> +
> >> #ifdef CONFIG_X86_64
> >> if (adev->flags & AMD_IS_APU) {
> >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET))
> << 22;
> >> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> index c087b00..7761ad3 100644
> >> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
> amdgpu_device *adev)
> >> break;
> >> }
> >> adev->mc.vram_width = numchan * chansize;
> >> - /* Could aper size report 0 ? */
> >> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> /* size in MB on si */
> >> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> 1024ULL * 1024ULL;
> >>
> >> + if (!(adev->flags & AMD_IS_APU))
> >> + amdgpu_resize_bar0(adev);
> >> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> +
> >> #ifdef CONFIG_X86_64
> >> if (adev->flags & AMD_IS_APU) {
> >> adev->mc.aper_base = ((u64)RREG32(mmMC_VM_FB_OFFSET))
> << 22;
> >> --
> >> 2.7.4
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-15 16:08 ` Deucher, Alexander
0 siblings, 0 replies; 71+ messages in thread
From: Deucher, Alexander @ 2017-03-15 16:08 UTC (permalink / raw)
To: 'Christian König', Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBhbWQtZ2Z4IFttYWlsdG86YW1k
LWdmeC1ib3VuY2VzQGxpc3RzLmZyZWVkZXNrdG9wLm9yZ10gT24gQmVoYWxmDQo+IE9mIENocmlz
dGlhbiBLw7ZuaWcNCj4gU2VudDogV2VkbmVzZGF5LCBNYXJjaCAxNSwgMjAxNyAzOjM4IEFNDQo+
IFRvOiBBeXlhcHBhIENoDQo+IENjOiBsaW51eC1wY2lAdmdlci5rZXJuZWwub3JnOyBsaW51eC1r
ZXJuZWxAdmdlci5rZXJuZWwub3JnOyBhbWQtDQo+IGdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7
IHBsYXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwub3JnOw0KPiBoZWxnYWFzQGtlcm5lbC5v
cmc7IGRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcNCj4gU3ViamVjdDogUmU6IFtQQVRD
SCA0LzRdIGRybS9hbWRncHU6IHJlc2l6ZSBWUkFNIEJBUiBmb3IgQ1BVIGFjY2Vzcw0KPiANCj4g
Q2FyaXp6byBpcyBhbiBBUFUgYW5kIHJlc2l6aW5nIEJBUnMgaXNuJ3QgbmVlZGVkIG5vciBzdXBw
b3J0ZWQgdGhlcmUuDQo+IFRoZSBDUFUgY2FuIGFjY2VzcyB0aGUgZnVsbCBzdG9sZW4gVlJBTSBk
aXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KPiANCj4gQXMgZmFyIGFzIEkga25vdyBBU0lDcyB3
aXRoIHN1cHBvcnQgZm9yIHRoaXMgYXJlIFRvbmdhLCBGaWppIGFuZCBhbGwNCj4gUG9sYXJpcyB2
YXJpYW50cy4NCg0KSSB0aGluayByZXNpemFibGUgQkFScyBhcmUgc3VwcG9ydGVkIGFzIGZhciBi
YWNrIGFzIGV2ZXJncmVlbiBvciBOSS4NCg0KQWxleA0KDQo+IA0KPiBDaHJpc3RpYW4uDQo+IA0K
PiBBbSAxNS4wMy4yMDE3IHVtIDA4OjIzIHNjaHJpZWIgQXl5YXBwYSBDaDoNCj4gPiBJcyBpdCBw
b3NzaWJsZSBvbiBDYXJyaXpvIGFzaWNzPyBPciBvbmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNz
Pw0KPiA+DQo+ID4gT24gTW9uLCBNYXIgMTMsIDIwMTcgYXQgNjoxMSBQTSwgQ2hyaXN0aWFuIEvD
tm5pZw0KPiA+IDxkZWF0aHNpbXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6DQo+ID4+IEZyb206IENo
cmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4NCj4gPj4NCj4gPj4gVHJ5
IHRvIHJlc2l6ZSBCQVIwIHRvIGxldCBDUFUgYWNjZXNzIGFsbCBvZiBWUkFNLg0KPiA+Pg0KPiA+
PiBTaWduZWQtb2ZmLWJ5OiBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5j
b20+DQo+ID4+IC0tLQ0KPiA+PiAgIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdS5o
ICAgICAgICB8ICAxICsNCj4gPj4gICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVf
ZGV2aWNlLmMgfCAyOQ0KPiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKw0KPiA+PiAgIGRy
aXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMgICAgICB8ICA4ICsrKysrLS0tDQo+
ID4+ICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYyAgICAgIHwgIDggKysr
KystLS0NCj4gPj4gICA0IGZpbGVzIGNoYW5nZWQsIDQwIGluc2VydGlvbnMoKyksIDYgZGVsZXRp
b25zKC0pDQo+ID4+DQo+ID4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdw
dS9hbWRncHUuaA0KPiBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+
IGluZGV4IDNiODFkZWQuLjkwNWRlZDkgMTAwNjQ0DQo+ID4+IC0tLSBhL2RyaXZlcnMvZ3B1L2Ry
bS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k
Z3B1L2FtZGdwdS5oDQo+ID4+IEBAIC0xNzE5LDYgKzE3MTksNyBAQCB1aW50NjRfdCBhbWRncHVf
dHRtX3R0X3B0ZV9mbGFncyhzdHJ1Y3QNCj4gYW1kZ3B1X2RldmljZSAqYWRldiwgc3RydWN0IHR0
bV90dCAqdHRtLA0KPiA+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3RydWN0
IHR0bV9tZW1fcmVnICptZW0pOw0KPiA+PiAgIHZvaWQgYW1kZ3B1X3ZyYW1fbG9jYXRpb24oc3Ry
dWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiBhbWRncHVfbWMgKm1jLCB1NjQgYmFz
ZSk7DQo+ID4+ICAgdm9pZCBhbWRncHVfZ3R0X2xvY2F0aW9uKHN0cnVjdCBhbWRncHVfZGV2aWNl
ICphZGV2LCBzdHJ1Y3QNCj4gYW1kZ3B1X21jICptYyk7DQo+ID4+ICt2b2lkIGFtZGdwdV9yZXNp
emVfYmFyMChzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldik7DQo+ID4+ICAgdm9pZCBhbWRncHVf
dHRtX3NldF9hY3RpdmVfdnJhbV9zaXplKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LA0KPiB1
NjQgc2l6ZSk7DQo+ID4+ICAgaW50IGFtZGdwdV90dG1faW5pdChzdHJ1Y3QgYW1kZ3B1X2Rldmlj
ZSAqYWRldik7DQo+ID4+ICAgdm9pZCBhbWRncHVfdHRtX2Zpbmkoc3RydWN0IGFtZGdwdV9kZXZp
Y2UgKmFkZXYpOw0KPiA+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv
YW1kZ3B1X2RldmljZS5jDQo+IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2Rl
dmljZS5jDQo+ID4+IGluZGV4IDExOGY0ZTYuLjkyOTU1ZmUgMTAwNjQ0DQo+ID4+IC0tLSBhL2Ry
aXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9kZXZpY2UuYw0KPiA+PiArKysgYi9kcml2
ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMNCj4gPj4gQEAgLTY5Miw2ICs2
OTIsMzUgQEAgdm9pZCBhbWRncHVfZ3R0X2xvY2F0aW9uKHN0cnVjdCBhbWRncHVfZGV2aWNlDQo+
ICphZGV2LCBzdHJ1Y3QgYW1kZ3B1X21jICptYykNCj4gPj4gICAgICAgICAgICAgICAgICAgICAg
ICAgIG1jLT5ndHRfc2l6ZSA+PiAyMCwgbWMtPmd0dF9zdGFydCwgbWMtPmd0dF9lbmQpOw0KPiA+
PiAgIH0NCj4gPj4NCj4gPj4gKy8qKg0KPiA+PiArICogYW1kZ3B1X3Jlc2l6ZV9iYXIwIC0gdHJ5
IHRvIHJlc2l6ZSBCQVIwDQo+ID4+ICsgKg0KPiA+PiArICogQGFkZXY6IGFtZGdwdV9kZXZpY2Ug
cG9pbnRlcg0KPiA+PiArICoNCj4gPj4gKyAqIFRyeSB0byByZXNpemUgQkFSMCB0byBtYWtlIGFs
bCBWUkFNIENQVSBhY2Nlc3NpYmxlLg0KPiA+PiArICovDQo+ID4+ICt2b2lkIGFtZGdwdV9yZXNp
emVfYmFyMChzdHJ1Y3QgYW1kZ3B1X2RldmljZSAqYWRldikNCj4gPj4gK3sNCj4gPj4gKyAgICAg
ICB1MzIgc2l6ZSA9IG1heChpbG9nMihhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSAtIDEpICsgMSwg
MjApIC0gMjA7DQo+ID4+ICsgICAgICAgaW50IHI7DQo+ID4+ICsNCj4gPj4gKyAgICAgICByID0g
cGNpX3Jlc2l6ZV9yZXNvdXJjZShhZGV2LT5wZGV2LCAwLCBzaXplKTsNCj4gPj4gKw0KPiA+PiAr
ICAgICAgIGlmIChyID09IC1FTk9UU1VQUCkgew0KPiA+PiArICAgICAgICAgICAgICAgLyogVGhl
IGhhcmR3YXJlIGRvbid0IHN1cHBvcnQgdGhlIGV4dGVuc2lvbi4gKi8NCj4gPj4gKyAgICAgICAg
ICAgICAgIHJldHVybjsNCj4gPj4gKw0KPiA+PiArICAgICAgIH0gZWxzZSBpZiAociA9PSAtRU5P
U1BDKSB7DQo+ID4+ICsgICAgICAgICAgICAgICBEUk1fSU5GTygiTm90IGVub2lnaCBQQ0kgYWRk
cmVzcyBzcGFjZSBmb3IgYSBsYXJnZSBCQVIuIik7DQo+ID4+ICsgICAgICAgfSBlbHNlIGlmIChy
KSB7DQo+ID4+ICsgICAgICAgICAgICAgICBEUk1fRVJST1IoIlByb2JsZW0gcmVzaXppbmcgQkFS
MCAoJWQpLiIsIHIpOw0KPiA+PiArICAgICAgIH0NCj4gPj4gKw0KPiA+PiArICAgICAgIC8qIFJl
aW5pdCB0aGUgZG9vcmJlbGwgbWFwcGluZywgaXQgaXMgbW9zdCBsaWtlbHkgbW92ZWQgYXMgd2Vs
bCAqLw0KPiA+PiArICAgICAgIGFtZGdwdV9kb29yYmVsbF9maW5pKGFkZXYpOw0KPiA+PiArICAg
ICAgIEJVR19PTihhbWRncHVfZG9vcmJlbGxfaW5pdChhZGV2KSk7DQo+ID4+ICt9DQo+ID4+ICsN
Cj4gPj4gICAvKg0KPiA+PiAgICAqIEdQVSBoZWxwZXJzIGZ1bmN0aW9uLg0KPiA+PiAgICAqLw0K
PiA+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0K
PiBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMNCj4gPj4gaW5kZXggZGM5
YjZkNi4uMzZhN2FhNSAxMDA2NDQNCj4gPj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRn
cHUvZ21jX3Y3XzAuYw0KPiA+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNf
djdfMC5jDQo+ID4+IEBAIC0zNjcsMTMgKzM2NywxNSBAQCBzdGF0aWMgaW50IGdtY192N18wX21j
X2luaXQoc3RydWN0DQo+IGFtZGdwdV9kZXZpY2UgKmFkZXYpDQo+ID4+ICAgICAgICAgICAgICAg
ICAgYnJlYWs7DQo+ID4+ICAgICAgICAgIH0NCj4gPj4gICAgICAgICAgYWRldi0+bWMudnJhbV93
aWR0aCA9IG51bWNoYW4gKiBjaGFuc2l6ZTsNCj4gPj4gLSAgICAgICAvKiBDb3VsZCBhcGVyIHNp
emUgcmVwb3J0IDAgPyAqLw0KPiA+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9y
ZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsNCj4gPj4gLSAgICAgICBhZGV2LT5tYy5hcGVy
X3NpemUgPSBwY2lfcmVzb3VyY2VfbGVuKGFkZXYtPnBkZXYsIDApOw0KPiA+PiAgICAgICAgICAv
KiBzaXplIGluIE1CIG9uIHNpICovDQo+ID4+ICAgICAgICAgIGFkZXYtPm1jLm1jX3ZyYW1fc2l6
ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqDQo+IDEwMjRVTEwgKiAxMDI0VUxMOw0KPiA+
PiAgICAgICAgICBhZGV2LT5tYy5yZWFsX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1T
SVpFKSAqDQo+IDEwMjRVTEwgKiAxMDI0VUxMOw0KPiA+Pg0KPiA+PiArICAgICAgIGlmICghKGFk
ZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkpDQo+ID4+ICsgICAgICAgICAgICAgICBhbWRncHVfcmVz
aXplX2JhcjAoYWRldik7DQo+ID4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jl
c291cmNlX3N0YXJ0KGFkZXYtPnBkZXYsIDApOw0KPiA+PiArICAgICAgIGFkZXYtPm1jLmFwZXJf
c2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+ICsNCj4gPj4gICAj
aWZkZWYgQ09ORklHX1g4Nl82NA0KPiA+PiAgICAgICAgICBpZiAoYWRldi0+ZmxhZ3MgJiBBTURf
SVNfQVBVKSB7DQo+ID4+ICAgICAgICAgICAgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gKCh1
NjQpUlJFRzMyKG1tTUNfVk1fRkJfT0ZGU0VUKSkNCj4gPDwgMjI7DQo+ID4+IGRpZmYgLS1naXQg
YS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+IGIvZHJpdmVycy9ncHUv
ZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYw0KPiA+PiBpbmRleCBjMDg3YjAwLi43NzYxYWQzIDEw
MDY0NA0KPiA+PiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+
ID4+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMNCj4gPj4gQEAg
LTQ1OSwxMyArNDU5LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y4XzBfbWNfaW5pdChzdHJ1Y3QNCj4g
YW1kZ3B1X2RldmljZSAqYWRldikNCj4gPj4gICAgICAgICAgICAgICAgICBicmVhazsNCj4gPj4g
ICAgICAgICAgfQ0KPiA+PiAgICAgICAgICBhZGV2LT5tYy52cmFtX3dpZHRoID0gbnVtY2hhbiAq
IGNoYW5zaXplOw0KPiA+PiAtICAgICAgIC8qIENvdWxkIGFwZXIgc2l6ZSByZXBvcnQgMCA/ICov
DQo+ID4+IC0gICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jlc291cmNlX3N0YXJ0KGFk
ZXYtPnBkZXYsIDApOw0KPiA+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNv
dXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+ICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24g
c2kgKi8NCj4gPj4gICAgICAgICAgYWRldi0+bWMubWNfdnJhbV9zaXplID0gUlJFRzMyKG1tQ09O
RklHX01FTVNJWkUpICoNCj4gMTAyNFVMTCAqIDEwMjRVTEw7DQo+ID4+ICAgICAgICAgIGFkZXYt
Pm1jLnJlYWxfdnJhbV9zaXplID0gUlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gMTAyNFVM
TCAqIDEwMjRVTEw7DQo+ID4+DQo+ID4+ICsgICAgICAgaWYgKCEoYWRldi0+ZmxhZ3MgJiBBTURf
SVNfQVBVKSkNCj4gPj4gKyAgICAgICAgICAgICAgIGFtZGdwdV9yZXNpemVfYmFyMChhZGV2KTsN
Cj4gPj4gKyAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRl
di0+cGRldiwgMCk7DQo+ID4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9zaXplID0gcGNpX3Jlc291
cmNlX2xlbihhZGV2LT5wZGV2LCAwKTsNCj4gPj4gKw0KPiA+PiAgICNpZmRlZiBDT05GSUdfWDg2
XzY0DQo+ID4+ICAgICAgICAgIGlmIChhZGV2LT5mbGFncyAmIEFNRF9JU19BUFUpIHsNCj4gPj4g
ICAgICAgICAgICAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSAoKHU2NClSUkVHMzIobW1NQ19W
TV9GQl9PRkZTRVQpKQ0KPiA8PCAyMjsNCj4gPj4gLS0NCj4gPj4gMi43LjQNCj4gPj4NCj4gPj4g
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4gZHJp
LWRldmVsIG1haWxpbmcgbGlzdA0KPiA+PiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3Jn
DQo+ID4+IGh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJp
LWRldmVsDQo+IA0KPiANCj4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX18NCj4gYW1kLWdmeCBtYWlsaW5nIGxpc3QNCj4gYW1kLWdmeEBsaXN0cy5mcmVlZGVz
a3RvcC5vcmcNCj4gaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m
by9hbWQtZ2Z4DQo=
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:19 ` Zhang, Jerry
0 siblings, 0 replies; 71+ messages in thread
From: Zhang, Jerry @ 2017-03-16 2:19 UTC (permalink / raw)
To: Christian König, Zhou, David(ChunMing), Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
> -----Original Message-----
> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of
> Christian K?nig
> Sent: Wednesday, March 15, 2017 17:29
> To: Zhou, David(ChunMing); Ayyappa Ch
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Yes, exactly that.
(I'm not familiar with PCI too much.)
Is there any restrict for PCI device?
I'm concerning if any PCI couldn't support it on some motherboard.
>
> Christian.
>
> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> > Does that means we don't need invisible vram later?
> >
> > David
> >
> > -----Original Message-----
> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> > Behalf Of Christian K?nig
> > Sent: Wednesday, March 15, 2017 3:38 PM
> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >
> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
> > The CPU can access the full stolen VRAM directly on that hardware.
> >
> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
> >
> > Christian.
> >
> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> >> Is it possible on Carrizo asics? Or only supports on newer asics?
> >>
> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> >> <deathsimple@vodafone.de> wrote:
> >>> From: Christian König <christian.koenig@amd.com>
> >>>
> >>> Try to resize BAR0 to let CPU access all of VRAM.
> >>>
> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
> >>> ---
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> +++++++++++++++++++++++++++++
> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
> >>>
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> index 3b81ded..905ded9 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> amdgpu_device *adev, struct ttm_tt *ttm,
> >>> struct ttm_mem_reg *mem);
> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> amdgpu_mc *mc, u64 base);
> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> >>> amdgpu_mc *mc);
> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
> u64 size);
> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> index 118f4e6..92955fe 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
> *adev, struct amdgpu_mc *mc)
> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >>> }
> >>>
> >>> +/**
> >>> + * amdgpu_resize_bar0 - try to resize BAR0
> >>> + *
> >>> + * @adev: amdgpu_device pointer
> >>> + *
> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >>> + */
> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >>> + int r;
> >>> +
> >>> + r = pci_resize_resource(adev->pdev, 0, size);
> >>> +
> >>> + if (r == -ENOTSUPP) {
> >>> + /* The hardware don't support the extension. */
> >>> + return;
> >>> +
> >>> + } else if (r == -ENOSPC) {
> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >>> + } else if (r) {
> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >>> + }
> >>> +
> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >>> + amdgpu_doorbell_fini(adev);
> >>> + BUG_ON(amdgpu_doorbell_init(adev));
> >>> +}
> >>> +
> >>> /*
> >>> * GPU helpers function.
> >>> */
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> index dc9b6d6..36a7aa5 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
> *adev)
> >>> break;
> >>> }
> >>> adev->mc.vram_width = numchan * chansize;
> >>> - /* Could aper size report 0 ? */
> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> /* size in MB on si */
> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
> 1024ULL;
> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >>> 1024ULL
> >>> * 1024ULL;
> >>>
> >>> + if (!(adev->flags & AMD_IS_APU))
> >>> + amdgpu_resize_bar0(adev);
> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> +
> >>> #ifdef CONFIG_X86_64
> >>> if (adev->flags & AMD_IS_APU) {
> >>> adev->mc.aper_base =
> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> index c087b00..7761ad3 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
> *adev)
> >>> break;
> >>> }
> >>> adev->mc.vram_width = numchan * chansize;
> >>> - /* Could aper size report 0 ? */
> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> /* size in MB on si */
> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
> 1024ULL;
> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >>> 1024ULL
> >>> * 1024ULL;
> >>>
> >>> + if (!(adev->flags & AMD_IS_APU))
> >>> + amdgpu_resize_bar0(adev);
> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> +
> >>> #ifdef CONFIG_X86_64
> >>> if (adev->flags & AMD_IS_APU) {
> >>> adev->mc.aper_base =
> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> >>> --
> >>> 2.7.4
> >>>
> >>> _______________________________________________
> >>> dri-devel mailing list
> >>> dri-devel@lists.freedesktop.org
> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >
> > _______________________________________________
> > dri-devel mailing list
> > dri-devel@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:19 ` Zhang, Jerry
0 siblings, 0 replies; 71+ messages in thread
From: Zhang, Jerry @ 2017-03-16 2:19 UTC (permalink / raw)
To: Christian König, Zhou, David(ChunMing), Ayyappa Ch
Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA,
helgaas-DgEjT+Ai2ygdnm+yROfE0A,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
> -----Original Message-----
> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of
> Christian K?nig
> Sent: Wednesday, March 15, 2017 17:29
> To: Zhou, David(ChunMing); Ayyappa Ch
> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Yes, exactly that.
(I'm not familiar with PCI too much.)
Is there any restrict for PCI device?
I'm concerning if any PCI couldn't support it on some motherboard.
>
> Christian.
>
> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> > Does that means we don't need invisible vram later?
> >
> > David
> >
> > -----Original Message-----
> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> > Behalf Of Christian K?nig
> > Sent: Wednesday, March 15, 2017 3:38 PM
> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >
> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
> > The CPU can access the full stolen VRAM directly on that hardware.
> >
> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
> >
> > Christian.
> >
> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> >> Is it possible on Carrizo asics? Or only supports on newer asics?
> >>
> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> >> <deathsimple@vodafone.de> wrote:
> >>> From: Christian König <christian.koenig@amd.com>
> >>>
> >>> Try to resize BAR0 to let CPU access all of VRAM.
> >>>
> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
> >>> ---
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> +++++++++++++++++++++++++++++
> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
> >>>
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> index 3b81ded..905ded9 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> amdgpu_device *adev, struct ttm_tt *ttm,
> >>> struct ttm_mem_reg *mem);
> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> amdgpu_mc *mc, u64 base);
> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> >>> amdgpu_mc *mc);
> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
> u64 size);
> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> index 118f4e6..92955fe 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
> *adev, struct amdgpu_mc *mc)
> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >>> }
> >>>
> >>> +/**
> >>> + * amdgpu_resize_bar0 - try to resize BAR0
> >>> + *
> >>> + * @adev: amdgpu_device pointer
> >>> + *
> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >>> + */
> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >>> + int r;
> >>> +
> >>> + r = pci_resize_resource(adev->pdev, 0, size);
> >>> +
> >>> + if (r == -ENOTSUPP) {
> >>> + /* The hardware don't support the extension. */
> >>> + return;
> >>> +
> >>> + } else if (r == -ENOSPC) {
> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >>> + } else if (r) {
> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >>> + }
> >>> +
> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >>> + amdgpu_doorbell_fini(adev);
> >>> + BUG_ON(amdgpu_doorbell_init(adev));
> >>> +}
> >>> +
> >>> /*
> >>> * GPU helpers function.
> >>> */
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> index dc9b6d6..36a7aa5 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
> *adev)
> >>> break;
> >>> }
> >>> adev->mc.vram_width = numchan * chansize;
> >>> - /* Could aper size report 0 ? */
> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> /* size in MB on si */
> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
> 1024ULL;
> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >>> 1024ULL
> >>> * 1024ULL;
> >>>
> >>> + if (!(adev->flags & AMD_IS_APU))
> >>> + amdgpu_resize_bar0(adev);
> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> +
> >>> #ifdef CONFIG_X86_64
> >>> if (adev->flags & AMD_IS_APU) {
> >>> adev->mc.aper_base =
> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> index c087b00..7761ad3 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
> *adev)
> >>> break;
> >>> }
> >>> adev->mc.vram_width = numchan * chansize;
> >>> - /* Could aper size report 0 ? */
> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> /* size in MB on si */
> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
> 1024ULL;
> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >>> 1024ULL
> >>> * 1024ULL;
> >>>
> >>> + if (!(adev->flags & AMD_IS_APU))
> >>> + amdgpu_resize_bar0(adev);
> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >>> +
> >>> #ifdef CONFIG_X86_64
> >>> if (adev->flags & AMD_IS_APU) {
> >>> adev->mc.aper_base =
> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> >>> --
> >>> 2.7.4
> >>>
> >>> _______________________________________________
> >>> dri-devel mailing list
> >>> dri-devel@lists.freedesktop.org
> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >
> > _______________________________________________
> > dri-devel mailing list
> > dri-devel@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>
>
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:19 ` Zhang, Jerry
0 siblings, 0 replies; 71+ messages in thread
From: Zhang, Jerry @ 2017-03-16 2:19 UTC (permalink / raw)
To: Christian König, Zhou, David(ChunMing), Ayyappa Ch
Cc: linux-pci, linux-kernel, amd-gfx, platform-driver-x86, helgaas,
dri-devel
PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBkcmktZGV2ZWwgW21haWx0bzpk
cmktZGV2ZWwtYm91bmNlc0BsaXN0cy5mcmVlZGVza3RvcC5vcmddIE9uIEJlaGFsZiBPZg0KPiBD
aHJpc3RpYW4gSz9uaWcNCj4gU2VudDogV2VkbmVzZGF5LCBNYXJjaCAxNSwgMjAxNyAxNzoyOQ0K
PiBUbzogWmhvdSwgRGF2aWQoQ2h1bk1pbmcpOyBBeXlhcHBhIENoDQo+IENjOiBsaW51eC1wY2lA
dmdlci5rZXJuZWwub3JnOyBsaW51eC1rZXJuZWxAdmdlci5rZXJuZWwub3JnOyBhbWQtDQo+IGdm
eEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7IHBsYXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwu
b3JnOw0KPiBoZWxnYWFzQGtlcm5lbC5vcmc7IGRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5v
cmcNCj4gU3ViamVjdDogUmU6IFtQQVRDSCA0LzRdIGRybS9hbWRncHU6IHJlc2l6ZSBWUkFNIEJB
UiBmb3IgQ1BVIGFjY2Vzcw0KPiANCj4gWWVzLCBleGFjdGx5IHRoYXQuDQoNCihJJ20gbm90IGZh
bWlsaWFyIHdpdGggUENJIHRvbyBtdWNoLikNCklzIHRoZXJlIGFueSByZXN0cmljdCBmb3IgUENJ
IGRldmljZT8NCkknbSBjb25jZXJuaW5nIGlmIGFueSBQQ0kgY291bGRuJ3Qgc3VwcG9ydCBpdCBv
biBzb21lIG1vdGhlcmJvYXJkLg0KDQo+IA0KPiBDaHJpc3RpYW4uDQo+IA0KPiBBbSAxNS4wMy4y
MDE3IHVtIDA5OjI1IHNjaHJpZWIgWmhvdSwgRGF2aWQoQ2h1bk1pbmcpOg0KPiA+IERvZXMgdGhh
dCBtZWFucyB3ZSBkb24ndCBuZWVkIGludmlzaWJsZSB2cmFtIGxhdGVyPw0KPiA+DQo+ID4gRGF2
aWQNCj4gPg0KPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4gRnJvbTogZHJpLWRl
dmVsIFttYWlsdG86ZHJpLWRldmVsLWJvdW5jZXNAbGlzdHMuZnJlZWRlc2t0b3Aub3JnXSBPbg0K
PiA+IEJlaGFsZiBPZiBDaHJpc3RpYW4gSz9uaWcNCj4gPiBTZW50OiBXZWRuZXNkYXksIE1hcmNo
IDE1LCAyMDE3IDM6MzggUE0NCj4gPiBUbzogQXl5YXBwYSBDaCA8YXl5YXBwYS5jaC5saW51eEBn
bWFpbC5jb20+DQo+ID4gQ2M6IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5l
bEB2Z2VyLmtlcm5lbC5vcmc7DQo+ID4gYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7IHBs
YXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwub3JnOw0KPiA+IGhlbGdhYXNAa2VybmVsLm9y
ZzsgZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+IFN1YmplY3Q6IFJlOiBbUEFU
Q0ggNC80XSBkcm0vYW1kZ3B1OiByZXNpemUgVlJBTSBCQVIgZm9yIENQVSBhY2Nlc3MNCj4gPg0K
PiA+IENhcml6em8gaXMgYW4gQVBVIGFuZCByZXNpemluZyBCQVJzIGlzbid0IG5lZWRlZCBub3Ig
c3VwcG9ydGVkIHRoZXJlLg0KPiA+IFRoZSBDUFUgY2FuIGFjY2VzcyB0aGUgZnVsbCBzdG9sZW4g
VlJBTSBkaXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KPiA+DQo+ID4gQXMgZmFyIGFzIEkga25v
dyBBU0lDcyB3aXRoIHN1cHBvcnQgZm9yIHRoaXMgYXJlIFRvbmdhLCBGaWppIGFuZCBhbGwgUG9s
YXJpcyB2YXJpYW50cy4NCj4gPg0KPiA+IENocmlzdGlhbi4NCj4gPg0KPiA+IEFtIDE1LjAzLjIw
MTcgdW0gMDg6MjMgc2NocmllYiBBeXlhcHBhIENoOg0KPiA+PiBJcyBpdCBwb3NzaWJsZSBvbiBD
YXJyaXpvIGFzaWNzPyBPciBvbmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNzPw0KPiA+Pg0KPiA+
PiBPbiBNb24sIE1hciAxMywgMjAxNyBhdCA2OjExIFBNLCBDaHJpc3RpYW4gS8O2bmlnDQo+ID4+
IDxkZWF0aHNpbXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6DQo+ID4+PiBGcm9tOiBDaHJpc3RpYW4g
S8O2bmlnIDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+DQo+ID4+Pg0KPiA+Pj4gVHJ5IHRvIHJl
c2l6ZSBCQVIwIHRvIGxldCBDUFUgYWNjZXNzIGFsbCBvZiBWUkFNLg0KPiA+Pj4NCj4gPj4+IFNp
Z25lZC1vZmYtYnk6IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4N
Cj4gPj4+IC0tLQ0KPiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1Lmgg
ICAgICAgIHwgIDEgKw0KPiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1
X2RldmljZS5jIHwgMjkNCj4gKysrKysrKysrKysrKysrKysrKysrKysrKysrKysNCj4gPj4+ICAg
IGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMgICAgICB8ICA4ICsrKysrLS0t
DQo+ID4+PiAgICBkcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jICAgICAgfCAg
OCArKysrKy0tLQ0KPiA+Pj4gICAgNCBmaWxlcyBjaGFuZ2VkLCA0MCBpbnNlcnRpb25zKCspLCA2
IGRlbGV0aW9ucygtKQ0KPiA+Pj4NCj4gPj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0v
YW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9h
bWRncHUuaA0KPiA+Pj4gaW5kZXggM2I4MWRlZC4uOTA1ZGVkOSAxMDA2NDQNCj4gPj4+IC0tLSBh
L2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+PiArKysgYi9kcml2ZXJz
L2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+Pj4gQEAgLTE3MTksNiArMTcxOSw3IEBA
IHVpbnQ2NF90IGFtZGdwdV90dG1fdHRfcHRlX2ZsYWdzKHN0cnVjdA0KPiBhbWRncHVfZGV2aWNl
ICphZGV2LCBzdHJ1Y3QgdHRtX3R0ICp0dG0sDQo+ID4+PiAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgIHN0cnVjdCB0dG1fbWVtX3JlZyAqbWVtKTsNCj4gPj4+ICAgIHZvaWQgYW1k
Z3B1X3ZyYW1fbG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiBh
bWRncHVfbWMgKm1jLCB1NjQgYmFzZSk7DQo+ID4+PiAgICB2b2lkIGFtZGdwdV9ndHRfbG9jYXRp
b24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiA+Pj4gYW1kZ3B1X21jICpt
Yyk7DQo+ID4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFtZGdwdV9kZXZpY2Ug
KmFkZXYpOw0KPiA+Pj4gICAgdm9pZCBhbWRncHVfdHRtX3NldF9hY3RpdmVfdnJhbV9zaXplKHN0
cnVjdCBhbWRncHVfZGV2aWNlICphZGV2LA0KPiB1NjQgc2l6ZSk7DQo+ID4+PiAgICBpbnQgYW1k
Z3B1X3R0bV9pbml0KHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KTsNCj4gPj4+ICAgIHZvaWQg
YW1kZ3B1X3R0bV9maW5pKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KTsgZGlmZiAtLWdpdA0K
PiA+Pj4gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZGV2aWNlLmMNCj4gPj4+
IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+ID4+PiBpbmRl
eCAxMThmNGU2Li45Mjk1NWZlIDEwMDY0NA0KPiA+Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2Ft
ZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+ID4+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1k
L2FtZGdwdS9hbWRncHVfZGV2aWNlLmMNCj4gPj4+IEBAIC02OTIsNiArNjkyLDM1IEBAIHZvaWQg
YW1kZ3B1X2d0dF9sb2NhdGlvbihzdHJ1Y3QgYW1kZ3B1X2RldmljZQ0KPiAqYWRldiwgc3RydWN0
IGFtZGdwdV9tYyAqbWMpDQo+ID4+PiAgICAgICAgICAgICAgICAgICAgICAgICAgIG1jLT5ndHRf
c2l6ZSA+PiAyMCwgbWMtPmd0dF9zdGFydCwgbWMtPmd0dF9lbmQpOw0KPiA+Pj4gICAgfQ0KPiA+
Pj4NCj4gPj4+ICsvKioNCj4gPj4+ICsgKiBhbWRncHVfcmVzaXplX2JhcjAgLSB0cnkgdG8gcmVz
aXplIEJBUjANCj4gPj4+ICsgKg0KPiA+Pj4gKyAqIEBhZGV2OiBhbWRncHVfZGV2aWNlIHBvaW50
ZXINCj4gPj4+ICsgKg0KPiA+Pj4gKyAqIFRyeSB0byByZXNpemUgQkFSMCB0byBtYWtlIGFsbCBW
UkFNIENQVSBhY2Nlc3NpYmxlLg0KPiA+Pj4gKyAqLw0KPiA+Pj4gK3ZvaWQgYW1kZ3B1X3Jlc2l6
ZV9iYXIwKHN0cnVjdCBhbWRncHVfZGV2aWNlICphZGV2KSB7DQo+ID4+PiArICAgICAgIHUzMiBz
aXplID0gbWF4KGlsb2cyKGFkZXYtPm1jLnJlYWxfdnJhbV9zaXplIC0gMSkgKyAxLCAyMCkgLSAy
MDsNCj4gPj4+ICsgICAgICAgaW50IHI7DQo+ID4+PiArDQo+ID4+PiArICAgICAgIHIgPSBwY2lf
cmVzaXplX3Jlc291cmNlKGFkZXYtPnBkZXYsIDAsIHNpemUpOw0KPiA+Pj4gKw0KPiA+Pj4gKyAg
ICAgICBpZiAociA9PSAtRU5PVFNVUFApIHsNCj4gPj4+ICsgICAgICAgICAgICAgICAvKiBUaGUg
aGFyZHdhcmUgZG9uJ3Qgc3VwcG9ydCB0aGUgZXh0ZW5zaW9uLiAqLw0KPiA+Pj4gKyAgICAgICAg
ICAgICAgIHJldHVybjsNCj4gPj4+ICsNCj4gPj4+ICsgICAgICAgfSBlbHNlIGlmIChyID09IC1F
Tk9TUEMpIHsNCj4gPj4+ICsgICAgICAgICAgICAgICBEUk1fSU5GTygiTm90IGVub2lnaCBQQ0kg
YWRkcmVzcyBzcGFjZSBmb3IgYSBsYXJnZSBCQVIuIik7DQo+ID4+PiArICAgICAgIH0gZWxzZSBp
ZiAocikgew0KPiA+Pj4gKyAgICAgICAgICAgICAgIERSTV9FUlJPUigiUHJvYmxlbSByZXNpemlu
ZyBCQVIwICglZCkuIiwgcik7DQo+ID4+PiArICAgICAgIH0NCj4gPj4+ICsNCj4gPj4+ICsgICAg
ICAgLyogUmVpbml0IHRoZSBkb29yYmVsbCBtYXBwaW5nLCBpdCBpcyBtb3N0IGxpa2VseSBtb3Zl
ZCBhcyB3ZWxsICovDQo+ID4+PiArICAgICAgIGFtZGdwdV9kb29yYmVsbF9maW5pKGFkZXYpOw0K
PiA+Pj4gKyAgICAgICBCVUdfT04oYW1kZ3B1X2Rvb3JiZWxsX2luaXQoYWRldikpOw0KPiA+Pj4g
K30NCj4gPj4+ICsNCj4gPj4+ICAgIC8qDQo+ID4+PiAgICAgKiBHUFUgaGVscGVycyBmdW5jdGlv
bi4NCj4gPj4+ICAgICAqLw0KPiA+Pj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQv
YW1kZ3B1L2dtY192N18wLmMNCj4gPj4+IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21j
X3Y3XzAuYw0KPiA+Pj4gaW5kZXggZGM5YjZkNi4uMzZhN2FhNSAxMDA2NDQNCj4gPj4+IC0tLSBh
L2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMNCj4gPj4+ICsrKyBiL2RyaXZl
cnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18wLmMNCj4gPj4+IEBAIC0zNjcsMTMgKzM2Nywx
NSBAQCBzdGF0aWMgaW50IGdtY192N18wX21jX2luaXQoc3RydWN0IGFtZGdwdV9kZXZpY2UNCj4g
KmFkZXYpDQo+ID4+PiAgICAgICAgICAgICAgICAgICBicmVhazsNCj4gPj4+ICAgICAgICAgICB9
DQo+ID4+PiAgICAgICAgICAgYWRldi0+bWMudnJhbV93aWR0aCA9IG51bWNoYW4gKiBjaGFuc2l6
ZTsNCj4gPj4+IC0gICAgICAgLyogQ291bGQgYXBlciBzaXplIHJlcG9ydCAwID8gKi8NCj4gPj4+
IC0gICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jlc291cmNlX3N0YXJ0KGFkZXYtPnBk
ZXYsIDApOw0KPiA+Pj4gLSAgICAgICBhZGV2LT5tYy5hcGVyX3NpemUgPSBwY2lfcmVzb3VyY2Vf
bGVuKGFkZXYtPnBkZXYsIDApOw0KPiA+Pj4gICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24gc2kg
Ki8NCj4gPj4+ICAgICAgICAgICBhZGV2LT5tYy5tY192cmFtX3NpemUgPSBSUkVHMzIobW1DT05G
SUdfTUVNU0laRSkgKiAxMDI0VUxMICoNCj4gMTAyNFVMTDsNCj4gPj4+ICAgICAgICAgICBhZGV2
LT5tYy5yZWFsX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqDQo+ID4+PiAx
MDI0VUxMDQo+ID4+PiAqIDEwMjRVTEw7DQo+ID4+Pg0KPiA+Pj4gKyAgICAgICBpZiAoIShhZGV2
LT5mbGFncyAmIEFNRF9JU19BUFUpKQ0KPiA+Pj4gKyAgICAgICAgICAgICAgIGFtZGdwdV9yZXNp
emVfYmFyMChhZGV2KTsNCj4gPj4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0gcGNpX3Jl
c291cmNlX3N0YXJ0KGFkZXYtPnBkZXYsIDApOw0KPiA+Pj4gKyAgICAgICBhZGV2LT5tYy5hcGVy
X3NpemUgPSBwY2lfcmVzb3VyY2VfbGVuKGFkZXYtPnBkZXYsIDApOw0KPiA+Pj4gKw0KPiA+Pj4g
ICAgI2lmZGVmIENPTkZJR19YODZfNjQNCj4gPj4+ICAgICAgICAgICBpZiAoYWRldi0+ZmxhZ3Mg
JiBBTURfSVNfQVBVKSB7DQo+ID4+PiAgICAgICAgICAgICAgICAgICBhZGV2LT5tYy5hcGVyX2Jh
c2UgPQ0KPiA+Pj4gKCh1NjQpUlJFRzMyKG1tTUNfVk1fRkJfT0ZGU0VUKSkgPDwgMjI7IGRpZmYg
LS1naXQNCj4gPj4+IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYw0KPiA+
Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9nbWNfdjhfMC5jDQo+ID4+PiBpbmRleCBj
MDg3YjAwLi43NzYxYWQzIDEwMDY0NA0KPiA+Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9h
bWRncHUvZ21jX3Y4XzAuYw0KPiA+Pj4gKysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv
Z21jX3Y4XzAuYw0KPiA+Pj4gQEAgLTQ1OSwxMyArNDU5LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y4
XzBfbWNfaW5pdChzdHJ1Y3QgYW1kZ3B1X2RldmljZQ0KPiAqYWRldikNCj4gPj4+ICAgICAgICAg
ICAgICAgICAgIGJyZWFrOw0KPiA+Pj4gICAgICAgICAgIH0NCj4gPj4+ICAgICAgICAgICBhZGV2
LT5tYy52cmFtX3dpZHRoID0gbnVtY2hhbiAqIGNoYW5zaXplOw0KPiA+Pj4gLSAgICAgICAvKiBD
b3VsZCBhcGVyIHNpemUgcmVwb3J0IDAgPyAqLw0KPiA+Pj4gLSAgICAgICBhZGV2LT5tYy5hcGVy
X2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCk7DQo+ID4+PiAtICAgICAg
IGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+
ID4+PiAgICAgICAgICAgLyogc2l6ZSBpbiBNQiBvbiBzaSAqLw0KPiA+Pj4gICAgICAgICAgIGFk
ZXYtPm1jLm1jX3ZyYW1fc2l6ZSA9IFJSRUczMihtbUNPTkZJR19NRU1TSVpFKSAqIDEwMjRVTEwg
Kg0KPiAxMDI0VUxMOw0KPiA+Pj4gICAgICAgICAgIGFkZXYtPm1jLnJlYWxfdnJhbV9zaXplID0g
UlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gPj4+IDEwMjRVTEwNCj4gPj4+ICogMTAyNFVM
TDsNCj4gPj4+DQo+ID4+PiArICAgICAgIGlmICghKGFkZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkp
DQo+ID4+PiArICAgICAgICAgICAgICAgYW1kZ3B1X3Jlc2l6ZV9iYXIwKGFkZXYpOw0KPiA+Pj4g
KyAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVzb3VyY2Vfc3RhcnQoYWRldi0+cGRl
diwgMCk7DQo+ID4+PiArICAgICAgIGFkZXYtPm1jLmFwZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9s
ZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+PiArDQo+ID4+PiAgICAjaWZkZWYgQ09ORklHX1g4Nl82
NA0KPiA+Pj4gICAgICAgICAgIGlmIChhZGV2LT5mbGFncyAmIEFNRF9JU19BUFUpIHsNCj4gPj4+
ICAgICAgICAgICAgICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9DQo+ID4+PiAoKHU2NClSUkVH
MzIobW1NQ19WTV9GQl9PRkZTRVQpKSA8PCAyMjsNCj4gPj4+IC0tDQo+ID4+PiAyLjcuNA0KPiA+
Pj4NCj4gPj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
DQo+ID4+PiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+ID4+PiBkcmktZGV2ZWxAbGlzdHMuZnJl
ZWRlc2t0b3Aub3JnDQo+ID4+PiBodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFu
L2xpc3RpbmZvL2RyaS1kZXZlbA0KPiA+DQo+ID4gX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX18NCj4gPiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+ID4gZHJp
LWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+IGh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0
b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsDQo+ID4gX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPiBhbWQtZ2Z4IG1haWxpbmcgbGlzdA0K
PiA+IGFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQo+ID4gaHR0cHM6Ly9saXN0cy5mcmVl
ZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4DQo+IA0KPiANCj4gX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gZHJpLWRldmVsIG1haWxp
bmcgbGlzdA0KPiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQo+IGh0dHBzOi8vbGlz
dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsDQo=
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:25 ` Alex Deucher
0 siblings, 0 replies; 71+ messages in thread
From: Alex Deucher @ 2017-03-16 2:25 UTC (permalink / raw)
To: Zhang, Jerry
Cc: Christian König, Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
>> -----Original Message-----
>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of
>> Christian K?nig
>> Sent: Wednesday, March 15, 2017 17:29
>> To: Zhou, David(ChunMing); Ayyappa Ch
>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>
>> Yes, exactly that.
>
> (I'm not familiar with PCI too much.)
> Is there any restrict for PCI device?
> I'm concerning if any PCI couldn't support it on some motherboard.
It depends on the PCI root bridge. This patch set only implements
support for AMD root bridges. Intel and other vendors would need
similar code.
Alex
>
>>
>> Christian.
>>
>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
>> > Does that means we don't need invisible vram later?
>> >
>> > David
>> >
>> > -----Original Message-----
>> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>> > Behalf Of Christian K?nig
>> > Sent: Wednesday, March 15, 2017 3:38 PM
>> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
>> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
>> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
>> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>> >
>> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
>> > The CPU can access the full stolen VRAM directly on that hardware.
>> >
>> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
>> >
>> > Christian.
>> >
>> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>> >> Is it possible on Carrizo asics? Or only supports on newer asics?
>> >>
>> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> >> <deathsimple@vodafone.de> wrote:
>> >>> From: Christian König <christian.koenig@amd.com>
>> >>>
>> >>> Try to resize BAR0 to let CPU access all of VRAM.
>> >>>
>> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> >>> ---
>> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>> +++++++++++++++++++++++++++++
>> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
>> >>>
>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> index 3b81ded..905ded9 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>> amdgpu_device *adev, struct ttm_tt *ttm,
>> >>> struct ttm_mem_reg *mem);
>> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
>> amdgpu_mc *mc, u64 base);
>> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>> >>> amdgpu_mc *mc);
>> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
>> u64 size);
>> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> index 118f4e6..92955fe 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
>> *adev, struct amdgpu_mc *mc)
>> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>> >>> }
>> >>>
>> >>> +/**
>> >>> + * amdgpu_resize_bar0 - try to resize BAR0
>> >>> + *
>> >>> + * @adev: amdgpu_device pointer
>> >>> + *
>> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> >>> + */
>> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>> >>> + int r;
>> >>> +
>> >>> + r = pci_resize_resource(adev->pdev, 0, size);
>> >>> +
>> >>> + if (r == -ENOTSUPP) {
>> >>> + /* The hardware don't support the extension. */
>> >>> + return;
>> >>> +
>> >>> + } else if (r == -ENOSPC) {
>> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>> >>> + } else if (r) {
>> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> >>> + }
>> >>> +
>> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>> >>> + amdgpu_doorbell_fini(adev);
>> >>> + BUG_ON(amdgpu_doorbell_init(adev));
>> >>> +}
>> >>> +
>> >>> /*
>> >>> * GPU helpers function.
>> >>> */
>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> index dc9b6d6..36a7aa5 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>> *adev)
>> >>> break;
>> >>> }
>> >>> adev->mc.vram_width = numchan * chansize;
>> >>> - /* Could aper size report 0 ? */
>> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> /* size in MB on si */
>> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>> 1024ULL;
>> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>> >>> 1024ULL
>> >>> * 1024ULL;
>> >>>
>> >>> + if (!(adev->flags & AMD_IS_APU))
>> >>> + amdgpu_resize_bar0(adev);
>> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> +
>> >>> #ifdef CONFIG_X86_64
>> >>> if (adev->flags & AMD_IS_APU) {
>> >>> adev->mc.aper_base =
>> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> index c087b00..7761ad3 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>> *adev)
>> >>> break;
>> >>> }
>> >>> adev->mc.vram_width = numchan * chansize;
>> >>> - /* Could aper size report 0 ? */
>> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> /* size in MB on si */
>> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>> 1024ULL;
>> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>> >>> 1024ULL
>> >>> * 1024ULL;
>> >>>
>> >>> + if (!(adev->flags & AMD_IS_APU))
>> >>> + amdgpu_resize_bar0(adev);
>> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> +
>> >>> #ifdef CONFIG_X86_64
>> >>> if (adev->flags & AMD_IS_APU) {
>> >>> adev->mc.aper_base =
>> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> >>> --
>> >>> 2.7.4
>> >>>
>> >>> _______________________________________________
>> >>> dri-devel mailing list
>> >>> dri-devel@lists.freedesktop.org
>> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>> >
>> > _______________________________________________
>> > dri-devel mailing list
>> > dri-devel@lists.freedesktop.org
>> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
>> > _______________________________________________
>> > amd-gfx mailing list
>> > amd-gfx@lists.freedesktop.org
>> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:25 ` Alex Deucher
0 siblings, 0 replies; 71+ messages in thread
From: Alex Deucher @ 2017-03-16 2:25 UTC (permalink / raw)
To: Zhang, Jerry
Cc: Zhou, David(ChunMing),
Ayyappa Ch, linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA, Christian König,
helgaas-DgEjT+Ai2ygdnm+yROfE0A,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
>> -----Original Message-----
>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Behalf Of
>> Christian K?nig
>> Sent: Wednesday, March 15, 2017 17:29
>> To: Zhou, David(ChunMing); Ayyappa Ch
>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>
>> Yes, exactly that.
>
> (I'm not familiar with PCI too much.)
> Is there any restrict for PCI device?
> I'm concerning if any PCI couldn't support it on some motherboard.
It depends on the PCI root bridge. This patch set only implements
support for AMD root bridges. Intel and other vendors would need
similar code.
Alex
>
>>
>> Christian.
>>
>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
>> > Does that means we don't need invisible vram later?
>> >
>> > David
>> >
>> > -----Original Message-----
>> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>> > Behalf Of Christian K?nig
>> > Sent: Wednesday, March 15, 2017 3:38 PM
>> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
>> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
>> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
>> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>> >
>> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
>> > The CPU can access the full stolen VRAM directly on that hardware.
>> >
>> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris variants.
>> >
>> > Christian.
>> >
>> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>> >> Is it possible on Carrizo asics? Or only supports on newer asics?
>> >>
>> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>> >> <deathsimple@vodafone.de> wrote:
>> >>> From: Christian König <christian.koenig@amd.com>
>> >>>
>> >>> Try to resize BAR0 to let CPU access all of VRAM.
>> >>>
>> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> >>> ---
>> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>> +++++++++++++++++++++++++++++
>> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
>> >>>
>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> index 3b81ded..905ded9 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>> amdgpu_device *adev, struct ttm_tt *ttm,
>> >>> struct ttm_mem_reg *mem);
>> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
>> amdgpu_mc *mc, u64 base);
>> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>> >>> amdgpu_mc *mc);
>> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
>> u64 size);
>> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> index 118f4e6..92955fe 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
>> *adev, struct amdgpu_mc *mc)
>> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>> >>> }
>> >>>
>> >>> +/**
>> >>> + * amdgpu_resize_bar0 - try to resize BAR0
>> >>> + *
>> >>> + * @adev: amdgpu_device pointer
>> >>> + *
>> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> >>> + */
>> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>> >>> + int r;
>> >>> +
>> >>> + r = pci_resize_resource(adev->pdev, 0, size);
>> >>> +
>> >>> + if (r == -ENOTSUPP) {
>> >>> + /* The hardware don't support the extension. */
>> >>> + return;
>> >>> +
>> >>> + } else if (r == -ENOSPC) {
>> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>> >>> + } else if (r) {
>> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> >>> + }
>> >>> +
>> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>> >>> + amdgpu_doorbell_fini(adev);
>> >>> + BUG_ON(amdgpu_doorbell_init(adev));
>> >>> +}
>> >>> +
>> >>> /*
>> >>> * GPU helpers function.
>> >>> */
>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> index dc9b6d6..36a7aa5 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_device
>> *adev)
>> >>> break;
>> >>> }
>> >>> adev->mc.vram_width = numchan * chansize;
>> >>> - /* Could aper size report 0 ? */
>> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> /* size in MB on si */
>> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>> 1024ULL;
>> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>> >>> 1024ULL
>> >>> * 1024ULL;
>> >>>
>> >>> + if (!(adev->flags & AMD_IS_APU))
>> >>> + amdgpu_resize_bar0(adev);
>> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> +
>> >>> #ifdef CONFIG_X86_64
>> >>> if (adev->flags & AMD_IS_APU) {
>> >>> adev->mc.aper_base =
>> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> index c087b00..7761ad3 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_device
>> *adev)
>> >>> break;
>> >>> }
>> >>> adev->mc.vram_width = numchan * chansize;
>> >>> - /* Could aper size report 0 ? */
>> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> /* size in MB on si */
>> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) * 1024ULL *
>> 1024ULL;
>> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>> >>> 1024ULL
>> >>> * 1024ULL;
>> >>>
>> >>> + if (!(adev->flags & AMD_IS_APU))
>> >>> + amdgpu_resize_bar0(adev);
>> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>> >>> +
>> >>> #ifdef CONFIG_X86_64
>> >>> if (adev->flags & AMD_IS_APU) {
>> >>> adev->mc.aper_base =
>> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> >>> --
>> >>> 2.7.4
>> >>>
>> >>> _______________________________________________
>> >>> dri-devel mailing list
>> >>> dri-devel@lists.freedesktop.org
>> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>> >
>> > _______________________________________________
>> > dri-devel mailing list
>> > dri-devel@lists.freedesktop.org
>> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
>> > _______________________________________________
>> > amd-gfx mailing list
>> > amd-gfx@lists.freedesktop.org
>> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:25 ` Alex Deucher
0 siblings, 0 replies; 71+ messages in thread
From: Alex Deucher @ 2017-03-16 2:25 UTC (permalink / raw)
To: Zhang, Jerry
Cc: Christian König, Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
>> -----Original Message-----
>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On Beha=
lf Of
>> Christian K?nig
>> Sent: Wednesday, March 15, 2017 17:29
>> To: Zhou, David(ChunMing); Ayyappa Ch
>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>
>> Yes, exactly that.
>
> (I'm not familiar with PCI too much.)
> Is there any restrict for PCI device?
> I'm concerning if any PCI couldn't support it on some motherboard.
It depends on the PCI root bridge. This patch set only implements
support for AMD root bridges. Intel and other vendors would need
similar code.
Alex
>
>>
>> Christian.
>>
>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
>> > Does that means we don't need invisible vram later?
>> >
>> > David
>> >
>> > -----Original Message-----
>> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>> > Behalf Of Christian K?nig
>> > Sent: Wednesday, March 15, 2017 3:38 PM
>> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
>> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
>> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
>> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>> >
>> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
>> > The CPU can access the full stolen VRAM directly on that hardware.
>> >
>> > As far as I know ASICs with support for this are Tonga, Fiji and all P=
olaris variants.
>> >
>> > Christian.
>> >
>> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>> >> Is it possible on Carrizo asics? Or only supports on newer asics?
>> >>
>> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian K=C3=B6nig
>> >> <deathsimple@vodafone.de> wrote:
>> >>> From: Christian K=C3=B6nig <christian.koenig@amd.com>
>> >>>
>> >>> Try to resize BAR0 to let CPU access all of VRAM.
>> >>>
>> >>> Signed-off-by: Christian K=C3=B6nig <christian.koenig@amd.com>
>> >>> ---
>> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>> +++++++++++++++++++++++++++++
>> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
>> >>>
>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> index 3b81ded..905ded9 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>> amdgpu_device *adev, struct ttm_tt *ttm,
>> >>> struct ttm_mem_reg *mem);
>> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
>> amdgpu_mc *mc, u64 base);
>> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>> >>> amdgpu_mc *mc);
>> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev,
>> u64 size);
>> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> index 118f4e6..92955fe 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device
>> *adev, struct amdgpu_mc *mc)
>> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt=
_end);
>> >>> }
>> >>>
>> >>> +/**
>> >>> + * amdgpu_resize_bar0 - try to resize BAR0
>> >>> + *
>> >>> + * @adev: amdgpu_device pointer
>> >>> + *
>> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>> >>> + */
>> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>> >>> + u32 size =3D max(ilog2(adev->mc.real_vram_size - 1) + 1, 20)=
- 20;
>> >>> + int r;
>> >>> +
>> >>> + r =3D pci_resize_resource(adev->pdev, 0, size);
>> >>> +
>> >>> + if (r =3D=3D -ENOTSUPP) {
>> >>> + /* The hardware don't support the extension. */
>> >>> + return;
>> >>> +
>> >>> + } else if (r =3D=3D -ENOSPC) {
>> >>> + DRM_INFO("Not enoigh PCI address space for a large B=
AR.");
>> >>> + } else if (r) {
>> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>> >>> + }
>> >>> +
>> >>> + /* Reinit the doorbell mapping, it is most likely moved as w=
ell */
>> >>> + amdgpu_doorbell_fini(adev);
>> >>> + BUG_ON(amdgpu_doorbell_init(adev));
>> >>> +}
>> >>> +
>> >>> /*
>> >>> * GPU helpers function.
>> >>> */
>> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> index dc9b6d6..36a7aa5 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct amdgpu_devi=
ce
>> *adev)
>> >>> break;
>> >>> }
>> >>> adev->mc.vram_width =3D numchan * chansize;
>> >>> - /* Could aper size report 0 ? */
>> >>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>> >>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>> >>> /* size in MB on si */
>> >>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024U=
LL *
>> 1024ULL;
>> >>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) *
>> >>> 1024ULL
>> >>> * 1024ULL;
>> >>>
>> >>> + if (!(adev->flags & AMD_IS_APU))
>> >>> + amdgpu_resize_bar0(adev);
>> >>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>> >>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>> >>> +
>> >>> #ifdef CONFIG_X86_64
>> >>> if (adev->flags & AMD_IS_APU) {
>> >>> adev->mc.aper_base =3D
>> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> index c087b00..7761ad3 100644
>> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct amdgpu_devi=
ce
>> *adev)
>> >>> break;
>> >>> }
>> >>> adev->mc.vram_width =3D numchan * chansize;
>> >>> - /* Could aper size report 0 ? */
>> >>> - adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>> >>> - adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>> >>> /* size in MB on si */
>> >>> adev->mc.mc_vram_size =3D RREG32(mmCONFIG_MEMSIZE) * 1024U=
LL *
>> 1024ULL;
>> >>> adev->mc.real_vram_size =3D RREG32(mmCONFIG_MEMSIZE) *
>> >>> 1024ULL
>> >>> * 1024ULL;
>> >>>
>> >>> + if (!(adev->flags & AMD_IS_APU))
>> >>> + amdgpu_resize_bar0(adev);
>> >>> + adev->mc.aper_base =3D pci_resource_start(adev->pdev, 0);
>> >>> + adev->mc.aper_size =3D pci_resource_len(adev->pdev, 0);
>> >>> +
>> >>> #ifdef CONFIG_X86_64
>> >>> if (adev->flags & AMD_IS_APU) {
>> >>> adev->mc.aper_base =3D
>> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>> >>> --
>> >>> 2.7.4
>> >>>
>> >>> _______________________________________________
>> >>> dri-devel mailing list
>> >>> dri-devel@lists.freedesktop.org
>> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>> >
>> > _______________________________________________
>> > dri-devel mailing list
>> > dri-devel@lists.freedesktop.org
>> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
>> > _______________________________________________
>> > amd-gfx mailing list
>> > amd-gfx@lists.freedesktop.org
>> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-16 2:25 ` Alex Deucher
@ 2017-03-16 2:41 ` Zhang, Jerry
-1 siblings, 0 replies; 71+ messages in thread
From: Zhang, Jerry @ 2017-03-16 2:41 UTC (permalink / raw)
To: Alex Deucher
Cc: Christian König, Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
Thanks for your info.
I see.
Regards,
Jerry (Junwei Zhang)
Linux Base Graphics
SRDC Software Development
_____________________________________
> -----Original Message-----
> From: Alex Deucher [mailto:alexdeucher@gmail.com]
> Sent: Thursday, March 16, 2017 10:25
> To: Zhang, Jerry
> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux-
> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-
> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
> >> -----Original Message-----
> >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> Behalf Of Christian K?nig
> >> Sent: Wednesday, March 15, 2017 17:29
> >> To: Zhou, David(ChunMing); Ayyappa Ch
> >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >>
> >> Yes, exactly that.
> >
> > (I'm not familiar with PCI too much.)
> > Is there any restrict for PCI device?
> > I'm concerning if any PCI couldn't support it on some motherboard.
>
> It depends on the PCI root bridge. This patch set only implements support for
> AMD root bridges. Intel and other vendors would need similar code.
>
> Alex
>
> >
> >>
> >> Christian.
> >>
> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> >> > Does that means we don't need invisible vram later?
> >> >
> >> > David
> >> >
> >> > -----Original Message-----
> >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> > Behalf Of Christian K?nig
> >> > Sent: Wednesday, March 15, 2017 3:38 PM
> >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >> >
> >> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
> >> > The CPU can access the full stolen VRAM directly on that hardware.
> >> >
> >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
> variants.
> >> >
> >> > Christian.
> >> >
> >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> >> >> Is it possible on Carrizo asics? Or only supports on newer asics?
> >> >>
> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> >> >> <deathsimple@vodafone.de> wrote:
> >> >>> From: Christian König <christian.koenig@amd.com>
> >> >>>
> >> >>> Try to resize BAR0 to let CPU access all of VRAM.
> >> >>>
> >> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
> >> >>> ---
> >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> >> +++++++++++++++++++++++++++++
> >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
> >> >>>
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> index 3b81ded..905ded9 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> >> amdgpu_device *adev, struct ttm_tt *ttm,
> >> >>> struct ttm_mem_reg *mem);
> >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> >> amdgpu_mc *mc, u64 base);
> >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> >> >>> amdgpu_mc *mc);
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device
> >> >>> *adev,
> >> u64 size);
> >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> index 118f4e6..92955fe 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct
> >> >>> amdgpu_device
> >> *adev, struct amdgpu_mc *mc)
> >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >> >>> }
> >> >>>
> >> >>> +/**
> >> >>> + * amdgpu_resize_bar0 - try to resize BAR0
> >> >>> + *
> >> >>> + * @adev: amdgpu_device pointer
> >> >>> + *
> >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >> >>> + */
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
> >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >> >>> + int r;
> >> >>> +
> >> >>> + r = pci_resize_resource(adev->pdev, 0, size);
> >> >>> +
> >> >>> + if (r == -ENOTSUPP) {
> >> >>> + /* The hardware don't support the extension. */
> >> >>> + return;
> >> >>> +
> >> >>> + } else if (r == -ENOSPC) {
> >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >> >>> + } else if (r) {
> >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >> >>> + }
> >> >>> +
> >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >> >>> + amdgpu_doorbell_fini(adev);
> >> >>> + BUG_ON(amdgpu_doorbell_init(adev));
> >> >>> +}
> >> >>> +
> >> >>> /*
> >> >>> * GPU helpers function.
> >> >>> */
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> index dc9b6d6..36a7aa5 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>> break;
> >> >>> }
> >> >>> adev->mc.vram_width = numchan * chansize;
> >> >>> - /* Could aper size report 0 ? */
> >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> /* size in MB on si */
> >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> + if (!(adev->flags & AMD_IS_APU))
> >> >>> + amdgpu_resize_bar0(adev);
> >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> +
> >> >>> #ifdef CONFIG_X86_64
> >> >>> if (adev->flags & AMD_IS_APU) {
> >> >>> adev->mc.aper_base =
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> index c087b00..7761ad3 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>> break;
> >> >>> }
> >> >>> adev->mc.vram_width = numchan * chansize;
> >> >>> - /* Could aper size report 0 ? */
> >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> /* size in MB on si */
> >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> + if (!(adev->flags & AMD_IS_APU))
> >> >>> + amdgpu_resize_bar0(adev);
> >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> +
> >> >>> #ifdef CONFIG_X86_64
> >> >>> if (adev->flags & AMD_IS_APU) {
> >> >>> adev->mc.aper_base =
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> >> >>> --
> >> >>> 2.7.4
> >> >>>
> >> >>> _______________________________________________
> >> >>> dri-devel mailing list
> >> >>> dri-devel@lists.freedesktop.org
> >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> >
> >> > _______________________________________________
> >> > dri-devel mailing list
> >> > dri-devel@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> > _______________________________________________
> >> > amd-gfx mailing list
> >> > amd-gfx@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> >>
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-16 2:41 ` Zhang, Jerry
0 siblings, 0 replies; 71+ messages in thread
From: Zhang, Jerry @ 2017-03-16 2:41 UTC (permalink / raw)
To: Alex Deucher
Cc: Christian König, Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
VGhhbmtzIGZvciB5b3VyIGluZm8uDQpJIHNlZS4NCg0KUmVnYXJkcywNCkplcnJ5IChKdW53ZWkg
WmhhbmcpDQoNCkxpbnV4IEJhc2UgR3JhcGhpY3MNClNSREMgU29mdHdhcmUgRGV2ZWxvcG1lbnQN
Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCg0KDQo+IC0tLS0tT3JpZ2lu
YWwgTWVzc2FnZS0tLS0tDQo+IEZyb206IEFsZXggRGV1Y2hlciBbbWFpbHRvOmFsZXhkZXVjaGVy
QGdtYWlsLmNvbV0NCj4gU2VudDogVGh1cnNkYXksIE1hcmNoIDE2LCAyMDE3IDEwOjI1DQo+IFRv
OiBaaGFuZywgSmVycnkNCj4gQ2M6IENocmlzdGlhbiBLw7ZuaWc7IFpob3UsIERhdmlkKENodW5N
aW5nKTsgQXl5YXBwYSBDaDsgbGludXgtDQo+IHBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtl
cm5lbEB2Z2VyLmtlcm5lbC5vcmc7IGRyaS0NCj4gZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3Jn
OyBwbGF0Zm9ybS1kcml2ZXIteDg2QHZnZXIua2VybmVsLm9yZzsNCj4gaGVsZ2Fhc0BrZXJuZWwu
b3JnOyBhbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiBTdWJqZWN0OiBSZTogW1BBVENI
IDQvNF0gZHJtL2FtZGdwdTogcmVzaXplIFZSQU0gQkFSIGZvciBDUFUgYWNjZXNzDQo+IA0KPiBP
biBXZWQsIE1hciAxNSwgMjAxNyBhdCAxMDoxOSBQTSwgWmhhbmcsIEplcnJ5IDxKZXJyeS5aaGFu
Z0BhbWQuY29tPiB3cm90ZToNCj4gPj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gPj4g
RnJvbTogZHJpLWRldmVsIFttYWlsdG86ZHJpLWRldmVsLWJvdW5jZXNAbGlzdHMuZnJlZWRlc2t0
b3Aub3JnXSBPbg0KPiA+PiBCZWhhbGYgT2YgQ2hyaXN0aWFuIEs/bmlnDQo+ID4+IFNlbnQ6IFdl
ZG5lc2RheSwgTWFyY2ggMTUsIDIwMTcgMTc6MjkNCj4gPj4gVG86IFpob3UsIERhdmlkKENodW5N
aW5nKTsgQXl5YXBwYSBDaA0KPiA+PiBDYzogbGludXgtcGNpQHZnZXIua2VybmVsLm9yZzsgbGlu
dXgta2VybmVsQHZnZXIua2VybmVsLm9yZzsgYW1kLQ0KPiA+PiBnZnhAbGlzdHMuZnJlZWRlc2t0
b3Aub3JnOyBwbGF0Zm9ybS1kcml2ZXIteDg2QHZnZXIua2VybmVsLm9yZzsNCj4gPj4gaGVsZ2Fh
c0BrZXJuZWwub3JnOyBkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnDQo+ID4+IFN1Ympl
Y3Q6IFJlOiBbUEFUQ0ggNC80XSBkcm0vYW1kZ3B1OiByZXNpemUgVlJBTSBCQVIgZm9yIENQVSBh
Y2Nlc3MNCj4gPj4NCj4gPj4gWWVzLCBleGFjdGx5IHRoYXQuDQo+ID4NCj4gPiAoSSdtIG5vdCBm
YW1pbGlhciB3aXRoIFBDSSB0b28gbXVjaC4pDQo+ID4gSXMgdGhlcmUgYW55IHJlc3RyaWN0IGZv
ciBQQ0kgZGV2aWNlPw0KPiA+IEknbSBjb25jZXJuaW5nIGlmIGFueSBQQ0kgY291bGRuJ3Qgc3Vw
cG9ydCBpdCBvbiBzb21lIG1vdGhlcmJvYXJkLg0KPiANCj4gSXQgZGVwZW5kcyBvbiB0aGUgUENJ
IHJvb3QgYnJpZGdlLiAgVGhpcyBwYXRjaCBzZXQgb25seSBpbXBsZW1lbnRzIHN1cHBvcnQgZm9y
DQo+IEFNRCByb290IGJyaWRnZXMuICBJbnRlbCBhbmQgb3RoZXIgdmVuZG9ycyB3b3VsZCBuZWVk
IHNpbWlsYXIgY29kZS4NCj4gDQo+IEFsZXgNCj4gDQo+ID4NCj4gPj4NCj4gPj4gQ2hyaXN0aWFu
Lg0KPiA+Pg0KPiA+PiBBbSAxNS4wMy4yMDE3IHVtIDA5OjI1IHNjaHJpZWIgWmhvdSwgRGF2aWQo
Q2h1bk1pbmcpOg0KPiA+PiA+IERvZXMgdGhhdCBtZWFucyB3ZSBkb24ndCBuZWVkIGludmlzaWJs
ZSB2cmFtIGxhdGVyPw0KPiA+PiA+DQo+ID4+ID4gRGF2aWQNCj4gPj4gPg0KPiA+PiA+IC0tLS0t
T3JpZ2luYWwgTWVzc2FnZS0tLS0tDQo+ID4+ID4gRnJvbTogZHJpLWRldmVsIFttYWlsdG86ZHJp
LWRldmVsLWJvdW5jZXNAbGlzdHMuZnJlZWRlc2t0b3Aub3JnXSBPbg0KPiA+PiA+IEJlaGFsZiBP
ZiBDaHJpc3RpYW4gSz9uaWcNCj4gPj4gPiBTZW50OiBXZWRuZXNkYXksIE1hcmNoIDE1LCAyMDE3
IDM6MzggUE0NCj4gPj4gPiBUbzogQXl5YXBwYSBDaCA8YXl5YXBwYS5jaC5saW51eEBnbWFpbC5j
b20+DQo+ID4+ID4gQ2M6IGxpbnV4LXBjaUB2Z2VyLmtlcm5lbC5vcmc7IGxpbnV4LWtlcm5lbEB2
Z2VyLmtlcm5lbC5vcmc7DQo+ID4+ID4gYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmc7IHBs
YXRmb3JtLWRyaXZlci14ODZAdmdlci5rZXJuZWwub3JnOw0KPiA+PiA+IGhlbGdhYXNAa2VybmVs
Lm9yZzsgZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+PiA+IFN1YmplY3Q6IFJl
OiBbUEFUQ0ggNC80XSBkcm0vYW1kZ3B1OiByZXNpemUgVlJBTSBCQVIgZm9yIENQVSBhY2Nlc3MN
Cj4gPj4gPg0KPiA+PiA+IENhcml6em8gaXMgYW4gQVBVIGFuZCByZXNpemluZyBCQVJzIGlzbid0
IG5lZWRlZCBub3Igc3VwcG9ydGVkIHRoZXJlLg0KPiA+PiA+IFRoZSBDUFUgY2FuIGFjY2VzcyB0
aGUgZnVsbCBzdG9sZW4gVlJBTSBkaXJlY3RseSBvbiB0aGF0IGhhcmR3YXJlLg0KPiA+PiA+DQo+
ID4+ID4gQXMgZmFyIGFzIEkga25vdyBBU0lDcyB3aXRoIHN1cHBvcnQgZm9yIHRoaXMgYXJlIFRv
bmdhLCBGaWppIGFuZCBhbGwgUG9sYXJpcw0KPiB2YXJpYW50cy4NCj4gPj4gPg0KPiA+PiA+IENo
cmlzdGlhbi4NCj4gPj4gPg0KPiA+PiA+IEFtIDE1LjAzLjIwMTcgdW0gMDg6MjMgc2NocmllYiBB
eXlhcHBhIENoOg0KPiA+PiA+PiBJcyBpdCBwb3NzaWJsZSBvbiBDYXJyaXpvIGFzaWNzPyBPciBv
bmx5IHN1cHBvcnRzIG9uIG5ld2VyIGFzaWNzPw0KPiA+PiA+Pg0KPiA+PiA+PiBPbiBNb24sIE1h
ciAxMywgMjAxNyBhdCA2OjExIFBNLCBDaHJpc3RpYW4gS8O2bmlnDQo+ID4+ID4+IDxkZWF0aHNp
bXBsZUB2b2RhZm9uZS5kZT4gd3JvdGU6DQo+ID4+ID4+PiBGcm9tOiBDaHJpc3RpYW4gS8O2bmln
IDxjaHJpc3RpYW4ua29lbmlnQGFtZC5jb20+DQo+ID4+ID4+Pg0KPiA+PiA+Pj4gVHJ5IHRvIHJl
c2l6ZSBCQVIwIHRvIGxldCBDUFUgYWNjZXNzIGFsbCBvZiBWUkFNLg0KPiA+PiA+Pj4NCj4gPj4g
Pj4+IFNpZ25lZC1vZmYtYnk6IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1k
LmNvbT4NCj4gPj4gPj4+IC0tLQ0KPiA+PiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRn
cHUvYW1kZ3B1LmggICAgICAgIHwgIDEgKw0KPiA+PiA+Pj4gICAgZHJpdmVycy9ncHUvZHJtL2Ft
ZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jIHwgMjkNCj4gPj4gKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysNCj4gPj4gPj4+ICAgIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192N18w
LmMgICAgICB8ICA4ICsrKysrLS0tDQo+ID4+ID4+PiAgICBkcml2ZXJzL2dwdS9kcm0vYW1kL2Ft
ZGdwdS9nbWNfdjhfMC5jICAgICAgfCAgOCArKysrKy0tLQ0KPiA+PiA+Pj4gICAgNCBmaWxlcyBj
aGFuZ2VkLCA0MCBpbnNlcnRpb25zKCspLCA2IGRlbGV0aW9ucygtKQ0KPiA+PiA+Pj4NCj4gPj4g
Pj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+
PiA+Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHUuaA0KPiA+PiA+Pj4gaW5k
ZXggM2I4MWRlZC4uOTA1ZGVkOSAxMDA2NDQNCj4gPj4gPj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2Ry
bS9hbWQvYW1kZ3B1L2FtZGdwdS5oDQo+ID4+ID4+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1k
L2FtZGdwdS9hbWRncHUuaA0KPiA+PiA+Pj4gQEAgLTE3MTksNiArMTcxOSw3IEBAIHVpbnQ2NF90
IGFtZGdwdV90dG1fdHRfcHRlX2ZsYWdzKHN0cnVjdA0KPiA+PiBhbWRncHVfZGV2aWNlICphZGV2
LCBzdHJ1Y3QgdHRtX3R0ICp0dG0sDQo+ID4+ID4+PiAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgIHN0cnVjdCB0dG1fbWVtX3JlZyAqbWVtKTsNCj4gPj4gPj4+ICAgIHZvaWQgYW1k
Z3B1X3ZyYW1fbG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiA+
PiBhbWRncHVfbWMgKm1jLCB1NjQgYmFzZSk7DQo+ID4+ID4+PiAgICB2b2lkIGFtZGdwdV9ndHRf
bG9jYXRpb24oc3RydWN0IGFtZGdwdV9kZXZpY2UgKmFkZXYsIHN0cnVjdA0KPiA+PiA+Pj4gYW1k
Z3B1X21jICptYyk7DQo+ID4+ID4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFt
ZGdwdV9kZXZpY2UgKmFkZXYpOw0KPiA+PiA+Pj4gICAgdm9pZCBhbWRncHVfdHRtX3NldF9hY3Rp
dmVfdnJhbV9zaXplKHN0cnVjdCBhbWRncHVfZGV2aWNlDQo+ID4+ID4+PiAqYWRldiwNCj4gPj4g
dTY0IHNpemUpOw0KPiA+PiA+Pj4gICAgaW50IGFtZGdwdV90dG1faW5pdChzdHJ1Y3QgYW1kZ3B1
X2RldmljZSAqYWRldik7DQo+ID4+ID4+PiAgICB2b2lkIGFtZGdwdV90dG1fZmluaShzdHJ1Y3Qg
YW1kZ3B1X2RldmljZSAqYWRldik7IGRpZmYgLS1naXQNCj4gPj4gPj4+IGEvZHJpdmVycy9ncHUv
ZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2RldmljZS5jDQo+ID4+ID4+PiBiL2RyaXZlcnMvZ3B1L2Ry
bS9hbWQvYW1kZ3B1L2FtZGdwdV9kZXZpY2UuYw0KPiA+PiA+Pj4gaW5kZXggMTE4ZjRlNi4uOTI5
NTVmZSAxMDA2NDQNCj4gPj4gPj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2Ft
ZGdwdV9kZXZpY2UuYw0KPiA+PiA+Pj4gKysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUv
YW1kZ3B1X2RldmljZS5jDQo+ID4+ID4+PiBAQCAtNjkyLDYgKzY5MiwzNSBAQCB2b2lkIGFtZGdw
dV9ndHRfbG9jYXRpb24oc3RydWN0DQo+ID4+ID4+PiBhbWRncHVfZGV2aWNlDQo+ID4+ICphZGV2
LCBzdHJ1Y3QgYW1kZ3B1X21jICptYykNCj4gPj4gPj4+ICAgICAgICAgICAgICAgICAgICAgICAg
ICAgbWMtPmd0dF9zaXplID4+IDIwLCBtYy0+Z3R0X3N0YXJ0LCBtYy0+Z3R0X2VuZCk7DQo+ID4+
ID4+PiAgICB9DQo+ID4+ID4+Pg0KPiA+PiA+Pj4gKy8qKg0KPiA+PiA+Pj4gKyAqIGFtZGdwdV9y
ZXNpemVfYmFyMCAtIHRyeSB0byByZXNpemUgQkFSMA0KPiA+PiA+Pj4gKyAqDQo+ID4+ID4+PiAr
ICogQGFkZXY6IGFtZGdwdV9kZXZpY2UgcG9pbnRlcg0KPiA+PiA+Pj4gKyAqDQo+ID4+ID4+PiAr
ICogVHJ5IHRvIHJlc2l6ZSBCQVIwIHRvIG1ha2UgYWxsIFZSQU0gQ1BVIGFjY2Vzc2libGUuDQo+
ID4+ID4+PiArICovDQo+ID4+ID4+PiArdm9pZCBhbWRncHVfcmVzaXplX2JhcjAoc3RydWN0IGFt
ZGdwdV9kZXZpY2UgKmFkZXYpIHsNCj4gPj4gPj4+ICsgICAgICAgdTMyIHNpemUgPSBtYXgoaWxv
ZzIoYWRldi0+bWMucmVhbF92cmFtX3NpemUgLSAxKSArIDEsIDIwKSAtIDIwOw0KPiA+PiA+Pj4g
KyAgICAgICBpbnQgcjsNCj4gPj4gPj4+ICsNCj4gPj4gPj4+ICsgICAgICAgciA9IHBjaV9yZXNp
emVfcmVzb3VyY2UoYWRldi0+cGRldiwgMCwgc2l6ZSk7DQo+ID4+ID4+PiArDQo+ID4+ID4+PiAr
ICAgICAgIGlmIChyID09IC1FTk9UU1VQUCkgew0KPiA+PiA+Pj4gKyAgICAgICAgICAgICAgIC8q
IFRoZSBoYXJkd2FyZSBkb24ndCBzdXBwb3J0IHRoZSBleHRlbnNpb24uICovDQo+ID4+ID4+PiAr
ICAgICAgICAgICAgICAgcmV0dXJuOw0KPiA+PiA+Pj4gKw0KPiA+PiA+Pj4gKyAgICAgICB9IGVs
c2UgaWYgKHIgPT0gLUVOT1NQQykgew0KPiA+PiA+Pj4gKyAgICAgICAgICAgICAgIERSTV9JTkZP
KCJOb3QgZW5vaWdoIFBDSSBhZGRyZXNzIHNwYWNlIGZvciBhIGxhcmdlIEJBUi4iKTsNCj4gPj4g
Pj4+ICsgICAgICAgfSBlbHNlIGlmIChyKSB7DQo+ID4+ID4+PiArICAgICAgICAgICAgICAgRFJN
X0VSUk9SKCJQcm9ibGVtIHJlc2l6aW5nIEJBUjAgKCVkKS4iLCByKTsNCj4gPj4gPj4+ICsgICAg
ICAgfQ0KPiA+PiA+Pj4gKw0KPiA+PiA+Pj4gKyAgICAgICAvKiBSZWluaXQgdGhlIGRvb3JiZWxs
IG1hcHBpbmcsIGl0IGlzIG1vc3QgbGlrZWx5IG1vdmVkIGFzIHdlbGwgKi8NCj4gPj4gPj4+ICsg
ICAgICAgYW1kZ3B1X2Rvb3JiZWxsX2ZpbmkoYWRldik7DQo+ID4+ID4+PiArICAgICAgIEJVR19P
TihhbWRncHVfZG9vcmJlbGxfaW5pdChhZGV2KSk7DQo+ID4+ID4+PiArfQ0KPiA+PiA+Pj4gKw0K
PiA+PiA+Pj4gICAgLyoNCj4gPj4gPj4+ICAgICAqIEdQVSBoZWxwZXJzIGZ1bmN0aW9uLg0KPiA+
PiA+Pj4gICAgICovDQo+ID4+ID4+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJtL2FtZC9h
bWRncHUvZ21jX3Y3XzAuYw0KPiA+PiA+Pj4gYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9n
bWNfdjdfMC5jDQo+ID4+ID4+PiBpbmRleCBkYzliNmQ2Li4zNmE3YWE1IDEwMDY0NA0KPiA+PiA+
Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0KPiA+PiA+Pj4g
KysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y3XzAuYw0KPiA+PiA+Pj4gQEAg
LTM2NywxMyArMzY3LDE1IEBAIHN0YXRpYyBpbnQgZ21jX3Y3XzBfbWNfaW5pdChzdHJ1Y3QNCj4g
Pj4gPj4+IGFtZGdwdV9kZXZpY2UNCj4gPj4gKmFkZXYpDQo+ID4+ID4+PiAgICAgICAgICAgICAg
ICAgICBicmVhazsNCj4gPj4gPj4+ICAgICAgICAgICB9DQo+ID4+ID4+PiAgICAgICAgICAgYWRl
di0+bWMudnJhbV93aWR0aCA9IG51bWNoYW4gKiBjaGFuc2l6ZTsNCj4gPj4gPj4+IC0gICAgICAg
LyogQ291bGQgYXBlciBzaXplIHJlcG9ydCAwID8gKi8NCj4gPj4gPj4+IC0gICAgICAgYWRldi0+
bWMuYXBlcl9iYXNlID0gcGNpX3Jlc291cmNlX3N0YXJ0KGFkZXYtPnBkZXYsIDApOw0KPiA+PiA+
Pj4gLSAgICAgICBhZGV2LT5tYy5hcGVyX3NpemUgPSBwY2lfcmVzb3VyY2VfbGVuKGFkZXYtPnBk
ZXYsIDApOw0KPiA+PiA+Pj4gICAgICAgICAgIC8qIHNpemUgaW4gTUIgb24gc2kgKi8NCj4gPj4g
Pj4+ICAgICAgICAgICBhZGV2LT5tYy5tY192cmFtX3NpemUgPSBSUkVHMzIobW1DT05GSUdfTUVN
U0laRSkgKg0KPiA+PiA+Pj4gMTAyNFVMTCAqDQo+ID4+IDEwMjRVTEw7DQo+ID4+ID4+PiAgICAg
ICAgICAgYWRldi0+bWMucmVhbF92cmFtX3NpemUgPSBSUkVHMzIobW1DT05GSUdfTUVNU0laRSkg
Kg0KPiA+PiA+Pj4gMTAyNFVMTA0KPiA+PiA+Pj4gKiAxMDI0VUxMOw0KPiA+PiA+Pj4NCj4gPj4g
Pj4+ICsgICAgICAgaWYgKCEoYWRldi0+ZmxhZ3MgJiBBTURfSVNfQVBVKSkNCj4gPj4gPj4+ICsg
ICAgICAgICAgICAgICBhbWRncHVfcmVzaXplX2JhcjAoYWRldik7DQo+ID4+ID4+PiArICAgICAg
IGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9yZXNvdXJjZV9zdGFydChhZGV2LT5wZGV2LCAwKTsN
Cj4gPj4gPj4+ICsgICAgICAgYWRldi0+bWMuYXBlcl9zaXplID0gcGNpX3Jlc291cmNlX2xlbihh
ZGV2LT5wZGV2LCAwKTsNCj4gPj4gPj4+ICsNCj4gPj4gPj4+ICAgICNpZmRlZiBDT05GSUdfWDg2
XzY0DQo+ID4+ID4+PiAgICAgICAgICAgaWYgKGFkZXYtPmZsYWdzICYgQU1EX0lTX0FQVSkgew0K
PiA+PiA+Pj4gICAgICAgICAgICAgICAgICAgYWRldi0+bWMuYXBlcl9iYXNlID0NCj4gPj4gPj4+
ICgodTY0KVJSRUczMihtbU1DX1ZNX0ZCX09GRlNFVCkpIDw8IDIyOyBkaWZmIC0tZ2l0DQo+ID4+
ID4+PiBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2dtY192OF8wLmMNCj4gPj4gPj4+IGIv
ZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvZ21jX3Y4XzAuYw0KPiA+PiA+Pj4gaW5kZXggYzA4
N2IwMC4uNzc2MWFkMyAxMDA2NDQNCj4gPj4gPj4+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9hbWQv
YW1kZ3B1L2dtY192OF8wLmMNCj4gPj4gPj4+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1k
Z3B1L2dtY192OF8wLmMNCj4gPj4gPj4+IEBAIC00NTksMTMgKzQ1OSwxNSBAQCBzdGF0aWMgaW50
IGdtY192OF8wX21jX2luaXQoc3RydWN0DQo+ID4+ID4+PiBhbWRncHVfZGV2aWNlDQo+ID4+ICph
ZGV2KQ0KPiA+PiA+Pj4gICAgICAgICAgICAgICAgICAgYnJlYWs7DQo+ID4+ID4+PiAgICAgICAg
ICAgfQ0KPiA+PiA+Pj4gICAgICAgICAgIGFkZXYtPm1jLnZyYW1fd2lkdGggPSBudW1jaGFuICog
Y2hhbnNpemU7DQo+ID4+ID4+PiAtICAgICAgIC8qIENvdWxkIGFwZXIgc2l6ZSByZXBvcnQgMCA/
ICovDQo+ID4+ID4+PiAtICAgICAgIGFkZXYtPm1jLmFwZXJfYmFzZSA9IHBjaV9yZXNvdXJjZV9z
dGFydChhZGV2LT5wZGV2LCAwKTsNCj4gPj4gPj4+IC0gICAgICAgYWRldi0+bWMuYXBlcl9zaXpl
ID0gcGNpX3Jlc291cmNlX2xlbihhZGV2LT5wZGV2LCAwKTsNCj4gPj4gPj4+ICAgICAgICAgICAv
KiBzaXplIGluIE1CIG9uIHNpICovDQo+ID4+ID4+PiAgICAgICAgICAgYWRldi0+bWMubWNfdnJh
bV9zaXplID0gUlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gPj4gPj4+IDEwMjRVTEwgKg0K
PiA+PiAxMDI0VUxMOw0KPiA+PiA+Pj4gICAgICAgICAgIGFkZXYtPm1jLnJlYWxfdnJhbV9zaXpl
ID0gUlJFRzMyKG1tQ09ORklHX01FTVNJWkUpICoNCj4gPj4gPj4+IDEwMjRVTEwNCj4gPj4gPj4+
ICogMTAyNFVMTDsNCj4gPj4gPj4+DQo+ID4+ID4+PiArICAgICAgIGlmICghKGFkZXYtPmZsYWdz
ICYgQU1EX0lTX0FQVSkpDQo+ID4+ID4+PiArICAgICAgICAgICAgICAgYW1kZ3B1X3Jlc2l6ZV9i
YXIwKGFkZXYpOw0KPiA+PiA+Pj4gKyAgICAgICBhZGV2LT5tYy5hcGVyX2Jhc2UgPSBwY2lfcmVz
b3VyY2Vfc3RhcnQoYWRldi0+cGRldiwgMCk7DQo+ID4+ID4+PiArICAgICAgIGFkZXYtPm1jLmFw
ZXJfc2l6ZSA9IHBjaV9yZXNvdXJjZV9sZW4oYWRldi0+cGRldiwgMCk7DQo+ID4+ID4+PiArDQo+
ID4+ID4+PiAgICAjaWZkZWYgQ09ORklHX1g4Nl82NA0KPiA+PiA+Pj4gICAgICAgICAgIGlmIChh
ZGV2LT5mbGFncyAmIEFNRF9JU19BUFUpIHsNCj4gPj4gPj4+ICAgICAgICAgICAgICAgICAgIGFk
ZXYtPm1jLmFwZXJfYmFzZSA9DQo+ID4+ID4+PiAoKHU2NClSUkVHMzIobW1NQ19WTV9GQl9PRkZT
RVQpKSA8PCAyMjsNCj4gPj4gPj4+IC0tDQo+ID4+ID4+PiAyLjcuNA0KPiA+PiA+Pj4NCj4gPj4g
Pj4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+ID4+
ID4+PiBkcmktZGV2ZWwgbWFpbGluZyBsaXN0DQo+ID4+ID4+PiBkcmktZGV2ZWxAbGlzdHMuZnJl
ZWRlc2t0b3Aub3JnDQo+ID4+ID4+PiBodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWls
bWFuL2xpc3RpbmZvL2RyaS1kZXZlbA0KPiA+PiA+DQo+ID4+ID4gX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4gPiBkcmktZGV2ZWwgbWFpbGluZyBs
aXN0DQo+ID4+ID4gZHJpLWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+PiA+IGh0dHBz
Oi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsDQo+ID4+
ID4gX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NCj4gPj4g
PiBhbWQtZ2Z4IG1haWxpbmcgbGlzdA0KPiA+PiA+IGFtZC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Au
b3JnDQo+ID4+ID4gaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m
by9hbWQtZ2Z4DQo+ID4+DQo+ID4+DQo+ID4+IF9fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fDQo+ID4+IGRyaS1kZXZlbCBtYWlsaW5nIGxpc3QNCj4gPj4gZHJp
LWRldmVsQGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+PiBodHRwczovL2xpc3RzLmZyZWVkZXNr
dG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RyaS1kZXZlbA0KPiA+IF9fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQo+ID4gYW1kLWdmeCBtYWlsaW5nIGxpc3QN
Cj4gPiBhbWQtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9wLm9yZw0KPiA+IGh0dHBzOi8vbGlzdHMuZnJl
ZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vYW1kLWdmeA0K
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-16 2:41 ` Zhang, Jerry
(?)
@ 2017-03-23 14:30 ` Sagalovitch, Serguei
-1 siblings, 0 replies; 71+ messages in thread
From: Sagalovitch, Serguei @ 2017-03-23 14:30 UTC (permalink / raw)
To: Zhang, Jerry, Alex Deucher, Christian König
Cc: Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
Christian,
- Are we going to support resizing BAR when kernel
modesetting is not enabled and we are running in console
under VBIOS control (VESA/VGA)?
- Should we restore PCI configuration if amdgpu
will be unloaded?
- In function amdgpu_resize_bar0():
If resizing for "max" size failed should we try other
sizes? What do you think?
Sincerely yours,
Serguei Sagalovitch
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com>
Sent: March 15, 2017 10:41 PM
To: Alex Deucher
Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org
Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
Thanks for your info.
I see.
Regards,
Jerry (Junwei Zhang)
Linux Base Graphics
SRDC Software Development
_____________________________________
> -----Original Message-----
> From: Alex Deucher [mailto:alexdeucher@gmail.com]
> Sent: Thursday, March 16, 2017 10:25
> To: Zhang, Jerry
> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux-
> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-
> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
> >> -----Original Message-----
> >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> Behalf Of Christian K?nig
> >> Sent: Wednesday, March 15, 2017 17:29
> >> To: Zhou, David(ChunMing); Ayyappa Ch
> >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >>
> >> Yes, exactly that.
> >
> > (I'm not familiar with PCI too much.)
> > Is there any restrict for PCI device?
> > I'm concerning if any PCI couldn't support it on some motherboard.
>
> It depends on the PCI root bridge. This patch set only implements support for
> AMD root bridges. Intel and other vendors would need similar code.
>
> Alex
>
> >
> >>
> >> Christian.
> >>
> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> >> > Does that means we don't need invisible vram later?
> >> >
> >> > David
> >> >
> >> > -----Original Message-----
> >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> > Behalf Of Christian K?nig
> >> > Sent: Wednesday, March 15, 2017 3:38 PM
> >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >> >
> >> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
> >> > The CPU can access the full stolen VRAM directly on that hardware.
> >> >
> >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
> variants.
> >> >
> >> > Christian.
> >> >
> >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> >> >> Is it possible on Carrizo asics? Or only supports on newer asics?
> >> >>
> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> >> >> <deathsimple@vodafone.de> wrote:
> >> >>> From: Christian König <christian.koenig@amd.com>
> >> >>>
> >> >>> Try to resize BAR0 to let CPU access all of VRAM.
> >> >>>
> >> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
> >> >>> ---
> >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> >> +++++++++++++++++++++++++++++
> >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
> >> >>>
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> index 3b81ded..905ded9 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> >> amdgpu_device *adev, struct ttm_tt *ttm,
> >> >>> struct ttm_mem_reg *mem);
> >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> >> amdgpu_mc *mc, u64 base);
> >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> >> >>> amdgpu_mc *mc);
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device
> >> >>> *adev,
> >> u64 size);
> >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> index 118f4e6..92955fe 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct
> >> >>> amdgpu_device
> >> *adev, struct amdgpu_mc *mc)
> >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >> >>> }
> >> >>>
> >> >>> +/**
> >> >>> + * amdgpu_resize_bar0 - try to resize BAR0
> >> >>> + *
> >> >>> + * @adev: amdgpu_device pointer
> >> >>> + *
> >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >> >>> + */
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
> >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >> >>> + int r;
> >> >>> +
> >> >>> + r = pci_resize_resource(adev->pdev, 0, size);
> >> >>> +
> >> >>> + if (r == -ENOTSUPP) {
> >> >>> + /* The hardware don't support the extension. */
> >> >>> + return;
> >> >>> +
> >> >>> + } else if (r == -ENOSPC) {
> >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >> >>> + } else if (r) {
> >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >> >>> + }
> >> >>> +
> >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >> >>> + amdgpu_doorbell_fini(adev);
> >> >>> + BUG_ON(amdgpu_doorbell_init(adev));
> >> >>> +}
> >> >>> +
> >> >>> /*
> >> >>> * GPU helpers function.
> >> >>> */
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> index dc9b6d6..36a7aa5 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>> break;
> >> >>> }
> >> >>> adev->mc.vram_width = numchan * chansize;
> >> >>> - /* Could aper size report 0 ? */
> >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> /* size in MB on si */
> >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> + if (!(adev->flags & AMD_IS_APU))
> >> >>> + amdgpu_resize_bar0(adev);
> >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> +
> >> >>> #ifdef CONFIG_X86_64
> >> >>> if (adev->flags & AMD_IS_APU) {
> >> >>> adev->mc.aper_base =
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> index c087b00..7761ad3 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>> break;
> >> >>> }
> >> >>> adev->mc.vram_width = numchan * chansize;
> >> >>> - /* Could aper size report 0 ? */
> >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> /* size in MB on si */
> >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> + if (!(adev->flags & AMD_IS_APU))
> >> >>> + amdgpu_resize_bar0(adev);
> >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> +
> >> >>> #ifdef CONFIG_X86_64
> >> >>> if (adev->flags & AMD_IS_APU) {
> >> >>> adev->mc.aper_base =
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> >> >>> --
> >> >>> 2.7.4
> >> >>>
> >> >>> _______________________________________________
> >> >>> dri-devel mailing list
> >> >>> dri-devel@lists.freedesktop.org
> >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> >
> >> > _______________________________________________
> >> > dri-devel mailing list
> >> > dri-devel@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> > _______________________________________________
> >> > amd-gfx mailing list
> >> > amd-gfx@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> >>
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-23 14:30 ` Sagalovitch, Serguei
0 siblings, 0 replies; 71+ messages in thread
From: Sagalovitch, Serguei @ 2017-03-23 14:30 UTC (permalink / raw)
To: Zhang, Jerry, Alex Deucher, Christian König
Cc: Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
Christian,
- Are we going to support resizing BAR when kernel
modesetting is not enabled and we are running in console
under VBIOS control (VESA/VGA)?
- Should we restore PCI configuration if amdgpu
will be unloaded?
- In function amdgpu_resize_bar0():
If resizing for "max" size failed should we try other
sizes? What do you think?
Sincerely yours,
Serguei Sagalovitch
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com>
Sent: March 15, 2017 10:41 PM
To: Alex Deucher
Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org
Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
Thanks for your info.
I see.
Regards,
Jerry (Junwei Zhang)
Linux Base Graphics
SRDC Software Development
_____________________________________
> -----Original Message-----
> From: Alex Deucher [mailto:alexdeucher@gmail.com]
> Sent: Thursday, March 16, 2017 10:25
> To: Zhang, Jerry
> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux-
> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-
> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
> >> -----Original Message-----
> >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> Behalf Of Christian K?nig
> >> Sent: Wednesday, March 15, 2017 17:29
> >> To: Zhou, David(ChunMing); Ayyappa Ch
> >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >>
> >> Yes, exactly that.
> >
> > (I'm not familiar with PCI too much.)
> > Is there any restrict for PCI device?
> > I'm concerning if any PCI couldn't support it on some motherboard.
>
> It depends on the PCI root bridge. This patch set only implements support for
> AMD root bridges. Intel and other vendors would need similar code.
>
> Alex
>
> >
> >>
> >> Christian.
> >>
> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> >> > Does that means we don't need invisible vram later?
> >> >
> >> > David
> >> >
> >> > -----Original Message-----
> >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> > Behalf Of Christian K?nig
> >> > Sent: Wednesday, March 15, 2017 3:38 PM
> >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >> >
> >> > Carizzo is an APU and resizing BARs isn't needed nor supported there.
> >> > The CPU can access the full stolen VRAM directly on that hardware.
> >> >
> >> > As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
> variants.
> >> >
> >> > Christian.
> >> >
> >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> >> >> Is it possible on Carrizo asics? Or only supports on newer asics?
> >> >>
> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
> >> >> <deathsimple@vodafone.de> wrote:
> >> >>> From: Christian König <christian.koenig@amd.com>
> >> >>>
> >> >>> Try to resize BAR0 to let CPU access all of VRAM.
> >> >>>
> >> >>> Signed-off-by: Christian König <christian.koenig@amd.com>
> >> >>> ---
> >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> >> >>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> >> +++++++++++++++++++++++++++++
> >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> >> >>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> >> >>> 4 files changed, 40 insertions(+), 6 deletions(-)
> >> >>>
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> index 3b81ded..905ded9 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> >> amdgpu_device *adev, struct ttm_tt *ttm,
> >> >>> struct ttm_mem_reg *mem);
> >> >>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
> >> amdgpu_mc *mc, u64 base);
> >> >>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
> >> >>> amdgpu_mc *mc);
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >> >>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device
> >> >>> *adev,
> >> u64 size);
> >> >>> int amdgpu_ttm_init(struct amdgpu_device *adev);
> >> >>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> index 118f4e6..92955fe 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct
> >> >>> amdgpu_device
> >> *adev, struct amdgpu_mc *mc)
> >> >>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >> >>> }
> >> >>>
> >> >>> +/**
> >> >>> + * amdgpu_resize_bar0 - try to resize BAR0
> >> >>> + *
> >> >>> + * @adev: amdgpu_device pointer
> >> >>> + *
> >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >> >>> + */
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
> >> >>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> >> >>> + int r;
> >> >>> +
> >> >>> + r = pci_resize_resource(adev->pdev, 0, size);
> >> >>> +
> >> >>> + if (r == -ENOTSUPP) {
> >> >>> + /* The hardware don't support the extension. */
> >> >>> + return;
> >> >>> +
> >> >>> + } else if (r == -ENOSPC) {
> >> >>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
> >> >>> + } else if (r) {
> >> >>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> >> >>> + }
> >> >>> +
> >> >>> + /* Reinit the doorbell mapping, it is most likely moved as well */
> >> >>> + amdgpu_doorbell_fini(adev);
> >> >>> + BUG_ON(amdgpu_doorbell_init(adev));
> >> >>> +}
> >> >>> +
> >> >>> /*
> >> >>> * GPU helpers function.
> >> >>> */
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> index dc9b6d6..36a7aa5 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>> break;
> >> >>> }
> >> >>> adev->mc.vram_width = numchan * chansize;
> >> >>> - /* Could aper size report 0 ? */
> >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> /* size in MB on si */
> >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> + if (!(adev->flags & AMD_IS_APU))
> >> >>> + amdgpu_resize_bar0(adev);
> >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> +
> >> >>> #ifdef CONFIG_X86_64
> >> >>> if (adev->flags & AMD_IS_APU) {
> >> >>> adev->mc.aper_base =
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> index c087b00..7761ad3 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>> break;
> >> >>> }
> >> >>> adev->mc.vram_width = numchan * chansize;
> >> >>> - /* Could aper size report 0 ? */
> >> >>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> /* size in MB on si */
> >> >>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> + if (!(adev->flags & AMD_IS_APU))
> >> >>> + amdgpu_resize_bar0(adev);
> >> >>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
> >> >>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
> >> >>> +
> >> >>> #ifdef CONFIG_X86_64
> >> >>> if (adev->flags & AMD_IS_APU) {
> >> >>> adev->mc.aper_base =
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> >> >>> --
> >> >>> 2.7.4
> >> >>>
> >> >>> _______________________________________________
> >> >>> dri-devel mailing list
> >> >>> dri-devel@lists.freedesktop.org
> >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> >
> >> > _______________________________________________
> >> > dri-devel mailing list
> >> > dri-devel@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> > _______________________________________________
> >> > amd-gfx mailing list
> >> > amd-gfx@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> >>
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-23 14:30 ` Sagalovitch, Serguei
0 siblings, 0 replies; 71+ messages in thread
From: Sagalovitch, Serguei @ 2017-03-23 14:30 UTC (permalink / raw)
To: Zhang, Jerry, Alex Deucher, Christian König
Cc: Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
Christian,
- Are we going to support resizing BAR when kernel=20
modesetting is not enabled and we are running in console=20
under VBIOS control (VESA/VGA)?=20
- Should we restore PCI configuration if amdgpu
will be unloaded?
- In function amdgpu_resize_bar0():
If resizing for "max" size failed should we try other=20
sizes? What do you think?
Sincerely yours,
Serguei Sagalovitch
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, J=
erry <Jerry.Zhang@amd.com>
Sent: March 15, 2017 10:41 PM
To: Alex Deucher
Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-ker=
nel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@v=
ger.kernel.org; Christian K=F6nig; helgaas@kernel.org; amd-gfx@lists.freede=
sktop.org
Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
=A0 =20
Thanks for your info.
I see.
Regards,
Jerry (Junwei Zhang)
Linux Base Graphics
SRDC Software Development
_____________________________________
> -----Original Message-----
> From: Alex Deucher [mailto:alexdeucher@gmail.com]
> Sent: Thursday, March 16, 2017 10:25
> To: Zhang, Jerry
> Cc: Christian K=F6nig; Zhou, David(ChunMing); Ayyappa Ch; linux-
> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-
> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> helgaas@kernel.org; amd-gfx@lists.freedesktop.org
> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>=20
> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrot=
e:
> >> -----Original Message-----
> >> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> Behalf Of Christian K?nig
> >> Sent: Wednesday, March 15, 2017 17:29
> >> To: Zhou, David(ChunMing); Ayyappa Ch
> >> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
> >> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >>
> >> Yes, exactly that.
> >
> > (I'm not familiar with PCI too much.)
> > Is there any restrict for PCI device?
> > I'm concerning if any PCI couldn't support it on some motherboard.
>=20
> It depends on the PCI root bridge.=A0 This patch set only implements supp=
ort for
> AMD root bridges.=A0 Intel and other vendors would need similar code.
>=20
> Alex
>=20
> >
> >>
> >> Christian.
> >>
> >> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
> >> > Does that means we don't need invisible vram later?
> >> >
> >> > David
> >> >
> >> > -----Original Message-----
> >> > From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
> >> > Behalf Of Christian K?nig
> >> > Sent: Wednesday, March 15, 2017 3:38 PM
> >> > To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
> >> > Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
> >> > amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
> >> > helgaas@kernel.org; dri-devel@lists.freedesktop.org
> >> > Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
> >> >
> >> > Carizzo is an APU and resizing BARs isn't needed nor supported there=
.
> >> > The CPU can access the full stolen VRAM directly on that hardware.
> >> >
> >> > As far as I know ASICs with support for this are Tonga, Fiji and all=
Polaris
> variants.
> >> >
> >> > Christian.
> >> >
> >> > Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
> >> >> Is it possible on Carrizo asics? Or only supports on newer asics?
> >> >>
> >> >> On Mon, Mar 13, 2017 at 6:11 PM, Christian K=F6nig
> >> >> <deathsimple@vodafone.de> wrote:
> >> >>> From: Christian K=F6nig <christian.koenig@amd.com>
> >> >>>
> >> >>> Try to resize BAR0 to let CPU access all of VRAM.
> >> >>>
> >> >>> Signed-off-by: Christian K=F6nig <christian.koenig@amd.com>
> >> >>> ---
> >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/amdgpu.h=A0=A0=A0=A0=A0=A0=A0 =
|=A0 1 +
> >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
> >> +++++++++++++++++++++++++++++
> >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c=A0=A0=A0=A0=A0 |=A0=
8 +++++---
> >> >>>=A0=A0=A0 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c=A0=A0=A0=A0=A0 |=A0=
8 +++++---
> >> >>>=A0=A0=A0 4 files changed, 40 insertions(+), 6 deletions(-)
> >> >>>
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> index 3b81ded..905ded9 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> >> >>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
> >> amdgpu_device *adev, struct ttm_tt *ttm,
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 struct ttm_mem_reg *mem);
> >> >>>=A0=A0=A0 void amdgpu_vram_location(struct amdgpu_device *adev, str=
uct
> >> amdgpu_mc *mc, u64 base);
> >> >>>=A0=A0=A0 void amdgpu_gtt_location(struct amdgpu_device *adev, stru=
ct
> >> >>> amdgpu_mc *mc);
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> >> >>>=A0=A0=A0 void amdgpu_ttm_set_active_vram_size(struct amdgpu_device
> >> >>> *adev,
> >> u64 size);
> >> >>>=A0=A0=A0 int amdgpu_ttm_init(struct amdgpu_device *adev);
> >> >>>=A0=A0=A0 void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --=
git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> index 118f4e6..92955fe 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> >> >>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct
> >> >>> amdgpu_device
> >> *adev, struct amdgpu_mc *mc)
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=
=A0=A0=A0=A0 mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> >> >>>=A0=A0=A0 }
> >> >>>
> >> >>> +/**
> >> >>> + * amdgpu_resize_bar0 - try to resize BAR0
> >> >>> + *
> >> >>> + * @adev: amdgpu_device pointer
> >> >>> + *
> >> >>> + * Try to resize BAR0 to make all VRAM CPU accessible.
> >> >>> + */
> >> >>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
> >> >>> +=A0=A0=A0=A0=A0=A0 u32 size =3D max(ilog2(adev->mc.real_vram_size=
- 1) + 1, 20) - 20;
> >> >>> +=A0=A0=A0=A0=A0=A0 int r;
> >> >>> +
> >> >>> +=A0=A0=A0=A0=A0=A0 r =3D pci_resize_resource(adev->pdev, 0, size)=
;
> >> >>> +
> >> >>> +=A0=A0=A0=A0=A0=A0 if (r =3D=3D -ENOTSUPP) {
> >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* The hardware don't =
support the extension. */
> >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 return;
> >> >>> +
> >> >>> +=A0=A0=A0=A0=A0=A0 } else if (r =3D=3D -ENOSPC) {
> >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 DRM_INFO("Not enoigh P=
CI address space for a large BAR.");
> >> >>> +=A0=A0=A0=A0=A0=A0 } else if (r) {
> >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 DRM_ERROR("Problem res=
izing BAR0 (%d).", r);
> >> >>> +=A0=A0=A0=A0=A0=A0 }
> >> >>> +
> >> >>> +=A0=A0=A0=A0=A0=A0 /* Reinit the doorbell mapping, it is most lik=
ely moved as well */
> >> >>> +=A0=A0=A0=A0=A0=A0 amdgpu_doorbell_fini(adev);
> >> >>> +=A0=A0=A0=A0=A0=A0 BUG_ON(amdgpu_doorbell_init(adev));
> >> >>> +}
> >> >>> +
> >> >>>=A0=A0=A0 /*
> >> >>>=A0=A0=A0=A0 * GPU helpers function.
> >> >>>=A0=A0=A0=A0 */
> >> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> index dc9b6d6..36a7aa5 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
> >> >>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.vram_width =3D numchan * ch=
ansize;
> >> >>> -=A0=A0=A0=A0=A0=A0 /* Could aper size report 0 ? */
> >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev=
->pdev, 0);
> >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->=
pdev, 0);
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* size in MB on si */
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.mc_vram_size =3D RREG32(mmC=
ONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.real_vram_size =3D RREG32(m=
mCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> +=A0=A0=A0=A0=A0=A0 if (!(adev->flags & AMD_IS_APU))
> >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 amdgpu_resize_bar0(ade=
v);
> >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev=
->pdev, 0);
> >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->=
pdev, 0);
> >> >>> +
> >> >>>=A0=A0=A0 #ifdef CONFIG_X86_64
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (adev->flags & AMD_IS_APU) {
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.ape=
r_base =3D
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
> >> >>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> index c087b00..7761ad3 100644
> >> >>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
> >> >>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
> >> >>> amdgpu_device
> >> *adev)
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 break;
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 }
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.vram_width =3D numchan * ch=
ansize;
> >> >>> -=A0=A0=A0=A0=A0=A0 /* Could aper size report 0 ? */
> >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev=
->pdev, 0);
> >> >>> -=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->=
pdev, 0);
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 /* size in MB on si */
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.mc_vram_size =3D RREG32(mmC=
ONFIG_MEMSIZE) *
> >> >>> 1024ULL *
> >> 1024ULL;
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.real_vram_size =3D RREG32(m=
mCONFIG_MEMSIZE) *
> >> >>> 1024ULL
> >> >>> * 1024ULL;
> >> >>>
> >> >>> +=A0=A0=A0=A0=A0=A0 if (!(adev->flags & AMD_IS_APU))
> >> >>> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 amdgpu_resize_bar0(ade=
v);
> >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_base =3D pci_resource_start(adev=
->pdev, 0);
> >> >>> +=A0=A0=A0=A0=A0=A0 adev->mc.aper_size =3D pci_resource_len(adev->=
pdev, 0);
> >> >>> +
> >> >>>=A0=A0=A0 #ifdef CONFIG_X86_64
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 if (adev->flags & AMD_IS_APU) {
> >> >>>=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 adev->mc.ape=
r_base =3D
> >> >>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
> >> >>> --
> >> >>> 2.7.4
> >> >>>
> >> >>> _______________________________________________
> >> >>> dri-devel mailing list
> >> >>> dri-devel@lists.freedesktop.org
> >> >>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> >
> >> > _______________________________________________
> >> > dri-devel mailing list
> >> > dri-devel@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/dri-devel
> >> > _______________________________________________
> >> > amd-gfx mailing list
> >> > amd-gfx@lists.freedesktop.org
> >> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> >>
> >>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
> > _______________________________________________
> > amd-gfx mailing list
> > amd-gfx@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
=
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-23 14:30 ` Sagalovitch, Serguei
@ 2017-03-23 15:56 ` Christian König
-1 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-23 15:56 UTC (permalink / raw)
To: Sagalovitch, Serguei, Zhang, Jerry, Alex Deucher
Cc: Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
> - Are we going to support resizing BAR when kernel
> modesetting is not enabled and we are running in console
> under VBIOS control (VESA/VGA)?
No, initial I've tried to resize the PCI BAR during probing without the
help of the driver at all. But the VESA/EFI/VBIOS don't seem to be able
to handle addresses above 4GB for some reason.
So the approach is to let the driver kick the VESA/EFI drivers out and
then resize when we know that nobody is accessing the BAR.
That's the only approach I've found without either blacklisting VESA/EFI
drivers or crashing the system during the resize.
> - Should we restore PCI configuration if amdgpu
> will be unloaded?
Yeah, thought about the as well. I'm just not sure how to do it.
There is a lot of stuff we need to save and reset when the driver
unloads for not much gain.
> - In function amdgpu_resize_bar0():
> If resizing for "max" size failed should we try other
> sizes? What do you think?
Probably not worth it. If we get the BAR moved to a 64bit address we
should have enough address space in almost all cases, so setting it to
the maximum should succeed.
But I think we could add another parameter to allow limiting the resized
size for all corner cases and for testing.
Regards,
Christian.
Am 23.03.2017 um 15:30 schrieb Sagalovitch, Serguei:
> Christian,
>
> - Are we going to support resizing BAR when kernel
> modesetting is not enabled and we are running in console
> under VBIOS control (VESA/VGA)?
>
> - Should we restore PCI configuration if amdgpu
> will be unloaded?
>
> - In function amdgpu_resize_bar0():
> If resizing for "max" size failed should we try other
> sizes? What do you think?
>
>
> Sincerely yours,
> Serguei Sagalovitch
>
>
> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com>
> Sent: March 15, 2017 10:41 PM
> To: Alex Deucher
> Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org
> Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Thanks for your info.
> I see.
>
> Regards,
> Jerry (Junwei Zhang)
>
> Linux Base Graphics
> SRDC Software Development
> _____________________________________
>
>
>> -----Original Message-----
>> From: Alex Deucher [mailto:alexdeucher@gmail.com]
>> Sent: Thursday, March 16, 2017 10:25
>> To: Zhang, Jerry
>> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux-
>> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-
>> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> helgaas@kernel.org; amd-gfx@lists.freedesktop.org
>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>
>> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
>>>> -----Original Message-----
>>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>>>> Behalf Of Christian K?nig
>>>> Sent: Wednesday, March 15, 2017 17:29
>>>> To: Zhou, David(ChunMing); Ayyappa Ch
>>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
>>>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>>>
>>>> Yes, exactly that.
>>> (I'm not familiar with PCI too much.)
>>> Is there any restrict for PCI device?
>>> I'm concerning if any PCI couldn't support it on some motherboard.
>> It depends on the PCI root bridge. This patch set only implements support for
>> AMD root bridges. Intel and other vendors would need similar code.
>>
>> Alex
>>
>>>> Christian.
>>>>
>>>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
>>>>> Does that means we don't need invisible vram later?
>>>>>
>>>>> David
>>>>>
>>>>> -----Original Message-----
>>>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>>>>> Behalf Of Christian K?nig
>>>>> Sent: Wednesday, March 15, 2017 3:38 PM
>>>>> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
>>>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
>>>>> amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>>>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>>>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>>>>
>>>>> Carizzo is an APU and resizing BARs isn't needed nor supported there.
>>>>> The CPU can access the full stolen VRAM directly on that hardware.
>>>>>
>>>>> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
>> variants.
>>>>> Christian.
>>>>>
>>>>> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>>>>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>>>>>
>>>>>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>>>>>> <deathsimple@vodafone.de> wrote:
>>>>>>> From: Christian König <christian.koenig@amd.com>
>>>>>>>
>>>>>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>>>>>
>>>>>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>>>>>> ---
>>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>>> +++++++++++++++++++++++++++++
>>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>>>>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> index 3b81ded..905ded9 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>>>>>> struct ttm_mem_reg *mem);
>>>>>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
>>>> amdgpu_mc *mc, u64 base);
>>>>>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>>>>>>> amdgpu_mc *mc);
>>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>>>>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device
>>>>>>> *adev,
>>>> u64 size);
>>>>>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>>>>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>>>>>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> index 118f4e6..92955fe 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct
>>>>>>> amdgpu_device
>>>> *adev, struct amdgpu_mc *mc)
>>>>>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>>>>>> }
>>>>>>>
>>>>>>> +/**
>>>>>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>>>>>> + *
>>>>>>> + * @adev: amdgpu_device pointer
>>>>>>> + *
>>>>>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>>>>>> + */
>>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>>>>>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>>>>>> + int r;
>>>>>>> +
>>>>>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>>>>>> +
>>>>>>> + if (r == -ENOTSUPP) {
>>>>>>> + /* The hardware don't support the extension. */
>>>>>>> + return;
>>>>>>> +
>>>>>>> + } else if (r == -ENOSPC) {
>>>>>>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>>>>>>> + } else if (r) {
>>>>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>>>>>> + }
>>>>>>> +
>>>>>>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>>>>>>> + amdgpu_doorbell_fini(adev);
>>>>>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>>>>>> +}
>>>>>>> +
>>>>>>> /*
>>>>>>> * GPU helpers function.
>>>>>>> */
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> index dc9b6d6..36a7aa5 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
>>>>>>> amdgpu_device
>>>> *adev)
>>>>>>> break;
>>>>>>> }
>>>>>>> adev->mc.vram_width = numchan * chansize;
>>>>>>> - /* Could aper size report 0 ? */
>>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> /* size in MB on si */
>>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL *
>>>> 1024ULL;
>>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL
>>>>>>> * 1024ULL;
>>>>>>>
>>>>>>> + if (!(adev->flags & AMD_IS_APU))
>>>>>>> + amdgpu_resize_bar0(adev);
>>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> +
>>>>>>> #ifdef CONFIG_X86_64
>>>>>>> if (adev->flags & AMD_IS_APU) {
>>>>>>> adev->mc.aper_base =
>>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>>>>>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> index c087b00..7761ad3 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
>>>>>>> amdgpu_device
>>>> *adev)
>>>>>>> break;
>>>>>>> }
>>>>>>> adev->mc.vram_width = numchan * chansize;
>>>>>>> - /* Could aper size report 0 ? */
>>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> /* size in MB on si */
>>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL *
>>>> 1024ULL;
>>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL
>>>>>>> * 1024ULL;
>>>>>>>
>>>>>>> + if (!(adev->flags & AMD_IS_APU))
>>>>>>> + amdgpu_resize_bar0(adev);
>>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> +
>>>>>>> #ifdef CONFIG_X86_64
>>>>>>> if (adev->flags & AMD_IS_APU) {
>>>>>>> adev->mc.aper_base =
>>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>>>>>>> --
>>>>>>> 2.7.4
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> dri-devel mailing list
>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>> _______________________________________________
>>>>> dri-devel mailing list
>>>>> dri-devel@lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>> _______________________________________________
>>>>> amd-gfx mailing list
>>>>> amd-gfx@lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>> _______________________________________________
>>> amd-gfx mailing list
>>> amd-gfx@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
@ 2017-03-23 15:56 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-03-23 15:56 UTC (permalink / raw)
To: Sagalovitch, Serguei, Zhang, Jerry, Alex Deucher
Cc: Zhou, David(ChunMing),
Ayyappa Ch, linux-pci, linux-kernel, dri-devel,
platform-driver-x86, helgaas, amd-gfx
> - Are we going to support resizing BAR when kernel
> modesetting is not enabled and we are running in console
> under VBIOS control (VESA/VGA)?
No, initial I've tried to resize the PCI BAR during probing without the
help of the driver at all. But the VESA/EFI/VBIOS don't seem to be able
to handle addresses above 4GB for some reason.
So the approach is to let the driver kick the VESA/EFI drivers out and
then resize when we know that nobody is accessing the BAR.
That's the only approach I've found without either blacklisting VESA/EFI
drivers or crashing the system during the resize.
> - Should we restore PCI configuration if amdgpu
> will be unloaded?
Yeah, thought about the as well. I'm just not sure how to do it.
There is a lot of stuff we need to save and reset when the driver
unloads for not much gain.
> - In function amdgpu_resize_bar0():
> If resizing for "max" size failed should we try other
> sizes? What do you think?
Probably not worth it. If we get the BAR moved to a 64bit address we
should have enough address space in almost all cases, so setting it to
the maximum should succeed.
But I think we could add another parameter to allow limiting the resized
size for all corner cases and for testing.
Regards,
Christian.
Am 23.03.2017 um 15:30 schrieb Sagalovitch, Serguei:
> Christian,
>
> - Are we going to support resizing BAR when kernel
> modesetting is not enabled and we are running in console
> under VBIOS control (VESA/VGA)?
>
> - Should we restore PCI configuration if amdgpu
> will be unloaded?
>
> - In function amdgpu_resize_bar0():
> If resizing for "max" size failed should we try other
> sizes? What do you think?
>
>
> Sincerely yours,
> Serguei Sagalovitch
>
>
> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Zhang, Jerry <Jerry.Zhang@amd.com>
> Sent: March 15, 2017 10:41 PM
> To: Alex Deucher
> Cc: Zhou, David(ChunMing); Ayyappa Ch; linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org; Christian König; helgaas@kernel.org; amd-gfx@lists.freedesktop.org
> Subject: RE: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>
> Thanks for your info.
> I see.
>
> Regards,
> Jerry (Junwei Zhang)
>
> Linux Base Graphics
> SRDC Software Development
> _____________________________________
>
>
>> -----Original Message-----
>> From: Alex Deucher [mailto:alexdeucher@gmail.com]
>> Sent: Thursday, March 16, 2017 10:25
>> To: Zhang, Jerry
>> Cc: Christian König; Zhou, David(ChunMing); Ayyappa Ch; linux-
>> pci@vger.kernel.org; linux-kernel@vger.kernel.org; dri-
>> devel@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>> helgaas@kernel.org; amd-gfx@lists.freedesktop.org
>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>
>> On Wed, Mar 15, 2017 at 10:19 PM, Zhang, Jerry <Jerry.Zhang@amd.com> wrote:
>>>> -----Original Message-----
>>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>>>> Behalf Of Christian K?nig
>>>> Sent: Wednesday, March 15, 2017 17:29
>>>> To: Zhou, David(ChunMing); Ayyappa Ch
>>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org; amd-
>>>> gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>>>
>>>> Yes, exactly that.
>>> (I'm not familiar with PCI too much.)
>>> Is there any restrict for PCI device?
>>> I'm concerning if any PCI couldn't support it on some motherboard.
>> It depends on the PCI root bridge. This patch set only implements support for
>> AMD root bridges. Intel and other vendors would need similar code.
>>
>> Alex
>>
>>>> Christian.
>>>>
>>>> Am 15.03.2017 um 09:25 schrieb Zhou, David(ChunMing):
>>>>> Does that means we don't need invisible vram later?
>>>>>
>>>>> David
>>>>>
>>>>> -----Original Message-----
>>>>> From: dri-devel [mailto:dri-devel-bounces@lists.freedesktop.org] On
>>>>> Behalf Of Christian K?nig
>>>>> Sent: Wednesday, March 15, 2017 3:38 PM
>>>>> To: Ayyappa Ch <ayyappa.ch.linux@gmail.com>
>>>>> Cc: linux-pci@vger.kernel.org; linux-kernel@vger.kernel.org;
>>>>> amd-gfx@lists.freedesktop.org; platform-driver-x86@vger.kernel.org;
>>>>> helgaas@kernel.org; dri-devel@lists.freedesktop.org
>>>>> Subject: Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
>>>>>
>>>>> Carizzo is an APU and resizing BARs isn't needed nor supported there.
>>>>> The CPU can access the full stolen VRAM directly on that hardware.
>>>>>
>>>>> As far as I know ASICs with support for this are Tonga, Fiji and all Polaris
>> variants.
>>>>> Christian.
>>>>>
>>>>> Am 15.03.2017 um 08:23 schrieb Ayyappa Ch:
>>>>>> Is it possible on Carrizo asics? Or only supports on newer asics?
>>>>>>
>>>>>> On Mon, Mar 13, 2017 at 6:11 PM, Christian König
>>>>>> <deathsimple@vodafone.de> wrote:
>>>>>>> From: Christian König <christian.koenig@amd.com>
>>>>>>>
>>>>>>> Try to resize BAR0 to let CPU access all of VRAM.
>>>>>>>
>>>>>>> Signed-off-by: Christian König <christian.koenig@amd.com>
>>>>>>> ---
>>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
>>>>>>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29
>>>> +++++++++++++++++++++++++++++
>>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
>>>>>>> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
>>>>>>> 4 files changed, 40 insertions(+), 6 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> index 3b81ded..905ded9 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>>>> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct
>>>> amdgpu_device *adev, struct ttm_tt *ttm,
>>>>>>> struct ttm_mem_reg *mem);
>>>>>>> void amdgpu_vram_location(struct amdgpu_device *adev, struct
>>>> amdgpu_mc *mc, u64 base);
>>>>>>> void amdgpu_gtt_location(struct amdgpu_device *adev, struct
>>>>>>> amdgpu_mc *mc);
>>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
>>>>>>> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device
>>>>>>> *adev,
>>>> u64 size);
>>>>>>> int amdgpu_ttm_init(struct amdgpu_device *adev);
>>>>>>> void amdgpu_ttm_fini(struct amdgpu_device *adev); diff --git
>>>>>>> a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> index 118f4e6..92955fe 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>>>> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct
>>>>>>> amdgpu_device
>>>> *adev, struct amdgpu_mc *mc)
>>>>>>> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
>>>>>>> }
>>>>>>>
>>>>>>> +/**
>>>>>>> + * amdgpu_resize_bar0 - try to resize BAR0
>>>>>>> + *
>>>>>>> + * @adev: amdgpu_device pointer
>>>>>>> + *
>>>>>>> + * Try to resize BAR0 to make all VRAM CPU accessible.
>>>>>>> + */
>>>>>>> +void amdgpu_resize_bar0(struct amdgpu_device *adev) {
>>>>>>> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
>>>>>>> + int r;
>>>>>>> +
>>>>>>> + r = pci_resize_resource(adev->pdev, 0, size);
>>>>>>> +
>>>>>>> + if (r == -ENOTSUPP) {
>>>>>>> + /* The hardware don't support the extension. */
>>>>>>> + return;
>>>>>>> +
>>>>>>> + } else if (r == -ENOSPC) {
>>>>>>> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
>>>>>>> + } else if (r) {
>>>>>>> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
>>>>>>> + }
>>>>>>> +
>>>>>>> + /* Reinit the doorbell mapping, it is most likely moved as well */
>>>>>>> + amdgpu_doorbell_fini(adev);
>>>>>>> + BUG_ON(amdgpu_doorbell_init(adev));
>>>>>>> +}
>>>>>>> +
>>>>>>> /*
>>>>>>> * GPU helpers function.
>>>>>>> */
>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> index dc9b6d6..36a7aa5 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>>>> @@ -367,13 +367,15 @@ static int gmc_v7_0_mc_init(struct
>>>>>>> amdgpu_device
>>>> *adev)
>>>>>>> break;
>>>>>>> }
>>>>>>> adev->mc.vram_width = numchan * chansize;
>>>>>>> - /* Could aper size report 0 ? */
>>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> /* size in MB on si */
>>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL *
>>>> 1024ULL;
>>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL
>>>>>>> * 1024ULL;
>>>>>>>
>>>>>>> + if (!(adev->flags & AMD_IS_APU))
>>>>>>> + amdgpu_resize_bar0(adev);
>>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> +
>>>>>>> #ifdef CONFIG_X86_64
>>>>>>> if (adev->flags & AMD_IS_APU) {
>>>>>>> adev->mc.aper_base =
>>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22; diff --git
>>>>>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> index c087b00..7761ad3 100644
>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
>>>>>>> @@ -459,13 +459,15 @@ static int gmc_v8_0_mc_init(struct
>>>>>>> amdgpu_device
>>>> *adev)
>>>>>>> break;
>>>>>>> }
>>>>>>> adev->mc.vram_width = numchan * chansize;
>>>>>>> - /* Could aper size report 0 ? */
>>>>>>> - adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> - adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> /* size in MB on si */
>>>>>>> adev->mc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL *
>>>> 1024ULL;
>>>>>>> adev->mc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>>>> 1024ULL
>>>>>>> * 1024ULL;
>>>>>>>
>>>>>>> + if (!(adev->flags & AMD_IS_APU))
>>>>>>> + amdgpu_resize_bar0(adev);
>>>>>>> + adev->mc.aper_base = pci_resource_start(adev->pdev, 0);
>>>>>>> + adev->mc.aper_size = pci_resource_len(adev->pdev, 0);
>>>>>>> +
>>>>>>> #ifdef CONFIG_X86_64
>>>>>>> if (adev->flags & AMD_IS_APU) {
>>>>>>> adev->mc.aper_base =
>>>>>>> ((u64)RREG32(mmMC_VM_FB_OFFSET)) << 22;
>>>>>>> --
>>>>>>> 2.7.4
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> dri-devel mailing list
>>>>>>> dri-devel@lists.freedesktop.org
>>>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>> _______________________________________________
>>>>> dri-devel mailing list
>>>>> dri-devel@lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>>>> _______________________________________________
>>>>> amd-gfx mailing list
>>>>> amd-gfx@lists.freedesktop.org
>>>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>>>
>>>> _______________________________________________
>>>> dri-devel mailing list
>>>> dri-devel@lists.freedesktop.org
>>>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>>> _______________________________________________
>>> amd-gfx mailing list
>>> amd-gfx@lists.freedesktop.org
>>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 1/4] PCI: add resizeable BAR infrastructure v3
2017-03-13 12:41 ` Christian König
(?)
(?)
@ 2017-03-24 15:28 ` Bjorn Helgaas
-1 siblings, 0 replies; 71+ messages in thread
From: Bjorn Helgaas @ 2017-03-24 15:28 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 01:41:33PM +0100, Christian König wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Just the defines and helper functions to read the possible sizes of a BAR and
> update it's size.
s/it's/its/
> See https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdf.
It's good to have the public ECN that anybody can read, but we should
also have a reference to the full spec that incorporates it, e.g.,
PCIe r3.1, sec 7.22.
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1946,6 +1946,9 @@ void pci_request_acs(void);
> bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags);
> bool pci_acs_path_enabled(struct pci_dev *start,
> struct pci_dev *end, u16 acs_flags);
> +u32 pci_rbar_get_possible_sizes(struct pci_dev *pdev, int bar);
> +int pci_rbar_get_current_size(struct pci_dev *pdev, int bar);
> +int pci_rbar_set_size(struct pci_dev *pdev, int bar, int size);
These should be declared in drivers/pci/pci.h unless they're needed
outside drivers/pci. I hope they aren't needed outside, because
they're not safe to use after the PCI core has claimed resources.
> #define PCI_VPD_LRDT 0x80 /* Large Resource Data Type */
> #define PCI_VPD_LRDT_ID(x) ((x) | PCI_VPD_LRDT)
> diff --git a/include/uapi/linux/pci_regs.h b/include/uapi/linux/pci_regs.h
> index e5a2e68..6de29d6 100644
> --- a/include/uapi/linux/pci_regs.h
> +++ b/include/uapi/linux/pci_regs.h
> @@ -932,9 +932,16 @@
> #define PCI_SATA_SIZEOF_LONG 16
>
> /* Resizable BARs */
> +#define PCI_REBAR_CAP 4 /* capability register */
> +#define PCI_REBAR_CTRL_SIZES_MASK (0xFFFFF << 4) /* mask for sizes */
> +#define PCI_REBAR_CTRL_SIZES_SHIFT 4 /* shift for sizes */
> #define PCI_REBAR_CTRL 8 /* control register */
> +#define PCI_REBAR_CTRL_BAR_IDX_MASK (7 << 0) /* mask for bar index */
> +#define PCI_REBAR_CTRL_BAR_IDX_SHIFT 0 /* shift for bar index */
> #define PCI_REBAR_CTRL_NBAR_MASK (7 << 5) /* mask for # bars */
> #define PCI_REBAR_CTRL_NBAR_SHIFT 5 /* shift for # bars */
> +#define PCI_REBAR_CTRL_BAR_SIZE_MASK (0x1F << 8) /* mask for bar size */
> +#define PCI_REBAR_CTRL_BAR_SIZE_SHIFT 8 /* shift for bar size */
s/bar/BAR/ several places above
> /* Dynamic Power Allocation */
> #define PCI_DPA_CAP 4 /* capability register */
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
2017-03-13 12:41 ` Christian König
@ 2017-03-24 15:47 ` Bjorn Helgaas
-1 siblings, 0 replies; 71+ messages in thread
From: Bjorn Helgaas @ 2017-03-24 15:47 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Most BIOS don't enable this because of compatibility reasons.
Can you give any more details here? Without more hints, it's hard to
know whether any of the compatibility reasons might apply to Linux as
well.
> Manually enable a 64bit BAR of 64GB size so that we have
> enough room for PCI devices.
>From the context, I'm guessing this is enabling a new 64GB window
through the PCI host bridge? That might be documented as a "BAR", but
it's not anything the Linux PCI core would recognize as a BAR.
I think the specs would envision this being done via an ACPI _SRS
method on the PNP0A03 host bridge device. That would be a more
generic path that would work on any host bridge. Did you explore that
possibility? I would prefer to avoid adding device-specific code if
that's possible.
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 53 insertions(+)
>
> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
> index 6d52b94..bff5242 100644
> --- a/arch/x86/pci/fixup.c
> +++ b/arch/x86/pci/fixup.c
> @@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar);
> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar);
> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar);
> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar);
> +
> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
> +{
> + const uint64_t size = 64ULL * 1024 * 1024 * 1024;
> + uint32_t base, limit, high;
> + struct resource *res;
> + unsigned i;
> + int r;
> +
> + for (i = 0; i < 8; ++i) {
> +
> + pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
> + pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
> +
> + /* Is this slot free? */
> + if ((base & 0x3) == 0x0)
> + break;
> +
> + base >>= 8;
> + base |= high << 24;
> +
> + /* Abort if a slot already configures a 64bit BAR. */
> + if (base > 0x10000)
> + return;
> +
> + }
> +
> + if (i == 8)
> + return;
> +
> + res = kzalloc(sizeof(*res), GFP_KERNEL);
> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
> + IORESOURCE_WINDOW;
> + res->name = dev->bus->name;
> + r = allocate_resource(&iomem_resource, res, size, 0x100000000,
> + 0xfd00000000, size, NULL, NULL);
> + if (r) {
> + kfree(res);
> + return;
> + }
> +
> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
> + limit = ((res->end + 1) >> 8) & 0xffffff00;
> + high = ((res->start >> 40) & 0xff) |
> + ((((res->end + 1) >> 40) & 0xff) << 16);
> +
> + pci_write_config_dword(dev, 0x180 + i * 0x4, high);
> + pci_write_config_dword(dev, 0x84 + i * 0x8, limit);
> + pci_write_config_dword(dev, 0x80 + i * 0x8, base);
> +
> + pci_bus_add_resource(dev->bus, res, 0);
We would need some sort of printk here to explain how this new window
magically appeared.
> +}
> +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar);
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-03-24 15:47 ` Bjorn Helgaas
0 siblings, 0 replies; 71+ messages in thread
From: Bjorn Helgaas @ 2017-03-24 15:47 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Most BIOS don't enable this because of compatibility reasons.
Can you give any more details here? Without more hints, it's hard to
know whether any of the compatibility reasons might apply to Linux as
well.
> Manually enable a 64bit BAR of 64GB size so that we have
> enough room for PCI devices.
From the context, I'm guessing this is enabling a new 64GB window
through the PCI host bridge? That might be documented as a "BAR", but
it's not anything the Linux PCI core would recognize as a BAR.
I think the specs would envision this being done via an ACPI _SRS
method on the PNP0A03 host bridge device. That would be a more
generic path that would work on any host bridge. Did you explore that
possibility? I would prefer to avoid adding device-specific code if
that's possible.
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> arch/x86/pci/fixup.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 53 insertions(+)
>
> diff --git a/arch/x86/pci/fixup.c b/arch/x86/pci/fixup.c
> index 6d52b94..bff5242 100644
> --- a/arch/x86/pci/fixup.c
> +++ b/arch/x86/pci/fixup.c
> @@ -571,3 +571,56 @@ DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x2fc0, pci_invalid_bar);
> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6f60, pci_invalid_bar);
> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fa0, pci_invalid_bar);
> DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x6fc0, pci_invalid_bar);
> +
> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
> +{
> + const uint64_t size = 64ULL * 1024 * 1024 * 1024;
> + uint32_t base, limit, high;
> + struct resource *res;
> + unsigned i;
> + int r;
> +
> + for (i = 0; i < 8; ++i) {
> +
> + pci_read_config_dword(dev, 0x80 + i * 0x8, &base);
> + pci_read_config_dword(dev, 0x180 + i * 0x4, &high);
> +
> + /* Is this slot free? */
> + if ((base & 0x3) == 0x0)
> + break;
> +
> + base >>= 8;
> + base |= high << 24;
> +
> + /* Abort if a slot already configures a 64bit BAR. */
> + if (base > 0x10000)
> + return;
> +
> + }
> +
> + if (i == 8)
> + return;
> +
> + res = kzalloc(sizeof(*res), GFP_KERNEL);
> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
> + IORESOURCE_WINDOW;
> + res->name = dev->bus->name;
> + r = allocate_resource(&iomem_resource, res, size, 0x100000000,
> + 0xfd00000000, size, NULL, NULL);
> + if (r) {
> + kfree(res);
> + return;
> + }
> +
> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
> + limit = ((res->end + 1) >> 8) & 0xffffff00;
> + high = ((res->start >> 40) & 0xff) |
> + ((((res->end + 1) >> 40) & 0xff) << 16);
> +
> + pci_write_config_dword(dev, 0x180 + i * 0x4, high);
> + pci_write_config_dword(dev, 0x84 + i * 0x8, limit);
> + pci_write_config_dword(dev, 0x80 + i * 0x8, base);
> +
> + pci_bus_add_resource(dev->bus, res, 0);
We would need some sort of printk here to explain how this new window
magically appeared.
> +}
> +DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x141b, pci_amd_enable_64bit_bar);
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
2017-03-13 12:41 ` Christian König
` (2 preceding siblings ...)
(?)
@ 2017-03-24 21:34 ` Bjorn Helgaas
2017-04-11 15:37 ` Christian König
-1 siblings, 1 reply; 71+ messages in thread
From: Bjorn Helgaas @ 2017-03-24 21:34 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 01:41:34PM +0100, Christian König wrote:
> From: Christian König <christian.koenig@amd.com>
>
> This allows device drivers to request resizing their BARs.
>
> The function only tries to reprogram the windows of the bridge directly above
> the requesting device and only the BAR of the same type (usually mem, 64bit,
> prefetchable). This is done to make sure not to disturb other drivers by
> changing the BARs of their devices.
>
> If reprogramming the bridge BAR fails the old status is restored and -ENOSPC
> returned to the calling device driver.
>
> v2: rebase on changes in rbar support
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/pci/setup-bus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
> drivers/pci/setup-res.c | 51 +++++++++++++++++++++++++++++++++++++++++
> include/linux/pci.h | 2 ++
> 3 files changed, 114 insertions(+)
>
> diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
> index f30ca75..cfab2c7 100644
> --- a/drivers/pci/setup-bus.c
> +++ b/drivers/pci/setup-bus.c
> @@ -1923,6 +1923,67 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge)
> }
> EXPORT_SYMBOL_GPL(pci_assign_unassigned_bridge_resources);
>
> +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type)
> +{
> + const unsigned long type_mask = IORESOURCE_IO | IORESOURCE_MEM |
> + IORESOURCE_PREFETCH | IORESOURCE_MEM_64;
> +
> + struct resource saved;
> + LIST_HEAD(add_list);
> + LIST_HEAD(fail_head);
> + struct pci_dev_resource *fail_res;
> + unsigned i;
> + int ret = 0;
> +
> + /* Release all children from the matching bridge resource */
> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
Nit: use post-increment unless you need pre-increment.
> + struct resource *res = &bridge->resource[i];
> +
> + if ((res->flags & type_mask) != (type & type_mask))
> + continue;
> +
> + saved = *res;
> + if (res->parent) {
> + release_child_resources(res);
Doesn't this recursively release *all* child resources? There could
be BARs from several devices, or even windows of downstream bridges,
inside this window. The drivers of those other devices aren't
expecting things to change here.
> + release_resource(res);
> + }
> + res->start = 0;
> + res->end = 0;
> + break;
> + }
> +
> + if (i == PCI_BRIDGE_RESOURCE_END)
> + return -ENOENT;
> +
> + __pci_bus_size_bridges(bridge->subordinate, &add_list);
> + __pci_bridge_assign_resources(bridge, &add_list, &fail_head);
> + BUG_ON(!list_empty(&add_list));
> +
> + /* restore size and flags */
> + list_for_each_entry(fail_res, &fail_head, list) {
> + struct resource *res = fail_res->res;
> +
> + res->start = fail_res->start;
> + res->end = fail_res->end;
> + res->flags = fail_res->flags;
> + }
> +
> + /* Revert to the old configuration */
> + if (!list_empty(&fail_head)) {
> + struct resource *res = &bridge->resource[i];
> +
> + res->start = saved.start;
> + res->end = saved.end;
> + res->flags = saved.flags;
> +
> + pci_claim_resource(bridge, i);
> + ret = -ENOSPC;
> + }
> +
> + free_list(&fail_head);
> + return ret;
> +}
> +
> void pci_assign_unassigned_bus_resources(struct pci_bus *bus)
> {
> struct pci_dev *dev;
> diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c
> index 9526e34..3bb1e29 100644
> --- a/drivers/pci/setup-res.c
> +++ b/drivers/pci/setup-res.c
> @@ -363,6 +363,57 @@ int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsiz
> return 0;
> }
>
> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
> +{
> + struct resource *res = dev->resource + resno;
> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
> + int old = pci_rbar_get_current_size(dev, resno);
> + u64 bytes = 1ULL << (size + 20);
> + int ret = 0;
I think we should fail the request if the device is enabled, i.e., if
the PCI_COMMAND_MEMORY bit is set. We can't safely change the BAR
while memory decoding is enabled.
I know there's code in pci_std_update_resource() that turns off
PCI_COMMAND_MEMORY, but I think that's a mistake: I think it should
fail when asked to update an enabled BAR the same way
pci_iov_update_resource() does.
I'm not sure why you call pci_reenable_device() below, but I think I
would rather have the driver do something like this:
pci_disable_device(dev);
pci_resize_resource(dev, 0, size);
pci_enable_device(dev);
That way it's very clear to the driver that it must re-read all BARs
after resizing this one.
> + if (!sizes)
> + return -ENOTSUPP;
> +
> + if (!(sizes & (1 << size)))
> + return -EINVAL;
> +
> + if (old < 0)
> + return old;
> +
> + /* Make sure the resource isn't assigned before making it larger. */
> + if (resource_size(res) < bytes && res->parent) {
> + release_resource(res);
> + res->end = resource_size(res) - 1;
> + res->start = 0;
> + if (resno < PCI_BRIDGE_RESOURCES)
> + pci_update_resource(dev, resno);
Why do we need to write this zero to the BAR? If PCI_COMMAND_MEMORY
is set, I think this is dangerous, and if it's not set, I think it's
unnecessary.
I think we should set the IORESOURCE_UNSET bit to indicate that the
resource does not have an address assigned to it. It should get
cleared later anyway, but having IORESOURCE_UNSET will make any debug
messages emitted while reassigning resources make more sense.
> + }
> +
> + ret = pci_rbar_set_size(dev, resno, size);
> + if (ret)
> + goto error_reassign;
> +
> + res->end = res->start + bytes - 1;
> +
> + ret = pci_reassign_bridge_resources(dev->bus->self, res->flags);
> + if (ret)
> + goto error_resize;
> +
> + pci_reenable_device(dev->bus->self);
> + return 0;
> +
> +error_resize:
> + pci_rbar_set_size(dev, resno, old);
> + res->end = res->start + (1ULL << (old + 20)) - 1;
> +
> +error_reassign:
> + pci_assign_unassigned_bus_resources(dev->bus);
> + pci_setup_bridge(dev->bus);
Could this bridge-related recovery code go inside
pci_reassign_bridge_resources()?
> + pci_reenable_device(dev->bus->self);
> + return ret;
> +}
> +EXPORT_SYMBOL(pci_resize_resource);
> +
> int pci_enable_resources(struct pci_dev *dev, int mask)
> {
> u16 cmd, old_cmd;
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 6954e50..8eaebb4 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1055,6 +1055,7 @@ void pci_reset_bridge_secondary_bus(struct pci_dev *dev);
> void pci_update_resource(struct pci_dev *dev, int resno);
> int __must_check pci_assign_resource(struct pci_dev *dev, int i);
> int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align);
> +int __must_check pci_resize_resource(struct pci_dev *dev, int i, int size);
> int pci_select_bars(struct pci_dev *dev, unsigned long flags);
> bool pci_device_is_present(struct pci_dev *pdev);
> void pci_ignore_hotplug(struct pci_dev *dev);
> @@ -1135,6 +1136,7 @@ void pci_assign_unassigned_resources(void);
> void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge);
> void pci_assign_unassigned_bus_resources(struct pci_bus *bus);
> void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus);
> +int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type);
> void pdev_enable_device(struct pci_dev *);
> int pci_enable_resources(struct pci_dev *, int mask);
> void pci_fixup_irqs(u8 (*)(struct pci_dev *, u8 *),
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access
2017-03-13 12:41 ` Christian König
` (2 preceding siblings ...)
(?)
@ 2017-03-24 21:42 ` Bjorn Helgaas
-1 siblings, 0 replies; 71+ messages in thread
From: Bjorn Helgaas @ 2017-03-24 21:42 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Mon, Mar 13, 2017 at 01:41:36PM +0100, Christian König wrote:
> From: Christian König <christian.koenig@amd.com>
>
> Try to resize BAR0 to let CPU access all of VRAM.
>
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 +
> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 +++++++++++++++++++++++++++++
> drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 8 +++++---
> drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 8 +++++---
> 4 files changed, 40 insertions(+), 6 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 3b81ded..905ded9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1719,6 +1719,7 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
> struct ttm_mem_reg *mem);
> void amdgpu_vram_location(struct amdgpu_device *adev, struct amdgpu_mc *mc, u64 base);
> void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc);
> +void amdgpu_resize_bar0(struct amdgpu_device *adev);
> void amdgpu_ttm_set_active_vram_size(struct amdgpu_device *adev, u64 size);
> int amdgpu_ttm_init(struct amdgpu_device *adev);
> void amdgpu_ttm_fini(struct amdgpu_device *adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 118f4e6..92955fe 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -692,6 +692,35 @@ void amdgpu_gtt_location(struct amdgpu_device *adev, struct amdgpu_mc *mc)
> mc->gtt_size >> 20, mc->gtt_start, mc->gtt_end);
> }
>
> +/**
> + * amdgpu_resize_bar0 - try to resize BAR0
> + *
> + * @adev: amdgpu_device pointer
> + *
> + * Try to resize BAR0 to make all VRAM CPU accessible.
> + */
> +void amdgpu_resize_bar0(struct amdgpu_device *adev)
> +{
> + u32 size = max(ilog2(adev->mc.real_vram_size - 1) + 1, 20) - 20;
> + int r;
> +
> + r = pci_resize_resource(adev->pdev, 0, size);
> +
> + if (r == -ENOTSUPP) {
> + /* The hardware don't support the extension. */
> + return;
> +
> + } else if (r == -ENOSPC) {
> + DRM_INFO("Not enoigh PCI address space for a large BAR.");
s/enoigh/enough/
> + } else if (r) {
> + DRM_ERROR("Problem resizing BAR0 (%d).", r);
> + }
> +
> + /* Reinit the doorbell mapping, it is most likely moved as well */
I think you should assume all BARs moved (I don't know how many you have;
maybe this already covers all of them).
> + amdgpu_doorbell_fini(adev);
> + BUG_ON(amdgpu_doorbell_init(adev));
I think things inside BUG_ON() tend to get overlooked, so I avoid things
that have side-effects. But that's just my personal preference.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
@ 2017-04-11 9:14 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-04-11 9:14 UTC (permalink / raw)
To: Andy Shevchenko
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
Sorry for the delayed response, have been busy with other stuff recently.
Am 13.03.2017 um 17:43 schrieb Andy Shevchenko:
>> v2: rebase on changes in rbar support
> This kind of comments usually goes after --- delimiter below.
That would remove it from the commit message which I don't want.
>> + unsigned i;
>> + int ret = 0;
>> +
>> + /* Release all children from the matching bridge resource */
>> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
>> + struct resource *res = &bridge->resource[i];
>> +
>
>> + if ((res->flags & type_mask) != (type & type_mask))
> IIUC it would be
> if ((res->flags ^ type) & type_mask)
>
> (I consider 'diff' as XOR operation is more understandable, but it's up to you)
I think like it is is easier to read.
>> + res->start = saved.start;
>> + res->end = saved.end;
>> + res->flags = saved.flags;
> Would
> *res = saved;
> work?
No, res also contains a bunch of pointers into the tree which we should
not override.
>> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
>> +{
>> + struct resource *res = dev->resource + resno;
>> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
>> + int old = pci_rbar_get_current_size(dev, resno);
>> + u64 bytes = 1ULL << (size + 20);
>> + int ret = 0;
>> +
> I would put
> sizes = pci_rbar_get_possible_sizes(dev, resno);
> here
Good idea, done.
>> + if (!sizes)
>> + return -ENOTSUPP;
>> +
>> + if (!(sizes & (1 << size)))
> BIT(size) ?
Done.
> and
> old = pci_rbar_get_current_size(dev, resno);
> here
Good idea as well.
>> +error_resize:
>> + pci_rbar_set_size(dev, resno, old);
>> + res->end = res->start + (1ULL << (old + 20)) - 1;
> BIT_ULL(old + 20) ?
Nope, that is actually a size in bytes. Not a bitfield. So while BIT_ULL
yields the right result it would be harder to understand.
Christian.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
@ 2017-04-11 9:14 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-04-11 9:14 UTC (permalink / raw)
To: Andy Shevchenko
Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Platform Driver,
helgaas-DgEjT+Ai2ygdnm+yROfE0A,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
Sorry for the delayed response, have been busy with other stuff recently.
Am 13.03.2017 um 17:43 schrieb Andy Shevchenko:
>> v2: rebase on changes in rbar support
> This kind of comments usually goes after --- delimiter below.
That would remove it from the commit message which I don't want.
>> + unsigned i;
>> + int ret = 0;
>> +
>> + /* Release all children from the matching bridge resource */
>> + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; ++i) {
>> + struct resource *res = &bridge->resource[i];
>> +
>
>> + if ((res->flags & type_mask) != (type & type_mask))
> IIUC it would be
> if ((res->flags ^ type) & type_mask)
>
> (I consider 'diff' as XOR operation is more understandable, but it's up to you)
I think like it is is easier to read.
>> + res->start = saved.start;
>> + res->end = saved.end;
>> + res->flags = saved.flags;
> Would
> *res = saved;
> work?
No, res also contains a bunch of pointers into the tree which we should
not override.
>> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
>> +{
>> + struct resource *res = dev->resource + resno;
>> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
>> + int old = pci_rbar_get_current_size(dev, resno);
>> + u64 bytes = 1ULL << (size + 20);
>> + int ret = 0;
>> +
> I would put
> sizes = pci_rbar_get_possible_sizes(dev, resno);
> here
Good idea, done.
>> + if (!sizes)
>> + return -ENOTSUPP;
>> +
>> + if (!(sizes & (1 << size)))
> BIT(size) ?
Done.
> and
> old = pci_rbar_get_current_size(dev, resno);
> here
Good idea as well.
>> +error_resize:
>> + pci_rbar_set_size(dev, resno, old);
>> + res->end = res->start + (1ULL << (old + 20)) - 1;
> BIT_ULL(old + 20) ?
Nope, that is actually a size in bytes. Not a bitfield. So while BIT_ULL
yields the right result it would be harder to understand.
Christian.
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-04-11 9:21 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-04-11 9:21 UTC (permalink / raw)
To: Andy Shevchenko
Cc: helgaas, linux-pci, dri-devel, Platform Driver, amd-gfx, linux-kernel
Am 13.03.2017 um 17:49 schrieb Andy Shevchenko:
> On Mon, Mar 13, 2017 at 2:41 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>
>> Most BIOS don't enable this because of compatibility reasons.
>>
>> Manually enable a 64bit BAR of 64GB size so that we have
>> enough room for PCI devices.
>> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
>> +{
>> + const uint64_t size = 64ULL * 1024 * 1024 * 1024;
> Perhaps extend <linux/sizes.h> and use SZ_64G here?
>
> It would be nice to do, since some of the drivers already are using
> sizes like 4GB and alike.
Actually using 64GB here was just for testing and to get some initial
feedback.
I think we want to use all the remaining address space for PCIe, but for
this we would need a new function in the resource management I think.
Going to take a deeper look when I'm sure we actually want this.
>> + if (i == 8)
>> + return;
>> +
>> + res = kzalloc(sizeof(*res), GFP_KERNEL);
>> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
>> + IORESOURCE_WINDOW;
>> + res->name = dev->bus->name;
>> + r = allocate_resource(&iomem_resource, res, size, 0x100000000,
>> + 0xfd00000000, size, NULL, NULL);
>> + if (r) {
>> + kfree(res);
>> + return;
>> + }
>> +
>> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
>> + limit = ((res->end + 1) >> 8) & 0xffffff00;
>> + high = ((res->start >> 40) & 0xff) |
>> + ((((res->end + 1) >> 40) & 0xff) << 16);
> Perhaps some of constants can be replaced by defines (I think some of
> them are already defined in ioport.h or somewhere else).
Yeah, good idea. But that stuff is purely AMD CPU specific, so won't
belong into ioport.h or similar common code.
Does anybody have any idea where I could put this?
Regards,
Christian.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-04-11 9:21 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-04-11 9:21 UTC (permalink / raw)
To: Andy Shevchenko
Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, Platform Driver,
helgaas-DgEjT+Ai2ygdnm+yROfE0A,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
Am 13.03.2017 um 17:49 schrieb Andy Shevchenko:
> On Mon, Mar 13, 2017 at 2:41 PM, Christian König
> <deathsimple@vodafone.de> wrote:
>
>> Most BIOS don't enable this because of compatibility reasons.
>>
>> Manually enable a 64bit BAR of 64GB size so that we have
>> enough room for PCI devices.
>> +static void pci_amd_enable_64bit_bar(struct pci_dev *dev)
>> +{
>> + const uint64_t size = 64ULL * 1024 * 1024 * 1024;
> Perhaps extend <linux/sizes.h> and use SZ_64G here?
>
> It would be nice to do, since some of the drivers already are using
> sizes like 4GB and alike.
Actually using 64GB here was just for testing and to get some initial
feedback.
I think we want to use all the remaining address space for PCIe, but for
this we would need a new function in the resource management I think.
Going to take a deeper look when I'm sure we actually want this.
>> + if (i == 8)
>> + return;
>> +
>> + res = kzalloc(sizeof(*res), GFP_KERNEL);
>> + res->flags = IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_MEM_64 |
>> + IORESOURCE_WINDOW;
>> + res->name = dev->bus->name;
>> + r = allocate_resource(&iomem_resource, res, size, 0x100000000,
>> + 0xfd00000000, size, NULL, NULL);
>> + if (r) {
>> + kfree(res);
>> + return;
>> + }
>> +
>> + base = ((res->start >> 8) & 0xffffff00) | 0x3;
>> + limit = ((res->end + 1) >> 8) & 0xffffff00;
>> + high = ((res->start >> 40) & 0xff) |
>> + ((((res->end + 1) >> 40) & 0xff) << 16);
> Perhaps some of constants can be replaced by defines (I think some of
> them are already defined in ioport.h or somewhere else).
Yeah, good idea. But that stuff is purely AMD CPU specific, so won't
belong into ioport.h or similar common code.
Does anybody have any idea where I could put this?
Regards,
Christian.
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
2017-03-24 21:34 ` Bjorn Helgaas
@ 2017-04-11 15:37 ` Christian König
2017-04-12 16:37 ` Bjorn Helgaas
0 siblings, 1 reply; 71+ messages in thread
From: Christian König @ 2017-04-11 15:37 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: linux-pci, linux-kernel, amd-gfx, dri-devel, platform-driver-x86
Hi Bjorn,
first of all sorry for the delay, had been busy with other stuff in the
last few weeks.
Am 24.03.2017 um 22:34 schrieb Bjorn Helgaas:
>> + release_child_resources(res);
> Doesn't this recursively release *all* child resources? There could
> be BARs from several devices, or even windows of downstream bridges,
> inside this window. The drivers of those other devices aren't
> expecting things to change here.
Yes, the original idea was that the driver calling this knows that the
BARs will be changed for the bridge it belongs to.
But you're right. I've changed it in the way that our device driver will
release all resource first and then call the function to resize its BAR.
The function will return an error in the next version of the patch if
the bridge BAR which needs to be moved for this is still used by child
resources.
>> +int pci_resize_resource(struct pci_dev *dev, int resno, int size)
>> +{
>> + struct resource *res = dev->resource + resno;
>> + u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
>> + int old = pci_rbar_get_current_size(dev, resno);
>> + u64 bytes = 1ULL << (size + 20);
>> + int ret = 0;
> I think we should fail the request if the device is enabled, i.e., if
> the PCI_COMMAND_MEMORY bit is set. We can't safely change the BAR
> while memory decoding is enabled.
>
> I know there's code in pci_std_update_resource() that turns off
> PCI_COMMAND_MEMORY, but I think that's a mistake: I think it should
> fail when asked to update an enabled BAR the same way
> pci_iov_update_resource() does.
>
> I'm not sure why you call pci_reenable_device() below, but I think I
> would rather have the driver do something like this:
>
> pci_disable_device(dev);
> pci_resize_resource(dev, 0, size);
> pci_enable_device(dev);
>
> That way it's very clear to the driver that it must re-read all BARs
> after resizing this one.
I've tried it, but this actually doesn't seem to work.
At least on the board I've tried pci_disable_device() doesn't clear the
PCI_COMMAND_MEMORY bit, it just clears the master bit.
Additional to that the power management reference counting
pci_disable_device() and pci_enable_device() doesn't look like what I
want for this functionality.
How about the driver needs to disabled memory decoding itself before
trying to resize anything?
>> + if (!sizes)
>> + return -ENOTSUPP;
>> +
>> + if (!(sizes & (1 << size)))
>> + return -EINVAL;
>> +
>> + if (old < 0)
>> + return old;
>> +
>> + /* Make sure the resource isn't assigned before making it larger. */
>> + if (resource_size(res) < bytes && res->parent) {
>> + release_resource(res);
>> + res->end = resource_size(res) - 1;
>> + res->start = 0;
>> + if (resno < PCI_BRIDGE_RESOURCES)
>> + pci_update_resource(dev, resno);
> Why do we need to write this zero to the BAR? If PCI_COMMAND_MEMORY
> is set, I think this is dangerous, and if it's not set, I think it's
> unnecessary.
>
> I think we should set the IORESOURCE_UNSET bit to indicate that the
> resource does not have an address assigned to it. It should get
> cleared later anyway, but having IORESOURCE_UNSET will make any debug
> messages emitted while reassigning resources make more sense.
Makes sense, changed.
>> + return 0;
>> +
>> +error_resize:
>> + pci_rbar_set_size(dev, resno, old);
>> + res->end = res->start + (1ULL << (old + 20)) - 1;
>> +
>> +error_reassign:
>> + pci_assign_unassigned_bus_resources(dev->bus);
>> + pci_setup_bridge(dev->bus);
> Could this bridge-related recovery code go inside
> pci_reassign_bridge_resources()?
No, we need to restore the original size of the resource before trying
the recovery code when something goes wrong.
I will address all other comments in the next version of the patch as well.
Regards,
Christian.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-04-11 15:48 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-04-11 15:48 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
Am 24.03.2017 um 16:47 schrieb Bjorn Helgaas:
> On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote:
>> From: Christian König <christian.koenig@amd.com>
>>
>> Most BIOS don't enable this because of compatibility reasons.
> Can you give any more details here? Without more hints, it's hard to
> know whether any of the compatibility reasons might apply to Linux as
> well.
Unfortunately not, I could try to ask a few more people at AMD if they
know the background.
I was told that there are a few boards which offers that as a BIOS
option, but so far I haven't found any (and I have quite a few here).
My best guess is that older windows versions have a problem with that.
>> Manually enable a 64bit BAR of 64GB size so that we have
>> enough room for PCI devices.
> From the context, I'm guessing this is enabling a new 64GB window
> through the PCI host bridge?
Yes, exactly. Sorry for the confusion.
> That might be documented as a "BAR", but
> it's not anything the Linux PCI core would recognize as a BAR.
At least the AMD NB documentation calls this the host BARs. But I'm
perfectly fine with any terminology.
How about calling it host bridge window instead?
> I think the specs would envision this being done via an ACPI _SRS
> method on the PNP0A03 host bridge device. That would be a more
> generic path that would work on any host bridge. Did you explore that
> possibility? I would prefer to avoid adding device-specific code if
> that's possible.
I've checked quite a few boards, but none of them actually implements it
this way.
M$ is working on a new ACPI table to enable this vendor neutral, but I
guess that will still take a while.
I want to support this for all AMD CPU released in the past 5 years or
so, so we are going to deal with a bunch of older boards as well.
>> + pci_bus_add_resource(dev->bus, res, 0);
> We would need some sort of printk here to explain how this new window
> magically appeared.
Good point, consider this done.
But is this actually the right place of doing it? Or would you prefer
something to be added to the probing code?
I think those fixups are applied a bit later, aren't they?
Best regards,
Christian.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
@ 2017-04-11 15:48 ` Christian König
0 siblings, 0 replies; 71+ messages in thread
From: Christian König @ 2017-04-11 15:48 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: linux-pci-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
platform-driver-x86-u79uwXL29TY76Z2rM5mHXA
Am 24.03.2017 um 16:47 schrieb Bjorn Helgaas:
> On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote:
>> From: Christian König <christian.koenig@amd.com>
>>
>> Most BIOS don't enable this because of compatibility reasons.
> Can you give any more details here? Without more hints, it's hard to
> know whether any of the compatibility reasons might apply to Linux as
> well.
Unfortunately not, I could try to ask a few more people at AMD if they
know the background.
I was told that there are a few boards which offers that as a BIOS
option, but so far I haven't found any (and I have quite a few here).
My best guess is that older windows versions have a problem with that.
>> Manually enable a 64bit BAR of 64GB size so that we have
>> enough room for PCI devices.
> From the context, I'm guessing this is enabling a new 64GB window
> through the PCI host bridge?
Yes, exactly. Sorry for the confusion.
> That might be documented as a "BAR", but
> it's not anything the Linux PCI core would recognize as a BAR.
At least the AMD NB documentation calls this the host BARs. But I'm
perfectly fine with any terminology.
How about calling it host bridge window instead?
> I think the specs would envision this being done via an ACPI _SRS
> method on the PNP0A03 host bridge device. That would be a more
> generic path that would work on any host bridge. Did you explore that
> possibility? I would prefer to avoid adding device-specific code if
> that's possible.
I've checked quite a few boards, but none of them actually implements it
this way.
M$ is working on a new ACPI table to enable this vendor neutral, but I
guess that will still take a while.
I want to support this for all AMD CPU released in the past 5 years or
so, so we are going to deal with a bunch of older boards as well.
>> + pci_bus_add_resource(dev->bus, res, 0);
> We would need some sort of printk here to explain how this new window
> magically appeared.
Good point, consider this done.
But is this actually the right place of doing it? Or would you prefer
something to be added to the probing code?
I think those fixups are applied a bit later, aren't they?
Best regards,
Christian.
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 2/4] PCI: add functionality for resizing resources v2
2017-04-11 15:37 ` Christian König
@ 2017-04-12 16:37 ` Bjorn Helgaas
0 siblings, 0 replies; 71+ messages in thread
From: Bjorn Helgaas @ 2017-04-12 16:37 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, linux-kernel, amd-gfx, dri-devel, platform-driver-x86
On Tue, Apr 11, 2017 at 05:37:04PM +0200, Christian König wrote:
> >>+int pci_resize_resource(struct pci_dev *dev, int resno, int size)
> >>+{
> >>+ struct resource *res = dev->resource + resno;
> >>+ u32 sizes = pci_rbar_get_possible_sizes(dev, resno);
> >>+ int old = pci_rbar_get_current_size(dev, resno);
> >>+ u64 bytes = 1ULL << (size + 20);
> >>+ int ret = 0;
> >I think we should fail the request if the device is enabled, i.e., if
> >the PCI_COMMAND_MEMORY bit is set. We can't safely change the BAR
> >while memory decoding is enabled.
> >
> >I know there's code in pci_std_update_resource() that turns off
> >PCI_COMMAND_MEMORY, but I think that's a mistake: I think it should
> >fail when asked to update an enabled BAR the same way
> >pci_iov_update_resource() does.
> >
> >I'm not sure why you call pci_reenable_device() below, but I think I
> >would rather have the driver do something like this:
> >
> > pci_disable_device(dev);
> > pci_resize_resource(dev, 0, size);
> > pci_enable_device(dev);
> >
> >That way it's very clear to the driver that it must re-read all BARs
> >after resizing this one.
>
> I've tried it, but this actually doesn't seem to work.
>
> At least on the board I've tried pci_disable_device() doesn't clear
> the PCI_COMMAND_MEMORY bit, it just clears the master bit.
Yeah, you're right. We generally turn *on* PCI_COMMAND_MEMORY in the
pci_enable_device() path, but the pci_disable_device() path doesn't
turn it off.
> Additional to that the power management reference counting
> pci_disable_device() and pci_enable_device() doesn't look like what
> I want for this functionality.
>
> How about the driver needs to disabled memory decoding itself before
> trying to resize anything?
There are a few places that do that, so maybe that's the best option.
Bjorn
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
2017-04-11 15:48 ` Christian König
(?)
@ 2017-04-12 16:55 ` Bjorn Helgaas
2017-04-25 13:01 ` Christian König
-1 siblings, 1 reply; 71+ messages in thread
From: Bjorn Helgaas @ 2017-04-12 16:55 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Tue, Apr 11, 2017 at 05:48:25PM +0200, Christian König wrote:
> Am 24.03.2017 um 16:47 schrieb Bjorn Helgaas:
> >On Mon, Mar 13, 2017 at 01:41:35PM +0100, Christian König wrote:
> >>From: Christian König <christian.koenig@amd.com>
> >>
> >>Most BIOS don't enable this because of compatibility reasons.
> >Can you give any more details here? Without more hints, it's hard to
> >know whether any of the compatibility reasons might apply to Linux as
> >well.
>
> Unfortunately not, I could try to ask a few more people at AMD if
> they know the background.
>
> I was told that there are a few boards which offers that as a BIOS
> option, but so far I haven't found any (and I have quite a few
> here).
>
> My best guess is that older windows versions have a problem with that.
>
> >>Manually enable a 64bit BAR of 64GB size so that we have
> >>enough room for PCI devices.
> > From the context, I'm guessing this is enabling a new 64GB window
> >through the PCI host bridge?
>
> Yes, exactly. Sorry for the confusion.
>
> >That might be documented as a "BAR", but
> >it's not anything the Linux PCI core would recognize as a BAR.
>
> At least the AMD NB documentation calls this the host BARs. But I'm
> perfectly fine with any terminology.
>
> How about calling it host bridge window instead?
That works for me.
> >I think the specs would envision this being done via an ACPI _SRS
> >method on the PNP0A03 host bridge device. That would be a more
> >generic path that would work on any host bridge. Did you explore that
> >possibility? I would prefer to avoid adding device-specific code if
> >that's possible.
>
> I've checked quite a few boards, but none of them actually
> implements it this way.
>
> M$ is working on a new ACPI table to enable this vendor neutral, but
> I guess that will still take a while.
>
> I want to support this for all AMD CPU released in the past 5 years
> or so, so we are going to deal with a bunch of older boards as well.
I've never seen _SRS for host bridges either. I'm curious about what
sort of new table will be proposed. It seems like the existing ACPI
resource framework could manage it, but I certainly don't know all the
issues.
> >>+ pci_bus_add_resource(dev->bus, res, 0);
> >We would need some sort of printk here to explain how this new window
> >magically appeared.
>
> Good point, consider this done.
>
> But is this actually the right place of doing it? Or would you
> prefer something to be added to the probing code?
>
> I think those fixups are applied a bit later, aren't they?
Logically, this should be done before we enumerate the PCI devices
below the host bridge, so a PCI device fixup is not the ideal place
for it, but it might be the most practical.
I could imagine some sort of quirk like the ones in
drivers/pnp/quirks.c that could add the window to the host bridge _CRS
and program the bridge to open it. But the PCI host bridges aren't
handled through the path that applies those fixups, and it would be
messy to identify your bridges (you currently use PCI vendor/device
IDs, which are only available after enumerating the device). So this
doesn't seem like a viable strategy.
Bjorn
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
2017-04-12 16:55 ` Bjorn Helgaas
@ 2017-04-25 13:01 ` Christian König
2017-05-17 21:36 ` Bjorn Helgaas
0 siblings, 1 reply; 71+ messages in thread
From: Christian König @ 2017-04-25 13:01 UTC (permalink / raw)
To: Bjorn Helgaas
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
Am 12.04.2017 um 18:55 schrieb Bjorn Helgaas:
> [SNIP]
>>> I think the specs would envision this being done via an ACPI _SRS
>>> method on the PNP0A03 host bridge device. That would be a more
>>> generic path that would work on any host bridge. Did you explore that
>>> possibility? I would prefer to avoid adding device-specific code if
>>> that's possible.
>> I've checked quite a few boards, but none of them actually
>> implements it this way.
>>
>> M$ is working on a new ACPI table to enable this vendor neutral, but
>> I guess that will still take a while.
>>
>> I want to support this for all AMD CPU released in the past 5 years
>> or so, so we are going to deal with a bunch of older boards as well.
> I've never seen _SRS for host bridges either. I'm curious about what
> sort of new table will be proposed. It seems like the existing ACPI
> resource framework could manage it, but I certainly don't know all the
> issues.
No idea either since I'm not involved into that. My job is to get it
working on the existing hw generations and that alone is enough work :)
My best guess is that MS is going to either make _SRS on the host bridge
or a pre-configured 64bit window mandatory for the BIOS.
>>>> + pci_bus_add_resource(dev->bus, res, 0);
>>> We would need some sort of printk here to explain how this new window
>>> magically appeared.
>> Good point, consider this done.
>>
>> But is this actually the right place of doing it? Or would you
>> prefer something to be added to the probing code?
>>
>> I think those fixups are applied a bit later, aren't they?
> Logically, this should be done before we enumerate the PCI devices
> below the host bridge, so a PCI device fixup is not the ideal place
> for it, but it might be the most practical.
Since the modification must be done on a device connected to the root
bus I run into quite a chicken and egg problem if I try to do it before
the enumeration.
> I could imagine some sort of quirk like the ones in
> drivers/pnp/quirks.c that could add the window to the host bridge _CRS
> and program the bridge to open it. But the PCI host bridges aren't
> handled through the path that applies those fixups, and it would be
> messy to identify your bridges (you currently use PCI vendor/device
> IDs, which are only available after enumerating the device). So this
> doesn't seem like a viable strategy.
I've tried that, but gave up rather quickly. Looks like the current
approach indeed work find even with "pci=realloc", so I'm going to stick
with that.
Regards,
Christian.
^ permalink raw reply [flat|nested] 71+ messages in thread
* Re: [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors
2017-04-25 13:01 ` Christian König
@ 2017-05-17 21:36 ` Bjorn Helgaas
0 siblings, 0 replies; 71+ messages in thread
From: Bjorn Helgaas @ 2017-05-17 21:36 UTC (permalink / raw)
To: Christian König
Cc: linux-pci, dri-devel, platform-driver-x86, amd-gfx, linux-kernel
On Tue, Apr 25, 2017 at 03:01:35PM +0200, Christian König wrote:
> Am 12.04.2017 um 18:55 schrieb Bjorn Helgaas:
> >[SNIP]
> >>>I think the specs would envision this being done via an ACPI _SRS
> >>>method on the PNP0A03 host bridge device. That would be a more
> >>>generic path that would work on any host bridge. Did you explore that
> >>>possibility? I would prefer to avoid adding device-specific code if
> >>>that's possible.
> >>I've checked quite a few boards, but none of them actually
> >>implements it this way.
> >>
> >>M$ is working on a new ACPI table to enable this vendor neutral, but
> >>I guess that will still take a while.
> >>
> >>I want to support this for all AMD CPU released in the past 5 years
> >>or so, so we are going to deal with a bunch of older boards as well.
> >I've never seen _SRS for host bridges either. I'm curious about what
> >sort of new table will be proposed. It seems like the existing ACPI
> >resource framework could manage it, but I certainly don't know all the
> >issues.
>
> No idea either since I'm not involved into that. My job is to get it
> working on the existing hw generations and that alone is enough work
> :)
>
> My best guess is that MS is going to either make _SRS on the host
> bridge or a pre-configured 64bit window mandatory for the BIOS.
While researching something else, I noticed that the PCI Firmware Spec, rev
3.2, does indeed call out _PRS and _SRS as the mechanism for the OS to
configure host bridge windows. See sections 4.1.3, 4.3.2.1, and 4.6.5.
Sec 4.6.5 also includes an implementation note that might be a clue about
the "compatibility issues" that prevent the BIOS from enabling the window
in the first place.
I'd like to incorporate some of this info into these changes, probably in a
code comment and changelog, so we can encourage a more generic approach in
the future, even if we can't use it in all existing cases.
Bjorn
^ permalink raw reply [flat|nested] 71+ messages in thread
end of thread, other threads:[~2017-05-17 21:37 UTC | newest]
Thread overview: 71+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-13 12:41 Resizeable PCI BAR support V3 Christian König
2017-03-13 12:41 ` Christian König
2017-03-13 12:41 ` [PATCH 1/4] PCI: add resizeable BAR infrastructure v3 Christian König
2017-03-13 12:41 ` Christian König
2017-03-14 13:09 ` kbuild test robot
2017-03-14 13:09 ` kbuild test robot
2017-03-24 15:28 ` Bjorn Helgaas
2017-03-13 12:41 ` [PATCH 2/4] PCI: add functionality for resizing resources v2 Christian König
2017-03-13 12:41 ` Christian König
2017-03-13 16:43 ` Andy Shevchenko
2017-03-13 16:43 ` Andy Shevchenko
2017-03-13 16:43 ` Andy Shevchenko
2017-04-11 9:14 ` Christian König
2017-04-11 9:14 ` Christian König
2017-03-14 9:01 ` kbuild test robot
2017-03-24 21:34 ` Bjorn Helgaas
2017-04-11 15:37 ` Christian König
2017-04-12 16:37 ` Bjorn Helgaas
2017-03-13 12:41 ` [PATCH 3/4] x86/PCI: Enable a 64bit BAR on AMD Family 15h (Models 30h-3fh) Processors Christian König
2017-03-13 12:41 ` Christian König
2017-03-13 16:49 ` Andy Shevchenko
2017-03-13 16:49 ` Andy Shevchenko
2017-03-13 16:49 ` Andy Shevchenko
2017-04-11 9:21 ` Christian König
2017-04-11 9:21 ` Christian König
2017-03-14 9:25 ` kbuild test robot
2017-03-14 9:25 ` kbuild test robot
2017-03-24 15:47 ` Bjorn Helgaas
2017-03-24 15:47 ` Bjorn Helgaas
2017-04-11 15:48 ` Christian König
2017-04-11 15:48 ` Christian König
2017-04-12 16:55 ` Bjorn Helgaas
2017-04-25 13:01 ` Christian König
2017-05-17 21:36 ` Bjorn Helgaas
2017-03-13 12:41 ` [PATCH 4/4] drm/amdgpu: resize VRAM BAR for CPU access Christian König
2017-03-13 12:41 ` Christian König
2017-03-13 16:51 ` Andy Shevchenko
2017-03-13 16:51 ` Andy Shevchenko
2017-03-15 7:23 ` Ayyappa Ch
2017-03-15 7:23 ` Ayyappa Ch
2017-03-15 7:23 ` Ayyappa Ch
2017-03-15 7:37 ` Christian König
2017-03-15 7:37 ` Christian König
2017-03-15 8:25 ` Zhou, David(ChunMing)
2017-03-15 8:25 ` Zhou, David(ChunMing)
2017-03-15 8:25 ` Zhou, David(ChunMing)
2017-03-15 9:29 ` Christian König
2017-03-15 9:29 ` Christian König
2017-03-15 9:29 ` Christian König
2017-03-16 2:19 ` Zhang, Jerry
2017-03-16 2:19 ` Zhang, Jerry
2017-03-16 2:19 ` Zhang, Jerry
2017-03-16 2:25 ` Alex Deucher
2017-03-16 2:25 ` Alex Deucher
2017-03-16 2:25 ` Alex Deucher
2017-03-16 2:41 ` Zhang, Jerry
2017-03-16 2:41 ` Zhang, Jerry
2017-03-23 14:30 ` Sagalovitch, Serguei
2017-03-23 14:30 ` Sagalovitch, Serguei
2017-03-23 14:30 ` Sagalovitch, Serguei
2017-03-23 15:56 ` Christian König
2017-03-23 15:56 ` Christian König
2017-03-15 10:42 ` Ayyappa Ch
2017-03-15 10:42 ` Ayyappa Ch
2017-03-15 10:42 ` Ayyappa Ch
2017-03-15 11:03 ` Christian König
2017-03-15 11:03 ` Christian König
2017-03-15 16:08 ` Deucher, Alexander
2017-03-15 16:08 ` Deucher, Alexander
2017-03-15 16:08 ` Deucher, Alexander
2017-03-24 21:42 ` Bjorn Helgaas
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.