* [PATCH v6 0/3] OF/PCI address PCI inbound memory limitations @ 2017-05-16 5:22 ` Oza Pawandeep 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep, Oza Pawandeep It is possible that PCI device supports 64-bit DMA addressing, and thus it's driver sets device's dma_mask to DMA_BIT_MASK(64), however PCI host bridge may have limitations on the inbound transaction addressing. This is particularly problematic on ARM/ARM64 SOCs where the IOMMU (i.e. SMMU) translates IOVA to PA for in-bound transactions only after PCI Host has forwarded these transactions on SOC IO bus. This means, on such ARM/ARM64 SOCs the IOVA of in-bound transactions has to honor the addressing restrictions of the PCI Host. Current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices but no implementation exists for pci devices. For e.g. iproc based SOCs and other SOCs (such as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; This patchset reserves the IOVA ranges for PCI masters based on the PCI world dma-ranges. fix of_dma_get_range to cater to PCI dma-ranges. fix of_dma_get_range which currently returns size 0 for PCI devices. IOVA allocation patch: [PATCH 2/3] iommu/pci: reserve iova for PCI masters Fix of_dma_get_range bug and address PCI master. [PATCH 3/3] PCI/of fix of_dma_get_range; get PCI specific this patch fixes the following problems of_dma_get_range. 1) return of wrong size as 0. 2) not handling absence of dma-ranges, which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also retuns the largest possible size based on dma-ranges, Base patch for both of the above patches: [PATCH 1/3] of/pci/dma: fix DMA configuration for PCI masters Changes since v6: - Robin's comments addressed. Changes since v5: Changes since v4: Changes since v3: Changes since v2: - minor changes, redudant checkes removed - removed internal review Changes since v1: - address Rob's comments. - Add a get_dma_ranges() function to of_bus struct.. - Convert existing contents of of_dma_get_range function to of_bus_default_dma_get_ranges and adding that to the default of_bus struct. - Make of_dma_get_range call of_bus_match() and then bus->get_dma_ranges. Oza Pawandeep (3): of/pci/dma: fix DMA configuration for PCI masters iommu/pci: reserve IOVA for PCI masters PCI/of fix of_dma_get_range; get PCI specific dma-ranges drivers/iommu/dma-iommu.c | 35 ++++++++ drivers/of/address.c | 215 ++++++++++++++++++++++++++++++++-------------- drivers/of/of_pci.c | 95 ++++++++++++++++++++ include/linux/of_pci.h | 7 ++ 4 files changed, 288 insertions(+), 64 deletions(-) -- 1.9.1 ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 0/3] OF/PCI address PCI inbound memory limitations @ 2017-05-16 5:22 ` Oza Pawandeep 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: linux-arm-kernel It is possible that PCI device supports 64-bit DMA addressing, and thus it's driver sets device's dma_mask to DMA_BIT_MASK(64), however PCI host bridge may have limitations on the inbound transaction addressing. This is particularly problematic on ARM/ARM64 SOCs where the IOMMU (i.e. SMMU) translates IOVA to PA for in-bound transactions only after PCI Host has forwarded these transactions on SOC IO bus. This means, on such ARM/ARM64 SOCs the IOVA of in-bound transactions has to honor the addressing restrictions of the PCI Host. Current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices but no implementation exists for pci devices. For e.g. iproc based SOCs and other SOCs (such as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; This patchset reserves the IOVA ranges for PCI masters based on the PCI world dma-ranges. fix of_dma_get_range to cater to PCI dma-ranges. fix of_dma_get_range which currently returns size 0 for PCI devices. IOVA allocation patch: [PATCH 2/3] iommu/pci: reserve iova for PCI masters Fix of_dma_get_range bug and address PCI master. [PATCH 3/3] PCI/of fix of_dma_get_range; get PCI specific this patch fixes the following problems of_dma_get_range. 1) return of wrong size as 0. 2) not handling absence of dma-ranges, which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also retuns the largest possible size based on dma-ranges, Base patch for both of the above patches: [PATCH 1/3] of/pci/dma: fix DMA configuration for PCI masters Changes since v6: - Robin's comments addressed. Changes since v5: Changes since v4: Changes since v3: Changes since v2: - minor changes, redudant checkes removed - removed internal review Changes since v1: - address Rob's comments. - Add a get_dma_ranges() function to of_bus struct.. - Convert existing contents of of_dma_get_range function to of_bus_default_dma_get_ranges and adding that to the default of_bus struct. - Make of_dma_get_range call of_bus_match() and then bus->get_dma_ranges. Oza Pawandeep (3): of/pci/dma: fix DMA configuration for PCI masters iommu/pci: reserve IOVA for PCI masters PCI/of fix of_dma_get_range; get PCI specific dma-ranges drivers/iommu/dma-iommu.c | 35 ++++++++ drivers/of/address.c | 215 ++++++++++++++++++++++++++++++++-------------- drivers/of/of_pci.c | 95 ++++++++++++++++++++ include/linux/of_pci.h | 7 ++ 4 files changed, 288 insertions(+), 64 deletions(-) -- 1.9.1 ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep, Oza Pawandeep current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices. but no implementation exists for pci to take care of pcie based memory ranges. for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; this patch serves following: 1) exposes interface to the pci host driver for their inbound memory ranges 2) provide an interface to callers such as of_dma_get_ranges. so then the returned size get best possible (largest) dma_mask. because PCI RC drivers do not call APIs such as dma_set_coherent_mask() and hence rather it shows its addressing capabilities based on dma-ranges. for e.g. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; we should get dev->coherent_dma_mask=0x7fffffffff. 3) this patch handles multiple inbound windows and dma-ranges. it is left to the caller, how it wants to use them. the new function returns the resources in a standard and unform way 4) this way the callers of for e.g. of_dma_get_ranges does not need to change. Signed-off-by: Oza Pawandeep <oza.oza@broadcom.com> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c index 0ee42c3..4005ed3 100644 --- a/drivers/of/of_pci.c +++ b/drivers/of/of_pci.c @@ -283,6 +283,101 @@ int of_pci_get_host_bridge_resources(struct device_node *dev, return err; } EXPORT_SYMBOL_GPL(of_pci_get_host_bridge_resources); + +/** + * of_pci_get_dma_ranges - Parse PCI host bridge inbound resources from DT + * @np: device node of the host bridge having the dma-ranges property + * @resources: list where the range of resources will be added after DT parsing + * + * It is the caller's job to free the @resources list. + * + * This function will parse the "dma-ranges" property of a + * PCI host bridge device node and setup the resource mapping based + * on its content. + * + * It returns zero if the range parsing has been successful or a standard error + * value if it failed. + */ + +int of_pci_get_dma_ranges(struct device_node *dn, struct list_head *resources) +{ + struct device_node *node = of_node_get(dn); + int rlen; + int pna = of_n_addr_cells(node); + const int na = 3, ns = 2; + int np = pna + na + ns; + int ret = 0; + struct resource *res; + const u32 *dma_ranges; + struct of_pci_range range; + + if (!node) + return -EINVAL; + + while (1) { + dma_ranges = of_get_property(node, "dma-ranges", &rlen); + + /* Ignore empty ranges, they imply no translation required. */ + if (dma_ranges && rlen > 0) + break; + + /* no dma-ranges, they imply no translation required. */ + if (!dma_ranges) + break; + + node = of_get_next_parent(node); + + if (!node) + break; + } + + if (!dma_ranges) { + pr_debug("pcie device has no dma-ranges defined for node(%s)\n", + dn->full_name); + ret = -EINVAL; + goto out; + } + + while ((rlen -= np * 4) >= 0) { + range.pci_space = dma_ranges[0]; + range.pci_addr = of_read_number(dma_ranges + 1, ns); + range.cpu_addr = of_translate_dma_address(node, + dma_ranges + na); + range.size = of_read_number(dma_ranges + pna + na, ns); + + dma_ranges += np; + + /* + * If we failed translation or got a zero-sized region + * then skip this range. + */ + if (range.cpu_addr == OF_BAD_ADDR || range.size == 0) + continue; + + res = kzalloc(sizeof(struct resource), GFP_KERNEL); + if (!res) { + ret = -ENOMEM; + goto parse_failed; + } + + ret = of_pci_range_to_resource(&range, dn, res); + if (ret) { + kfree(res); + continue; + } + + pci_add_resource_offset(resources, res, + res->start - range.pci_addr); + } + return ret; + +parse_failed: + pci_free_resource_list(resources); +out: + of_node_put(node); + return ret; +} +EXPORT_SYMBOL_GPL(of_pci_get_dma_ranges); #endif /* CONFIG_OF_ADDRESS */ #ifdef CONFIG_PCI_MSI diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h index 0e0974e..617b90d 100644 --- a/include/linux/of_pci.h +++ b/include/linux/of_pci.h @@ -76,6 +76,7 @@ static inline void of_pci_check_probe_only(void) { } int of_pci_get_host_bridge_resources(struct device_node *dev, unsigned char busno, unsigned char bus_max, struct list_head *resources, resource_size_t *io_base); +int of_pci_get_dma_ranges(struct device_node *np, struct list_head *resources); #else static inline int of_pci_get_host_bridge_resources(struct device_node *dev, unsigned char busno, unsigned char bus_max, @@ -83,6 +84,12 @@ static inline int of_pci_get_host_bridge_resources(struct device_node *dev, { return -EINVAL; } + +static inline int of_pci_get_dma_ranges(struct device_node *np, + struct list_head *resources) +{ + return -EINVAL; +} #endif #if defined(CONFIG_OF) && defined(CONFIG_PCI_MSI) -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: linux-arm-kernel current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices. but no implementation exists for pci to take care of pcie based memory ranges. for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; this patch serves following: 1) exposes interface to the pci host driver for their inbound memory ranges 2) provide an interface to callers such as of_dma_get_ranges. so then the returned size get best possible (largest) dma_mask. because PCI RC drivers do not call APIs such as dma_set_coherent_mask() and hence rather it shows its addressing capabilities based on dma-ranges. for e.g. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; we should get dev->coherent_dma_mask=0x7fffffffff. 3) this patch handles multiple inbound windows and dma-ranges. it is left to the caller, how it wants to use them. the new function returns the resources in a standard and unform way 4) this way the callers of for e.g. of_dma_get_ranges does not need to change. Signed-off-by: Oza Pawandeep <oza.oza@broadcom.com> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c index 0ee42c3..4005ed3 100644 --- a/drivers/of/of_pci.c +++ b/drivers/of/of_pci.c @@ -283,6 +283,101 @@ int of_pci_get_host_bridge_resources(struct device_node *dev, return err; } EXPORT_SYMBOL_GPL(of_pci_get_host_bridge_resources); + +/** + * of_pci_get_dma_ranges - Parse PCI host bridge inbound resources from DT + * @np: device node of the host bridge having the dma-ranges property + * @resources: list where the range of resources will be added after DT parsing + * + * It is the caller's job to free the @resources list. + * + * This function will parse the "dma-ranges" property of a + * PCI host bridge device node and setup the resource mapping based + * on its content. + * + * It returns zero if the range parsing has been successful or a standard error + * value if it failed. + */ + +int of_pci_get_dma_ranges(struct device_node *dn, struct list_head *resources) +{ + struct device_node *node = of_node_get(dn); + int rlen; + int pna = of_n_addr_cells(node); + const int na = 3, ns = 2; + int np = pna + na + ns; + int ret = 0; + struct resource *res; + const u32 *dma_ranges; + struct of_pci_range range; + + if (!node) + return -EINVAL; + + while (1) { + dma_ranges = of_get_property(node, "dma-ranges", &rlen); + + /* Ignore empty ranges, they imply no translation required. */ + if (dma_ranges && rlen > 0) + break; + + /* no dma-ranges, they imply no translation required. */ + if (!dma_ranges) + break; + + node = of_get_next_parent(node); + + if (!node) + break; + } + + if (!dma_ranges) { + pr_debug("pcie device has no dma-ranges defined for node(%s)\n", + dn->full_name); + ret = -EINVAL; + goto out; + } + + while ((rlen -= np * 4) >= 0) { + range.pci_space = dma_ranges[0]; + range.pci_addr = of_read_number(dma_ranges + 1, ns); + range.cpu_addr = of_translate_dma_address(node, + dma_ranges + na); + range.size = of_read_number(dma_ranges + pna + na, ns); + + dma_ranges += np; + + /* + * If we failed translation or got a zero-sized region + * then skip this range. + */ + if (range.cpu_addr == OF_BAD_ADDR || range.size == 0) + continue; + + res = kzalloc(sizeof(struct resource), GFP_KERNEL); + if (!res) { + ret = -ENOMEM; + goto parse_failed; + } + + ret = of_pci_range_to_resource(&range, dn, res); + if (ret) { + kfree(res); + continue; + } + + pci_add_resource_offset(resources, res, + res->start - range.pci_addr); + } + return ret; + +parse_failed: + pci_free_resource_list(resources); +out: + of_node_put(node); + return ret; +} +EXPORT_SYMBOL_GPL(of_pci_get_dma_ranges); #endif /* CONFIG_OF_ADDRESS */ #ifdef CONFIG_PCI_MSI diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h index 0e0974e..617b90d 100644 --- a/include/linux/of_pci.h +++ b/include/linux/of_pci.h @@ -76,6 +76,7 @@ static inline void of_pci_check_probe_only(void) { } int of_pci_get_host_bridge_resources(struct device_node *dev, unsigned char busno, unsigned char bus_max, struct list_head *resources, resource_size_t *io_base); +int of_pci_get_dma_ranges(struct device_node *np, struct list_head *resources); #else static inline int of_pci_get_host_bridge_resources(struct device_node *dev, unsigned char busno, unsigned char bus_max, @@ -83,6 +84,12 @@ static inline int of_pci_get_host_bridge_resources(struct device_node *dev, { return -EINVAL; } + +static inline int of_pci_get_dma_ranges(struct device_node *np, + struct list_head *resources) +{ + return -EINVAL; +} #endif #if defined(CONFIG_OF) && defined(CONFIG_PCI_MSI) -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep via iommu @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices. but no implementation exists for pci to take care of pcie based memory ranges. for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; this patch serves following: 1) exposes interface to the pci host driver for their inbound memory ranges 2) provide an interface to callers such as of_dma_get_ranges. so then the returned size get best possible (largest) dma_mask. because PCI RC drivers do not call APIs such as dma_set_coherent_mask() and hence rather it shows its addressing capabilities based on dma-ranges. for e.g. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; we should get dev->coherent_dma_mask=0x7fffffffff. 3) this patch handles multiple inbound windows and dma-ranges. it is left to the caller, how it wants to use them. the new function returns the resources in a standard and unform way 4) this way the callers of for e.g. of_dma_get_ranges does not need to change. Signed-off-by: Oza Pawandeep <oza.oza-dY08KVG/lbpWk0Htik3J/w@public.gmane.org> diff --git a/drivers/of/of_pci.c b/drivers/of/of_pci.c index 0ee42c3..4005ed3 100644 --- a/drivers/of/of_pci.c +++ b/drivers/of/of_pci.c @@ -283,6 +283,101 @@ int of_pci_get_host_bridge_resources(struct device_node *dev, return err; } EXPORT_SYMBOL_GPL(of_pci_get_host_bridge_resources); + +/** + * of_pci_get_dma_ranges - Parse PCI host bridge inbound resources from DT + * @np: device node of the host bridge having the dma-ranges property + * @resources: list where the range of resources will be added after DT parsing + * + * It is the caller's job to free the @resources list. + * + * This function will parse the "dma-ranges" property of a + * PCI host bridge device node and setup the resource mapping based + * on its content. + * + * It returns zero if the range parsing has been successful or a standard error + * value if it failed. + */ + +int of_pci_get_dma_ranges(struct device_node *dn, struct list_head *resources) +{ + struct device_node *node = of_node_get(dn); + int rlen; + int pna = of_n_addr_cells(node); + const int na = 3, ns = 2; + int np = pna + na + ns; + int ret = 0; + struct resource *res; + const u32 *dma_ranges; + struct of_pci_range range; + + if (!node) + return -EINVAL; + + while (1) { + dma_ranges = of_get_property(node, "dma-ranges", &rlen); + + /* Ignore empty ranges, they imply no translation required. */ + if (dma_ranges && rlen > 0) + break; + + /* no dma-ranges, they imply no translation required. */ + if (!dma_ranges) + break; + + node = of_get_next_parent(node); + + if (!node) + break; + } + + if (!dma_ranges) { + pr_debug("pcie device has no dma-ranges defined for node(%s)\n", + dn->full_name); + ret = -EINVAL; + goto out; + } + + while ((rlen -= np * 4) >= 0) { + range.pci_space = dma_ranges[0]; + range.pci_addr = of_read_number(dma_ranges + 1, ns); + range.cpu_addr = of_translate_dma_address(node, + dma_ranges + na); + range.size = of_read_number(dma_ranges + pna + na, ns); + + dma_ranges += np; + + /* + * If we failed translation or got a zero-sized region + * then skip this range. + */ + if (range.cpu_addr == OF_BAD_ADDR || range.size == 0) + continue; + + res = kzalloc(sizeof(struct resource), GFP_KERNEL); + if (!res) { + ret = -ENOMEM; + goto parse_failed; + } + + ret = of_pci_range_to_resource(&range, dn, res); + if (ret) { + kfree(res); + continue; + } + + pci_add_resource_offset(resources, res, + res->start - range.pci_addr); + } + return ret; + +parse_failed: + pci_free_resource_list(resources); +out: + of_node_put(node); + return ret; +} +EXPORT_SYMBOL_GPL(of_pci_get_dma_ranges); #endif /* CONFIG_OF_ADDRESS */ #ifdef CONFIG_PCI_MSI diff --git a/include/linux/of_pci.h b/include/linux/of_pci.h index 0e0974e..617b90d 100644 --- a/include/linux/of_pci.h +++ b/include/linux/of_pci.h @@ -76,6 +76,7 @@ static inline void of_pci_check_probe_only(void) { } int of_pci_get_host_bridge_resources(struct device_node *dev, unsigned char busno, unsigned char bus_max, struct list_head *resources, resource_size_t *io_base); +int of_pci_get_dma_ranges(struct device_node *np, struct list_head *resources); #else static inline int of_pci_get_host_bridge_resources(struct device_node *dev, unsigned char busno, unsigned char bus_max, @@ -83,6 +84,12 @@ static inline int of_pci_get_host_bridge_resources(struct device_node *dev, { return -EINVAL; } + +static inline int of_pci_get_dma_ranges(struct device_node *np, + struct list_head *resources) +{ + return -EINVAL; +} #endif #if defined(CONFIG_OF) && defined(CONFIG_PCI_MSI) -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-17 17:10 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:10 UTC (permalink / raw) To: Oza Pawandeep Cc: Joerg Roedel, Robin Murphy, iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: > current device framework and OF framework integration assumes s/current/The current/ > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. s/pci/PCI/ (also other occurrences below) s/pcie/PCIe/ I don't see how PCIe is relevant here. The bridge might support PCIe, but I don't think anything here is actually specific to PCIe. If that's the case, I think it's confusing to mention PCIe. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. > > 3) this patch handles multiple inbound windows and dma-ranges. > it is left to the caller, how it wants to use them. > the new function returns the resources in a standard and unform way > > 4) this way the callers of for e.g. of_dma_get_ranges > does not need to change. Please start sentences with a capital letter. ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-17 17:10 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:10 UTC (permalink / raw) To: linux-arm-kernel On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: > current device framework and OF framework integration assumes s/current/The current/ > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. s/pci/PCI/ (also other occurrences below) s/pcie/PCIe/ I don't see how PCIe is relevant here. The bridge might support PCIe, but I don't think anything here is actually specific to PCIe. If that's the case, I think it's confusing to mention PCIe. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. > > 3) this patch handles multiple inbound windows and dma-ranges. > it is left to the caller, how it wants to use them. > the new function returns the resources in a standard and unform way > > 4) this way the callers of for e.g. of_dma_get_ranges > does not need to change. Please start sentences with a capital letter. ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-17 17:10 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:10 UTC (permalink / raw) To: Oza Pawandeep Cc: devicetree, Oza Pawandeep, linux-pci, Joerg Roedel, linux-kernel, iommu, bcm-kernel-feedback-list, Robin Murphy, linux-arm-kernel On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: > current device framework and OF framework integration assumes s/current/The current/ > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. s/pci/PCI/ (also other occurrences below) s/pcie/PCIe/ I don't see how PCIe is relevant here. The bridge might support PCIe, but I don't think anything here is actually specific to PCIe. If that's the case, I think it's confusing to mention PCIe. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. > > 3) this patch handles multiple inbound windows and dma-ranges. > it is left to the caller, how it wants to use them. > the new function returns the resources in a standard and unform way > > 4) this way the callers of for e.g. of_dma_get_ranges > does not need to change. Please start sentences with a capital letter. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-17 17:10 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:10 UTC (permalink / raw) To: Oza Pawandeep Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: > current device framework and OF framework integration assumes s/current/The current/ > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. s/pci/PCI/ (also other occurrences below) s/pcie/PCIe/ I don't see how PCIe is relevant here. The bridge might support PCIe, but I don't think anything here is actually specific to PCIe. If that's the case, I think it's confusing to mention PCIe. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. > > 3) this patch handles multiple inbound windows and dma-ranges. > it is left to the caller, how it wants to use them. > the new function returns the resources in a standard and unform way > > 4) this way the callers of for e.g. of_dma_get_ranges > does not need to change. Please start sentences with a capital letter. ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-19 1:37 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-19 1:37 UTC (permalink / raw) To: Bjorn Helgaas Cc: Joerg Roedel, Robin Murphy, Linux IOMMU, linux-pci, linux-kernel, linux-arm-kernel, devicetree, BCM Kernel Feedback, Oza Pawandeep On Wed, May 17, 2017 at 10:40 PM, Bjorn Helgaas <helgaas@kernel.org> wrote: > On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: >> current device framework and OF framework integration assumes > > s/current/The current/ > >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > s/pci/PCI/ (also other occurrences below) > s/pcie/PCIe/ > > I don't see how PCIe is relevant here. The bridge might support PCIe, > but I don't think anything here is actually specific to PCIe. If > that's the case, I think it's confusing to mention PCIe. It attempts to fix of_dma_get_range for PCI master, because it currently it is returning *size as 0 (to the caller of_dma_configure) resulting into largest dma_mask which would be 64-bit mask on armv8. which usually has worked so far, because any other SOC's PCI RC, do not have the limitations as of Broadcom iproc based PCI RC. our RC will drop 64bit IOVAs, because it is not capable of addressing entire 64bit range. infact there are 2 real problems, please allow me to explain. please refer to my next mail in reply to Arnd Bergmann, > >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. >> >> 3) this patch handles multiple inbound windows and dma-ranges. >> it is left to the caller, how it wants to use them. >> the new function returns the resources in a standard and unform way >> >> 4) this way the callers of for e.g. of_dma_get_ranges >> does not need to change. > > Please start sentences with a capital letter. will take care of your comments. Thanks, Oza. ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-19 1:37 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-19 1:37 UTC (permalink / raw) To: linux-arm-kernel On Wed, May 17, 2017 at 10:40 PM, Bjorn Helgaas <helgaas@kernel.org> wrote: > On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: >> current device framework and OF framework integration assumes > > s/current/The current/ > >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > s/pci/PCI/ (also other occurrences below) > s/pcie/PCIe/ > > I don't see how PCIe is relevant here. The bridge might support PCIe, > but I don't think anything here is actually specific to PCIe. If > that's the case, I think it's confusing to mention PCIe. It attempts to fix of_dma_get_range for PCI master, because it currently it is returning *size as 0 (to the caller of_dma_configure) resulting into largest dma_mask which would be 64-bit mask on armv8. which usually has worked so far, because any other SOC's PCI RC, do not have the limitations as of Broadcom iproc based PCI RC. our RC will drop 64bit IOVAs, because it is not capable of addressing entire 64bit range. infact there are 2 real problems, please allow me to explain. please refer to my next mail in reply to Arnd Bergmann, > >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. >> >> 3) this patch handles multiple inbound windows and dma-ranges. >> it is left to the caller, how it wants to use them. >> the new function returns the resources in a standard and unform way >> >> 4) this way the callers of for e.g. of_dma_get_ranges >> does not need to change. > > Please start sentences with a capital letter. will take care of your comments. Thanks, Oza. ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-19 1:37 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza via iommu @ 2017-05-19 1:37 UTC (permalink / raw) To: Bjorn Helgaas Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Linux IOMMU, BCM Kernel Feedback, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r On Wed, May 17, 2017 at 10:40 PM, Bjorn Helgaas <helgaas-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote: > On Tue, May 16, 2017 at 10:52:05AM +0530, Oza Pawandeep wrote: >> current device framework and OF framework integration assumes > > s/current/The current/ > >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > s/pci/PCI/ (also other occurrences below) > s/pcie/PCIe/ > > I don't see how PCIe is relevant here. The bridge might support PCIe, > but I don't think anything here is actually specific to PCIe. If > that's the case, I think it's confusing to mention PCIe. It attempts to fix of_dma_get_range for PCI master, because it currently it is returning *size as 0 (to the caller of_dma_configure) resulting into largest dma_mask which would be 64-bit mask on armv8. which usually has worked so far, because any other SOC's PCI RC, do not have the limitations as of Broadcom iproc based PCI RC. our RC will drop 64bit IOVAs, because it is not capable of addressing entire 64bit range. infact there are 2 real problems, please allow me to explain. please refer to my next mail in reply to Arnd Bergmann, > >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. >> >> 3) this patch handles multiple inbound windows and dma-ranges. >> it is left to the caller, how it wants to use them. >> the new function returns the resources in a standard and unform way >> >> 4) this way the callers of for e.g. of_dma_get_ranges >> does not need to change. > > Please start sentences with a capital letter. will take care of your comments. Thanks, Oza. ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters 2017-05-16 5:22 ` Oza Pawandeep via iommu (?) @ 2017-05-17 19:13 ` Arnd Bergmann -1 siblings, 0 replies; 38+ messages in thread From: Arnd Bergmann @ 2017-05-17 19:13 UTC (permalink / raw) To: Oza Pawandeep Cc: Joerg Roedel, Robin Murphy, iommu, linux-pci, Linux Kernel Mailing List, Linux ARM, devicetree, bcm-kernel-feedback-list, Oza Pawandeep On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: > current device framework and OF framework integration assumes > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. Hi Oza, I'm trying to make sense of this, but am still rather puzzled. I have no idea what the distinction between memory-mapped devices and pcie based devices is in your description, as PCIe is usually memory mapped, and Linux doesn't actually support other kinds of PCIe devices on most architectures. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. do you mean the coherent_dma_mask of the PCI host bridge or an attached device here? If you require PCI devices to come up with an initial coherent_dma_mask other than 0xffffffffff, there are other problems involved. In particular, you will need to use swiotlb, which is not supported on arm32 at the moment, and the dma_set_mask()/dma_set_coherent_mask() functions need to be modified. > + while (1) { > + dma_ranges = of_get_property(node, "dma-ranges", &rlen); > + > + /* Ignore empty ranges, they imply no translation required. */ > + if (dma_ranges && rlen > 0) > + break; > + > + /* no dma-ranges, they imply no translation required. */ > + if (!dma_ranges) > + break; A missing parent dma-ranges property here should really indicate that there is no valid translation. If we have existing cases where this happens in DT files, we may treat it as allowing only 32-bit DMA (as we already do for having no dma-ranges at all), but treating it the same way as an empty dma-ranges property sounds wrong. Arnd ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-17 19:13 ` Arnd Bergmann 0 siblings, 0 replies; 38+ messages in thread From: Arnd Bergmann @ 2017-05-17 19:13 UTC (permalink / raw) To: linux-arm-kernel On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: > current device framework and OF framework integration assumes > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. Hi Oza, I'm trying to make sense of this, but am still rather puzzled. I have no idea what the distinction between memory-mapped devices and pcie based devices is in your description, as PCIe is usually memory mapped, and Linux doesn't actually support other kinds of PCIe devices on most architectures. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. do you mean the coherent_dma_mask of the PCI host bridge or an attached device here? If you require PCI devices to come up with an initial coherent_dma_mask other than 0xffffffffff, there are other problems involved. In particular, you will need to use swiotlb, which is not supported on arm32 at the moment, and the dma_set_mask()/dma_set_coherent_mask() functions need to be modified. > + while (1) { > + dma_ranges = of_get_property(node, "dma-ranges", &rlen); > + > + /* Ignore empty ranges, they imply no translation required. */ > + if (dma_ranges && rlen > 0) > + break; > + > + /* no dma-ranges, they imply no translation required. */ > + if (!dma_ranges) > + break; A missing parent dma-ranges property here should really indicate that there is no valid translation. If we have existing cases where this happens in DT files, we may treat it as allowing only 32-bit DMA (as we already do for having no dma-ranges at all), but treating it the same way as an empty dma-ranges property sounds wrong. Arnd ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-17 19:13 ` Arnd Bergmann 0 siblings, 0 replies; 38+ messages in thread From: Arnd Bergmann @ 2017-05-17 19:13 UTC (permalink / raw) To: Oza Pawandeep Cc: devicetree, Oza Pawandeep, linux-pci, Joerg Roedel, Linux Kernel Mailing List, iommu, bcm-kernel-feedback-list, Robin Murphy, Linux ARM On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: > current device framework and OF framework integration assumes > dma-ranges in a way where memory-mapped devices define their > dma-ranges. (child-bus-address, parent-bus-address, length). > > of_dma_configure is specifically written to take care of memory > mapped devices. but no implementation exists for pci to take > care of pcie based memory ranges. Hi Oza, I'm trying to make sense of this, but am still rather puzzled. I have no idea what the distinction between memory-mapped devices and pcie based devices is in your description, as PCIe is usually memory mapped, and Linux doesn't actually support other kinds of PCIe devices on most architectures. > for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI > world dma-ranges. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > > this patch serves following: > > 1) exposes interface to the pci host driver for their > inbound memory ranges > > 2) provide an interface to callers such as of_dma_get_ranges. > so then the returned size get best possible (largest) dma_mask. > because PCI RC drivers do not call APIs such as > dma_set_coherent_mask() and hence rather it shows its addressing > capabilities based on dma-ranges. > > for e.g. > dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; > we should get dev->coherent_dma_mask=0x7fffffffff. do you mean the coherent_dma_mask of the PCI host bridge or an attached device here? If you require PCI devices to come up with an initial coherent_dma_mask other than 0xffffffffff, there are other problems involved. In particular, you will need to use swiotlb, which is not supported on arm32 at the moment, and the dma_set_mask()/dma_set_coherent_mask() functions need to be modified. > + while (1) { > + dma_ranges = of_get_property(node, "dma-ranges", &rlen); > + > + /* Ignore empty ranges, they imply no translation required. */ > + if (dma_ranges && rlen > 0) > + break; > + > + /* no dma-ranges, they imply no translation required. */ > + if (!dma_ranges) > + break; A missing parent dma-ranges property here should really indicate that there is no valid translation. If we have existing cases where this happens in DT files, we may treat it as allowing only 32-bit DMA (as we already do for having no dma-ranges at all), but treating it the same way as an empty dma-ranges property sounds wrong. Arnd _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-19 2:58 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-19 2:58 UTC (permalink / raw) To: Arnd Bergmann Cc: Joerg Roedel, Robin Murphy, Linux IOMMU, linux-pci, Linux Kernel Mailing List, Linux ARM, devicetree, bcm-kernel-feedback-list, Oza Pawandeep On Thu, May 18, 2017 at 12:43 AM, Arnd Bergmann <arnd@arndb.de> wrote: > On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: >> current device framework and OF framework integration assumes >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > Hi Oza, > > I'm trying to make sense of this, but am still rather puzzled. I have > no idea what the distinction between memory-mapped devices and > pcie based devices is in your description, as PCIe is usually memory > mapped, and Linux doesn't actually support other kinds of PCIe > devices on most architectures. > there are 2 problems which I am trying to address here. problem-1: let me explain our PCI RC's limitations first. IOVA allocaiton honours device's coherent_dma_mask/dma_mask. in PCI case, current code honours DMA mask set by EP, there is no concept of PCI host bridge dma-mask, which should be there and could truely reflect the limitaiton of PCI host bridge. having said that we have dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; which means we can only address 512GB. now because of broken of_dma_get_range we end up getting 64bit dma_mask. please check the code:of_dma_configure() if (ret < 0) { dma_addr = offset = 0; size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1); now in this process I figred out problems in of_dma_get_range: hence the fix 1) return of wrong size as 0 becasue of whole parsing problem. 2) not handling absence of dma-ranges which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also returns the largest possible size based on dma-ranges, please have a look at [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges I just made is bus specific leaving origional of_dma_get_range unmodified and defining new PCI handling of_bus_pci_get_dma_ranges also when I say memory-mapped and PCI device, I only mean to say with respect to the dma-ranges format. (of coure PCI is memory mapped as well). probbaly my commit description is misleading, sorry about that. so Problem1: is just bug fix, Nothing else Problem2: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters we have memory banks <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ when I run SPDK (User sapce) which internally uses vfio to access PCI endpoint directly. vfio uses huge-pages which could coming from 640G/0x000000a0. and the way vfio maps the hugepage to user space and generates IOVA is different from the way kernel allocate IOVA. vfio just maps one to one IOVA to Physical address. it just calls directly remap_pfn_range. so the way kernel allocates IOVA (where it honours device dma_mask) and the way userspace gets IOVA is totally different. so dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; will not work. instead we have to go for scatterred dma-ranges leaving holes. having said that we have to reserve IOVA allocations for inbound memory. I am in a process of addressing Robin Murphy's comment on that and rebasing my patch on rc12. this problem statement is more important to us. because it makes both kernel and use space IOVA allocations work when IOMMU is enabled. probably thing might be confusing because I am clubbing my patches to address both the problems. going forward I should just try to first send out patch for problem2 alone (not sure) because my next patch-set would bring some changes in pci/probe.c as well. >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. > > do you mean the coherent_dma_mask of the PCI host bridge > or an attached device here? > > If you require PCI devices to come up with an initial > coherent_dma_mask other than 0xffffffffff, there are other > problems involved. In particular, you will need to use > swiotlb, which is not supported on arm32 at the moment, > and the dma_set_mask()/dma_set_coherent_mask() > functions need to be modified. even without this patch also it comes up with coherent_dma_mask of 64bits. since it considers dma_mask set by Endpoint. please check [RFC PATCH 2/3] iommu/dma: account pci host bridge dma_mask for IOVA allocation this patch was in-fact inspired by Robin Murphy's earlier discussions. > >> + while (1) { >> + dma_ranges = of_get_property(node, "dma-ranges", &rlen); >> + >> + /* Ignore empty ranges, they imply no translation required. */ >> + if (dma_ranges && rlen > 0) >> + break; >> + >> + /* no dma-ranges, they imply no translation required. */ >> + if (!dma_ranges) >> + break; > > A missing parent dma-ranges property here should really indicate that there > is no valid translation. If we have existing cases where this happens > in DT files, we may treat it as allowing only 32-bit DMA (as we already > do for having no dma-ranges at all), but treating it the same way > as an empty dma-ranges property sounds wrong. not sure if I understood you: but we have dma-ranges property optional for one of our SOC, in the sense... PCI RC will allow all the incoming transactions because RC does not translate anything. what mask should it generate if dma-ranges property is not present ? how do you suggest to handle ? > > Arnd ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-19 2:58 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-19 2:58 UTC (permalink / raw) To: linux-arm-kernel On Thu, May 18, 2017 at 12:43 AM, Arnd Bergmann <arnd@arndb.de> wrote: > On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: >> current device framework and OF framework integration assumes >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > Hi Oza, > > I'm trying to make sense of this, but am still rather puzzled. I have > no idea what the distinction between memory-mapped devices and > pcie based devices is in your description, as PCIe is usually memory > mapped, and Linux doesn't actually support other kinds of PCIe > devices on most architectures. > there are 2 problems which I am trying to address here. problem-1: let me explain our PCI RC's limitations first. IOVA allocaiton honours device's coherent_dma_mask/dma_mask. in PCI case, current code honours DMA mask set by EP, there is no concept of PCI host bridge dma-mask, which should be there and could truely reflect the limitaiton of PCI host bridge. having said that we have dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; which means we can only address 512GB. now because of broken of_dma_get_range we end up getting 64bit dma_mask. please check the code:of_dma_configure() if (ret < 0) { dma_addr = offset = 0; size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1); now in this process I figred out problems in of_dma_get_range: hence the fix 1) return of wrong size as 0 becasue of whole parsing problem. 2) not handling absence of dma-ranges which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also returns the largest possible size based on dma-ranges, please have a look at [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges I just made is bus specific leaving origional of_dma_get_range unmodified and defining new PCI handling of_bus_pci_get_dma_ranges also when I say memory-mapped and PCI device, I only mean to say with respect to the dma-ranges format. (of coure PCI is memory mapped as well). probbaly my commit description is misleading, sorry about that. so Problem1: is just bug fix, Nothing else Problem2: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters we have memory banks <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ when I run SPDK (User sapce) which internally uses vfio to access PCI endpoint directly. vfio uses huge-pages which could coming from 640G/0x000000a0. and the way vfio maps the hugepage to user space and generates IOVA is different from the way kernel allocate IOVA. vfio just maps one to one IOVA to Physical address. it just calls directly remap_pfn_range. so the way kernel allocates IOVA (where it honours device dma_mask) and the way userspace gets IOVA is totally different. so dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; will not work. instead we have to go for scatterred dma-ranges leaving holes. having said that we have to reserve IOVA allocations for inbound memory. I am in a process of addressing Robin Murphy's comment on that and rebasing my patch on rc12. this problem statement is more important to us. because it makes both kernel and use space IOVA allocations work when IOMMU is enabled. probably thing might be confusing because I am clubbing my patches to address both the problems. going forward I should just try to first send out patch for problem2 alone (not sure) because my next patch-set would bring some changes in pci/probe.c as well. >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. > > do you mean the coherent_dma_mask of the PCI host bridge > or an attached device here? > > If you require PCI devices to come up with an initial > coherent_dma_mask other than 0xffffffffff, there are other > problems involved. In particular, you will need to use > swiotlb, which is not supported on arm32 at the moment, > and the dma_set_mask()/dma_set_coherent_mask() > functions need to be modified. even without this patch also it comes up with coherent_dma_mask of 64bits. since it considers dma_mask set by Endpoint. please check [RFC PATCH 2/3] iommu/dma: account pci host bridge dma_mask for IOVA allocation this patch was in-fact inspired by Robin Murphy's earlier discussions. > >> + while (1) { >> + dma_ranges = of_get_property(node, "dma-ranges", &rlen); >> + >> + /* Ignore empty ranges, they imply no translation required. */ >> + if (dma_ranges && rlen > 0) >> + break; >> + >> + /* no dma-ranges, they imply no translation required. */ >> + if (!dma_ranges) >> + break; > > A missing parent dma-ranges property here should really indicate that there > is no valid translation. If we have existing cases where this happens > in DT files, we may treat it as allowing only 32-bit DMA (as we already > do for having no dma-ranges at all), but treating it the same way > as an empty dma-ranges property sounds wrong. not sure if I understood you: but we have dma-ranges property optional for one of our SOC, in the sense... PCI RC will allow all the incoming transactions because RC does not translate anything. what mask should it generate if dma-ranges property is not present ? how do you suggest to handle ? > > Arnd ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-19 2:58 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza via iommu @ 2017-05-19 2:58 UTC (permalink / raw) To: Arnd Bergmann Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci, Linux Kernel Mailing List, Linux IOMMU, bcm-kernel-feedback-list, Linux ARM On Thu, May 18, 2017 at 12:43 AM, Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote: > On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza-dY08KVG/lbpWk0Htik3J/w@public.gmane.org> wrote: >> current device framework and OF framework integration assumes >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > Hi Oza, > > I'm trying to make sense of this, but am still rather puzzled. I have > no idea what the distinction between memory-mapped devices and > pcie based devices is in your description, as PCIe is usually memory > mapped, and Linux doesn't actually support other kinds of PCIe > devices on most architectures. > there are 2 problems which I am trying to address here. problem-1: let me explain our PCI RC's limitations first. IOVA allocaiton honours device's coherent_dma_mask/dma_mask. in PCI case, current code honours DMA mask set by EP, there is no concept of PCI host bridge dma-mask, which should be there and could truely reflect the limitaiton of PCI host bridge. having said that we have dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; which means we can only address 512GB. now because of broken of_dma_get_range we end up getting 64bit dma_mask. please check the code:of_dma_configure() if (ret < 0) { dma_addr = offset = 0; size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1); now in this process I figred out problems in of_dma_get_range: hence the fix 1) return of wrong size as 0 becasue of whole parsing problem. 2) not handling absence of dma-ranges which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also returns the largest possible size based on dma-ranges, please have a look at [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges I just made is bus specific leaving origional of_dma_get_range unmodified and defining new PCI handling of_bus_pci_get_dma_ranges also when I say memory-mapped and PCI device, I only mean to say with respect to the dma-ranges format. (of coure PCI is memory mapped as well). probbaly my commit description is misleading, sorry about that. so Problem1: is just bug fix, Nothing else Problem2: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters we have memory banks <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ when I run SPDK (User sapce) which internally uses vfio to access PCI endpoint directly. vfio uses huge-pages which could coming from 640G/0x000000a0. and the way vfio maps the hugepage to user space and generates IOVA is different from the way kernel allocate IOVA. vfio just maps one to one IOVA to Physical address. it just calls directly remap_pfn_range. so the way kernel allocates IOVA (where it honours device dma_mask) and the way userspace gets IOVA is totally different. so dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; will not work. instead we have to go for scatterred dma-ranges leaving holes. having said that we have to reserve IOVA allocations for inbound memory. I am in a process of addressing Robin Murphy's comment on that and rebasing my patch on rc12. this problem statement is more important to us. because it makes both kernel and use space IOVA allocations work when IOMMU is enabled. probably thing might be confusing because I am clubbing my patches to address both the problems. going forward I should just try to first send out patch for problem2 alone (not sure) because my next patch-set would bring some changes in pci/probe.c as well. >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. > > do you mean the coherent_dma_mask of the PCI host bridge > or an attached device here? > > If you require PCI devices to come up with an initial > coherent_dma_mask other than 0xffffffffff, there are other > problems involved. In particular, you will need to use > swiotlb, which is not supported on arm32 at the moment, > and the dma_set_mask()/dma_set_coherent_mask() > functions need to be modified. even without this patch also it comes up with coherent_dma_mask of 64bits. since it considers dma_mask set by Endpoint. please check [RFC PATCH 2/3] iommu/dma: account pci host bridge dma_mask for IOVA allocation this patch was in-fact inspired by Robin Murphy's earlier discussions. > >> + while (1) { >> + dma_ranges = of_get_property(node, "dma-ranges", &rlen); >> + >> + /* Ignore empty ranges, they imply no translation required. */ >> + if (dma_ranges && rlen > 0) >> + break; >> + >> + /* no dma-ranges, they imply no translation required. */ >> + if (!dma_ranges) >> + break; > > A missing parent dma-ranges property here should really indicate that there > is no valid translation. If we have existing cases where this happens > in DT files, we may treat it as allowing only 32-bit DMA (as we already > do for having no dma-ranges at all), but treating it the same way > as an empty dma-ranges property sounds wrong. not sure if I understood you: but we have dma-ranges property optional for one of our SOC, in the sense... PCI RC will allow all the incoming transactions because RC does not translate anything. what mask should it generate if dma-ranges property is not present ? how do you suggest to handle ? > > Arnd ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-22 16:42 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-22 16:42 UTC (permalink / raw) To: Arnd Bergmann Cc: Joerg Roedel, Robin Murphy, Linux IOMMU, linux-pci, Linux Kernel Mailing List, Linux ARM, devicetree, bcm-kernel-feedback-list, Oza Pawandeep On Thu, May 18, 2017 at 12:43 AM, Arnd Bergmann <arnd@arndb.de> wrote: > On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: >> current device framework and OF framework integration assumes >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > Hi Oza, > > I'm trying to make sense of this, but am still rather puzzled. I have > no idea what the distinction between memory-mapped devices and > pcie based devices is in your description, as PCIe is usually memory > mapped, and Linux doesn't actually support other kinds of PCIe > devices on most architectures. > >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. > > do you mean the coherent_dma_mask of the PCI host bridge > or an attached device here? > > If you require PCI devices to come up with an initial > coherent_dma_mask other than 0xffffffffff, there are other > problems involved. In particular, you will need to use > swiotlb, which is not supported on arm32 at the moment, > and the dma_set_mask()/dma_set_coherent_mask() > functions need to be modified. > >> + while (1) { >> + dma_ranges = of_get_property(node, "dma-ranges", &rlen); >> + >> + /* Ignore empty ranges, they imply no translation required. */ >> + if (dma_ranges && rlen > 0) >> + break; >> + >> + /* no dma-ranges, they imply no translation required. */ >> + if (!dma_ranges) >> + break; > > A missing parent dma-ranges property here should really indicate that there > is no valid translation. If we have existing cases where this happens > in DT files, we may treat it as allowing only 32-bit DMA (as we already > do for having no dma-ranges at all), but treating it the same way > as an empty dma-ranges property sounds wrong. > > Arnd Hi Arnd and Bjorn, Can you please have a look at PATCH v7 ? It addresses problem2 alone. Regards, Oza ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-22 16:42 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-22 16:42 UTC (permalink / raw) To: linux-arm-kernel On Thu, May 18, 2017 at 12:43 AM, Arnd Bergmann <arnd@arndb.de> wrote: > On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza@broadcom.com> wrote: >> current device framework and OF framework integration assumes >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > Hi Oza, > > I'm trying to make sense of this, but am still rather puzzled. I have > no idea what the distinction between memory-mapped devices and > pcie based devices is in your description, as PCIe is usually memory > mapped, and Linux doesn't actually support other kinds of PCIe > devices on most architectures. > >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. > > do you mean the coherent_dma_mask of the PCI host bridge > or an attached device here? > > If you require PCI devices to come up with an initial > coherent_dma_mask other than 0xffffffffff, there are other > problems involved. In particular, you will need to use > swiotlb, which is not supported on arm32 at the moment, > and the dma_set_mask()/dma_set_coherent_mask() > functions need to be modified. > >> + while (1) { >> + dma_ranges = of_get_property(node, "dma-ranges", &rlen); >> + >> + /* Ignore empty ranges, they imply no translation required. */ >> + if (dma_ranges && rlen > 0) >> + break; >> + >> + /* no dma-ranges, they imply no translation required. */ >> + if (!dma_ranges) >> + break; > > A missing parent dma-ranges property here should really indicate that there > is no valid translation. If we have existing cases where this happens > in DT files, we may treat it as allowing only 32-bit DMA (as we already > do for having no dma-ranges at all), but treating it the same way > as an empty dma-ranges property sounds wrong. > > Arnd Hi Arnd and Bjorn, Can you please have a look at PATCH v7 ? It addresses problem2 alone. Regards, Oza ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters @ 2017-05-22 16:42 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza via iommu @ 2017-05-22 16:42 UTC (permalink / raw) To: Arnd Bergmann Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci, Linux Kernel Mailing List, Linux IOMMU, bcm-kernel-feedback-list, Linux ARM On Thu, May 18, 2017 at 12:43 AM, Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> wrote: > On Tue, May 16, 2017 at 7:22 AM, Oza Pawandeep <oza.oza-dY08KVG/lbpWk0Htik3J/w@public.gmane.org> wrote: >> current device framework and OF framework integration assumes >> dma-ranges in a way where memory-mapped devices define their >> dma-ranges. (child-bus-address, parent-bus-address, length). >> >> of_dma_configure is specifically written to take care of memory >> mapped devices. but no implementation exists for pci to take >> care of pcie based memory ranges. > > Hi Oza, > > I'm trying to make sense of this, but am still rather puzzled. I have > no idea what the distinction between memory-mapped devices and > pcie based devices is in your description, as PCIe is usually memory > mapped, and Linux doesn't actually support other kinds of PCIe > devices on most architectures. > >> for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI >> world dma-ranges. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> >> this patch serves following: >> >> 1) exposes interface to the pci host driver for their >> inbound memory ranges >> >> 2) provide an interface to callers such as of_dma_get_ranges. >> so then the returned size get best possible (largest) dma_mask. >> because PCI RC drivers do not call APIs such as >> dma_set_coherent_mask() and hence rather it shows its addressing >> capabilities based on dma-ranges. >> >> for e.g. >> dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; >> we should get dev->coherent_dma_mask=0x7fffffffff. > > do you mean the coherent_dma_mask of the PCI host bridge > or an attached device here? > > If you require PCI devices to come up with an initial > coherent_dma_mask other than 0xffffffffff, there are other > problems involved. In particular, you will need to use > swiotlb, which is not supported on arm32 at the moment, > and the dma_set_mask()/dma_set_coherent_mask() > functions need to be modified. > >> + while (1) { >> + dma_ranges = of_get_property(node, "dma-ranges", &rlen); >> + >> + /* Ignore empty ranges, they imply no translation required. */ >> + if (dma_ranges && rlen > 0) >> + break; >> + >> + /* no dma-ranges, they imply no translation required. */ >> + if (!dma_ranges) >> + break; > > A missing parent dma-ranges property here should really indicate that there > is no valid translation. If we have existing cases where this happens > in DT files, we may treat it as allowing only 32-bit DMA (as we already > do for having no dma-ranges at all), but treating it the same way > as an empty dma-ranges property sounds wrong. > > Arnd Hi Arnd and Bjorn, Can you please have a look at PATCH v7 ? It addresses problem2 alone. Regards, Oza ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep, Oza Pawandeep this patch reserves the IOVA for PCI masters. ARM64 based SOCs may have scattered memory banks. such as iproc based SOC has <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ but incoming PCI transcation addressing capability is limited by host bridge, for example if max incoming window capability is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. to address this problem, iommu has to avoid allocating IOVA which are reserved. which inturn does not allocate IOVA if it falls into hole. Signed-off-by: Oza Pawandeep <oza.oza@broadcom.com> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 48d36ce..08764b0 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -27,6 +27,7 @@ #include <linux/iova.h> #include <linux/irq.h> #include <linux/mm.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/scatterlist.h> #include <linux/vmalloc.h> @@ -171,8 +172,12 @@ static void iova_reserve_pci_windows(struct pci_dev *dev, struct iova_domain *iovad) { struct pci_host_bridge *bridge = pci_find_host_bridge(dev->bus); + struct device_node *np = bridge->dev.parent->of_node; struct resource_entry *window; unsigned long lo, hi; + int ret; + dma_addr_t tmp_dma_addr = 0, dma_addr; + LIST_HEAD(res); resource_list_for_each_entry(window, &bridge->windows) { if (resource_type(window->res) != IORESOURCE_MEM && @@ -183,6 +188,36 @@ static void iova_reserve_pci_windows(struct pci_dev *dev, hi = iova_pfn(iovad, window->res->end - window->offset); reserve_iova(iovad, lo, hi); } + + /* PCI inbound memory reservation. */ + ret = of_pci_get_dma_ranges(np, &res); + if (!ret) { + resource_list_for_each_entry(window, &res) { + struct resource *res_dma = window->res; + + dma_addr = res_dma->start - window->offset; + if (tmp_dma_addr > dma_addr) { + pr_warn("PCI: failed to reserve iovas; ranges should be sorted\n"); + return; + } + if (tmp_dma_addr != dma_addr) { + lo = iova_pfn(iovad, tmp_dma_addr); + hi = iova_pfn(iovad, dma_addr - 1); + reserve_iova(iovad, lo, hi); + } + tmp_dma_addr = window->res->end - window->offset; + } + /* + * the last dma-range should honour based on the + * 32/64-bit dma addresses. + */ + if (tmp_dma_addr < DMA_BIT_MASK(sizeof(dma_addr_t) * 8)) { + lo = iova_pfn(iovad, tmp_dma_addr); + hi = iova_pfn(iovad, + DMA_BIT_MASK(sizeof(dma_addr_t) * 8) - 1); + reserve_iova(iovad, lo, hi); + } + } } /** -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: linux-arm-kernel this patch reserves the IOVA for PCI masters. ARM64 based SOCs may have scattered memory banks. such as iproc based SOC has <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ but incoming PCI transcation addressing capability is limited by host bridge, for example if max incoming window capability is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. to address this problem, iommu has to avoid allocating IOVA which are reserved. which inturn does not allocate IOVA if it falls into hole. Signed-off-by: Oza Pawandeep <oza.oza@broadcom.com> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 48d36ce..08764b0 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -27,6 +27,7 @@ #include <linux/iova.h> #include <linux/irq.h> #include <linux/mm.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/scatterlist.h> #include <linux/vmalloc.h> @@ -171,8 +172,12 @@ static void iova_reserve_pci_windows(struct pci_dev *dev, struct iova_domain *iovad) { struct pci_host_bridge *bridge = pci_find_host_bridge(dev->bus); + struct device_node *np = bridge->dev.parent->of_node; struct resource_entry *window; unsigned long lo, hi; + int ret; + dma_addr_t tmp_dma_addr = 0, dma_addr; + LIST_HEAD(res); resource_list_for_each_entry(window, &bridge->windows) { if (resource_type(window->res) != IORESOURCE_MEM && @@ -183,6 +188,36 @@ static void iova_reserve_pci_windows(struct pci_dev *dev, hi = iova_pfn(iovad, window->res->end - window->offset); reserve_iova(iovad, lo, hi); } + + /* PCI inbound memory reservation. */ + ret = of_pci_get_dma_ranges(np, &res); + if (!ret) { + resource_list_for_each_entry(window, &res) { + struct resource *res_dma = window->res; + + dma_addr = res_dma->start - window->offset; + if (tmp_dma_addr > dma_addr) { + pr_warn("PCI: failed to reserve iovas; ranges should be sorted\n"); + return; + } + if (tmp_dma_addr != dma_addr) { + lo = iova_pfn(iovad, tmp_dma_addr); + hi = iova_pfn(iovad, dma_addr - 1); + reserve_iova(iovad, lo, hi); + } + tmp_dma_addr = window->res->end - window->offset; + } + /* + * the last dma-range should honour based on the + * 32/64-bit dma addresses. + */ + if (tmp_dma_addr < DMA_BIT_MASK(sizeof(dma_addr_t) * 8)) { + lo = iova_pfn(iovad, tmp_dma_addr); + hi = iova_pfn(iovad, + DMA_BIT_MASK(sizeof(dma_addr_t) * 8) - 1); + reserve_iova(iovad, lo, hi); + } + } } /** -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep via iommu @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r this patch reserves the IOVA for PCI masters. ARM64 based SOCs may have scattered memory banks. such as iproc based SOC has <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ but incoming PCI transcation addressing capability is limited by host bridge, for example if max incoming window capability is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. to address this problem, iommu has to avoid allocating IOVA which are reserved. which inturn does not allocate IOVA if it falls into hole. Signed-off-by: Oza Pawandeep <oza.oza-dY08KVG/lbpWk0Htik3J/w@public.gmane.org> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 48d36ce..08764b0 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -27,6 +27,7 @@ #include <linux/iova.h> #include <linux/irq.h> #include <linux/mm.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/scatterlist.h> #include <linux/vmalloc.h> @@ -171,8 +172,12 @@ static void iova_reserve_pci_windows(struct pci_dev *dev, struct iova_domain *iovad) { struct pci_host_bridge *bridge = pci_find_host_bridge(dev->bus); + struct device_node *np = bridge->dev.parent->of_node; struct resource_entry *window; unsigned long lo, hi; + int ret; + dma_addr_t tmp_dma_addr = 0, dma_addr; + LIST_HEAD(res); resource_list_for_each_entry(window, &bridge->windows) { if (resource_type(window->res) != IORESOURCE_MEM && @@ -183,6 +188,36 @@ static void iova_reserve_pci_windows(struct pci_dev *dev, hi = iova_pfn(iovad, window->res->end - window->offset); reserve_iova(iovad, lo, hi); } + + /* PCI inbound memory reservation. */ + ret = of_pci_get_dma_ranges(np, &res); + if (!ret) { + resource_list_for_each_entry(window, &res) { + struct resource *res_dma = window->res; + + dma_addr = res_dma->start - window->offset; + if (tmp_dma_addr > dma_addr) { + pr_warn("PCI: failed to reserve iovas; ranges should be sorted\n"); + return; + } + if (tmp_dma_addr != dma_addr) { + lo = iova_pfn(iovad, tmp_dma_addr); + hi = iova_pfn(iovad, dma_addr - 1); + reserve_iova(iovad, lo, hi); + } + tmp_dma_addr = window->res->end - window->offset; + } + /* + * the last dma-range should honour based on the + * 32/64-bit dma addresses. + */ + if (tmp_dma_addr < DMA_BIT_MASK(sizeof(dma_addr_t) * 8)) { + lo = iova_pfn(iovad, tmp_dma_addr); + hi = iova_pfn(iovad, + DMA_BIT_MASK(sizeof(dma_addr_t) * 8) - 1); + reserve_iova(iovad, lo, hi); + } + } } /** -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* Re: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-17 17:11 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:11 UTC (permalink / raw) To: Oza Pawandeep Cc: Joerg Roedel, Robin Murphy, iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: > this patch reserves the IOVA for PCI masters. > ARM64 based SOCs may have scattered memory banks. > such as iproc based SOC has > > <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ > <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ > <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ > <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ > > but incoming PCI transcation addressing capability is limited s/transcation/transaction/ > by host bridge, for example if max incoming window capability > is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. > > to address this problem, iommu has to avoid allocating IOVA which s/iommu/IOMMU/ > are reserved. which inturn does not allocate IOVA if it falls into hole. s/inturn/in turn/ ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-17 17:11 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:11 UTC (permalink / raw) To: linux-arm-kernel On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: > this patch reserves the IOVA for PCI masters. > ARM64 based SOCs may have scattered memory banks. > such as iproc based SOC has > > <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ > <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ > <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ > <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ > > but incoming PCI transcation addressing capability is limited s/transcation/transaction/ > by host bridge, for example if max incoming window capability > is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. > > to address this problem, iommu has to avoid allocating IOVA which s/iommu/IOMMU/ > are reserved. which inturn does not allocate IOVA if it falls into hole. s/inturn/in turn/ ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-17 17:11 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:11 UTC (permalink / raw) To: Oza Pawandeep Cc: devicetree, Oza Pawandeep, linux-pci, Joerg Roedel, linux-kernel, iommu, bcm-kernel-feedback-list, Robin Murphy, linux-arm-kernel On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: > this patch reserves the IOVA for PCI masters. > ARM64 based SOCs may have scattered memory banks. > such as iproc based SOC has > > <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ > <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ > <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ > <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ > > but incoming PCI transcation addressing capability is limited s/transcation/transaction/ > by host bridge, for example if max incoming window capability > is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. > > to address this problem, iommu has to avoid allocating IOVA which s/iommu/IOMMU/ > are reserved. which inturn does not allocate IOVA if it falls into hole. s/inturn/in turn/ _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-17 17:11 ` Bjorn Helgaas 0 siblings, 0 replies; 38+ messages in thread From: Bjorn Helgaas @ 2017-05-17 17:11 UTC (permalink / raw) To: Oza Pawandeep Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: > this patch reserves the IOVA for PCI masters. > ARM64 based SOCs may have scattered memory banks. > such as iproc based SOC has > > <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ > <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ > <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ > <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ > > but incoming PCI transcation addressing capability is limited s/transcation/transaction/ > by host bridge, for example if max incoming window capability > is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. > > to address this problem, iommu has to avoid allocating IOVA which s/iommu/IOMMU/ > are reserved. which inturn does not allocate IOVA if it falls into hole. s/inturn/in turn/ ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-19 1:21 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-19 1:21 UTC (permalink / raw) To: Bjorn Helgaas Cc: Joerg Roedel, Robin Murphy, Linux IOMMU, linux-pci, linux-kernel, linux-arm-kernel, devicetree, BCM Kernel Feedback, Oza Pawandeep On Wed, May 17, 2017 at 10:41 PM, Bjorn Helgaas <helgaas@kernel.org> wrote: > On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: >> this patch reserves the IOVA for PCI masters. >> ARM64 based SOCs may have scattered memory banks. >> such as iproc based SOC has >> >> <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ >> <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ >> <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ >> <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ >> >> but incoming PCI transcation addressing capability is limited > > s/transcation/transaction/ > >> by host bridge, for example if max incoming window capability >> is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. >> >> to address this problem, iommu has to avoid allocating IOVA which > > s/iommu/IOMMU/ > >> are reserved. which inturn does not allocate IOVA if it falls into hole. > > s/inturn/in turn/ Hi Bjorn, Thank you for the comments. Will take care of all your comments. Regards, Oza. ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-19 1:21 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza @ 2017-05-19 1:21 UTC (permalink / raw) To: linux-arm-kernel On Wed, May 17, 2017 at 10:41 PM, Bjorn Helgaas <helgaas@kernel.org> wrote: > On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: >> this patch reserves the IOVA for PCI masters. >> ARM64 based SOCs may have scattered memory banks. >> such as iproc based SOC has >> >> <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ >> <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ >> <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ >> <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ >> >> but incoming PCI transcation addressing capability is limited > > s/transcation/transaction/ > >> by host bridge, for example if max incoming window capability >> is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. >> >> to address this problem, iommu has to avoid allocating IOVA which > > s/iommu/IOMMU/ > >> are reserved. which inturn does not allocate IOVA if it falls into hole. > > s/inturn/in turn/ Hi Bjorn, Thank you for the comments. Will take care of all your comments. Regards, Oza. ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 2/3] iommu/pci: reserve IOVA for PCI masters @ 2017-05-19 1:21 ` Oza Oza via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Oza via iommu @ 2017-05-19 1:21 UTC (permalink / raw) To: Bjorn Helgaas Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Linux IOMMU, BCM Kernel Feedback, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r On Wed, May 17, 2017 at 10:41 PM, Bjorn Helgaas <helgaas-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote: > On Tue, May 16, 2017 at 10:52:06AM +0530, Oza Pawandeep wrote: >> this patch reserves the IOVA for PCI masters. >> ARM64 based SOCs may have scattered memory banks. >> such as iproc based SOC has >> >> <0x00000000 0x80000000 0x0 0x80000000>, /* 2G @ 2G */ >> <0x00000008 0x80000000 0x3 0x80000000>, /* 14G @ 34G */ >> <0x00000090 0x00000000 0x4 0x00000000>, /* 16G @ 576G */ >> <0x000000a0 0x00000000 0x4 0x00000000>; /* 16G @ 640G */ >> >> but incoming PCI transcation addressing capability is limited > > s/transcation/transaction/ > >> by host bridge, for example if max incoming window capability >> is 512 GB, then 0x00000090 and 0x000000a0 will fall beyond it. >> >> to address this problem, iommu has to avoid allocating IOVA which > > s/iommu/IOMMU/ > >> are reserved. which inturn does not allocate IOVA if it falls into hole. > > s/inturn/in turn/ Hi Bjorn, Thank you for the comments. Will take care of all your comments. Regards, Oza. ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep, Oza Pawandeep current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices. but no implementation exists for pci to take care of pcie based memory ranges. for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; this patch fixes the following problems of_dma_get_range. 1) return of wrong size as 0. 2) not handling absence of dma-ranges which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also retuns the largest possible size based on dma-ranges, for e.g. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; we should get dev->coherent_dma_mask=0x7fffffffff. based on which IOVA allocation space will honour PCI host bridge limitations. the implementation hooks bus specific callbacks for getting dma-ranges. Signed-off-by: Oza Pawandeep <oza.oza@broadcom.com> diff --git a/drivers/of/address.c b/drivers/of/address.c index 02b2903..800731c 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -6,6 +6,7 @@ #include <linux/ioport.h> #include <linux/module.h> #include <linux/of_address.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/pci_regs.h> #include <linux/sizes.h> @@ -46,6 +47,8 @@ struct of_bus { int na, int ns, int pna); int (*translate)(__be32 *addr, u64 offset, int na); unsigned int (*get_flags)(const __be32 *addr); + int (*get_dma_ranges)(struct device_node *np, + u64 *dma_addr, u64 *paddr, u64 *size); }; /* @@ -171,6 +174,144 @@ static int of_bus_pci_translate(__be32 *addr, u64 offset, int na) { return of_bus_default_translate(addr + 1, offset, na - 1); } + +static int of_bus_pci_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + struct device_node *node = of_node_get(np); + int ret = 0; + struct resource_entry *window; + LIST_HEAD(res); + + if (!node) + return -EINVAL; + + *size = 0; + /* + * PCI dma-ranges is not mandatory property. + * many devices do no need to have it, since + * host bridge does not require inbound memory + * configuration or rather have design limitations. + * so we look for dma-ranges, if missing we + * just return the caller full size, and also + * no dma-ranges suggests that, host bridge allows + * whatever comes in, so we set dma_addr to 0. + */ + ret = of_pci_get_dma_ranges(np, &res); + if (!ret) { + resource_list_for_each_entry(window, &res) { + struct resource *res_dma = window->res; + + if (*size < resource_size(res_dma)) { + *dma_addr = res_dma->start - window->offset; + *paddr = res_dma->start; + *size = resource_size(res_dma); + } + } + } + pci_free_resource_list(&res); + + /* + * return the largest possible size, + * since PCI master allows everything. + */ + if (*size == 0) { + pr_debug("empty/zero size dma-ranges found for node(%s)\n", + np->full_name); + *size = DMA_BIT_MASK(sizeof(dma_addr_t) * 8) - 1; + *dma_addr = *paddr = 0; + ret = 0; + } + + pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", + *dma_addr, *paddr, *size); + + of_node_put(node); + + return ret; +} + +static int get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + struct device_node *node = of_node_get(np); + const __be32 *ranges = NULL; + int len, naddr, nsize, pna; + int ret = 0; + u64 dmaaddr; + + if (!node) + return -EINVAL; + + while (1) { + naddr = of_n_addr_cells(node); + nsize = of_n_size_cells(node); + node = of_get_next_parent(node); + if (!node) + break; + + ranges = of_get_property(node, "dma-ranges", &len); + + /* Ignore empty ranges, they imply no translation required */ + if (ranges && len > 0) + break; + + /* + * At least empty ranges has to be defined for parent node if + * DMA is supported + */ + if (!ranges) + break; + } + + if (!ranges) { + pr_debug("no dma-ranges found for node(%s)\n", np->full_name); + ret = -ENODEV; + goto out; + } + + len /= sizeof(u32); + + pna = of_n_addr_cells(node); + + /* dma-ranges format: + * DMA addr : naddr cells + * CPU addr : pna cells + * size : nsize cells + */ + dmaaddr = of_read_number(ranges, naddr); + *paddr = of_translate_dma_address(np, ranges); + if (*paddr == OF_BAD_ADDR) { + pr_err("translation of DMA address(%pad) to CPU address failed node(%s)\n", + dma_addr, np->full_name); + ret = -EINVAL; + goto out; + } + *dma_addr = dmaaddr; + + *size = of_read_number(ranges + naddr + pna, nsize); + + pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", + *dma_addr, *paddr, *size); + +out: + of_node_put(node); + + return ret; +} + +static int of_bus_isa_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + return get_dma_ranges(np, dma_addr, paddr, size); +} + +static int of_bus_default_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + return get_dma_ranges(np, dma_addr, paddr, size); +} + #endif /* CONFIG_OF_ADDRESS_PCI */ #ifdef CONFIG_PCI @@ -424,6 +565,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_pci_map, .translate = of_bus_pci_translate, .get_flags = of_bus_pci_get_flags, + .get_dma_ranges = of_bus_pci_get_dma_ranges, }, #endif /* CONFIG_OF_ADDRESS_PCI */ /* ISA */ @@ -435,6 +577,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_isa_map, .translate = of_bus_isa_translate, .get_flags = of_bus_isa_get_flags, + .get_dma_ranges = of_bus_isa_get_dma_ranges, }, /* Default */ { @@ -445,6 +588,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_default_map, .translate = of_bus_default_translate, .get_flags = of_bus_default_get_flags, + .get_dma_ranges = of_bus_default_get_dma_ranges, }, }; @@ -820,74 +964,17 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index, * size : nsize cells * * It returns -ENODEV if "dma-ranges" property was not found - * for this device in DT. + * for this device in DT, except if PCI device then, dma-ranges + * can be optional property, and in that case returns size with + * entire host memory. */ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size) { - struct device_node *node = of_node_get(np); - const __be32 *ranges = NULL; - int len, naddr, nsize, pna; - int ret = 0; - u64 dmaaddr; - - if (!node) - return -EINVAL; - - while (1) { - naddr = of_n_addr_cells(node); - nsize = of_n_size_cells(node); - node = of_get_next_parent(node); - if (!node) - break; - - ranges = of_get_property(node, "dma-ranges", &len); - - /* Ignore empty ranges, they imply no translation required */ - if (ranges && len > 0) - break; - - /* - * At least empty ranges has to be defined for parent node if - * DMA is supported - */ - if (!ranges) - break; - } - - if (!ranges) { - pr_debug("no dma-ranges found for node(%s)\n", np->full_name); - ret = -ENODEV; - goto out; - } - - len /= sizeof(u32); - - pna = of_n_addr_cells(node); - - /* dma-ranges format: - * DMA addr : naddr cells - * CPU addr : pna cells - * size : nsize cells - */ - dmaaddr = of_read_number(ranges, naddr); - *paddr = of_translate_dma_address(np, ranges); - if (*paddr == OF_BAD_ADDR) { - pr_err("translation of DMA address(%pad) to CPU address failed node(%s)\n", - dma_addr, np->full_name); - ret = -EINVAL; - goto out; - } - *dma_addr = dmaaddr; - - *size = of_read_number(ranges + naddr + pna, nsize); - - pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", - *dma_addr, *paddr, *size); - -out: - of_node_put(node); + struct of_bus *bus; - return ret; + /* get bus specific dma-ranges. */ + bus = of_match_bus(np); + return bus->get_dma_ranges(np, dma_addr, paddr, size); } EXPORT_SYMBOL_GPL(of_dma_get_range); -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep @ 2017-05-16 5:22 UTC (permalink / raw) To: linux-arm-kernel current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices. but no implementation exists for pci to take care of pcie based memory ranges. for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; this patch fixes the following problems of_dma_get_range. 1) return of wrong size as 0. 2) not handling absence of dma-ranges which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also retuns the largest possible size based on dma-ranges, for e.g. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; we should get dev->coherent_dma_mask=0x7fffffffff. based on which IOVA allocation space will honour PCI host bridge limitations. the implementation hooks bus specific callbacks for getting dma-ranges. Signed-off-by: Oza Pawandeep <oza.oza@broadcom.com> diff --git a/drivers/of/address.c b/drivers/of/address.c index 02b2903..800731c 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -6,6 +6,7 @@ #include <linux/ioport.h> #include <linux/module.h> #include <linux/of_address.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/pci_regs.h> #include <linux/sizes.h> @@ -46,6 +47,8 @@ struct of_bus { int na, int ns, int pna); int (*translate)(__be32 *addr, u64 offset, int na); unsigned int (*get_flags)(const __be32 *addr); + int (*get_dma_ranges)(struct device_node *np, + u64 *dma_addr, u64 *paddr, u64 *size); }; /* @@ -171,6 +174,144 @@ static int of_bus_pci_translate(__be32 *addr, u64 offset, int na) { return of_bus_default_translate(addr + 1, offset, na - 1); } + +static int of_bus_pci_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + struct device_node *node = of_node_get(np); + int ret = 0; + struct resource_entry *window; + LIST_HEAD(res); + + if (!node) + return -EINVAL; + + *size = 0; + /* + * PCI dma-ranges is not mandatory property. + * many devices do no need to have it, since + * host bridge does not require inbound memory + * configuration or rather have design limitations. + * so we look for dma-ranges, if missing we + * just return the caller full size, and also + * no dma-ranges suggests that, host bridge allows + * whatever comes in, so we set dma_addr to 0. + */ + ret = of_pci_get_dma_ranges(np, &res); + if (!ret) { + resource_list_for_each_entry(window, &res) { + struct resource *res_dma = window->res; + + if (*size < resource_size(res_dma)) { + *dma_addr = res_dma->start - window->offset; + *paddr = res_dma->start; + *size = resource_size(res_dma); + } + } + } + pci_free_resource_list(&res); + + /* + * return the largest possible size, + * since PCI master allows everything. + */ + if (*size == 0) { + pr_debug("empty/zero size dma-ranges found for node(%s)\n", + np->full_name); + *size = DMA_BIT_MASK(sizeof(dma_addr_t) * 8) - 1; + *dma_addr = *paddr = 0; + ret = 0; + } + + pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", + *dma_addr, *paddr, *size); + + of_node_put(node); + + return ret; +} + +static int get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + struct device_node *node = of_node_get(np); + const __be32 *ranges = NULL; + int len, naddr, nsize, pna; + int ret = 0; + u64 dmaaddr; + + if (!node) + return -EINVAL; + + while (1) { + naddr = of_n_addr_cells(node); + nsize = of_n_size_cells(node); + node = of_get_next_parent(node); + if (!node) + break; + + ranges = of_get_property(node, "dma-ranges", &len); + + /* Ignore empty ranges, they imply no translation required */ + if (ranges && len > 0) + break; + + /* + * At least empty ranges has to be defined for parent node if + * DMA is supported + */ + if (!ranges) + break; + } + + if (!ranges) { + pr_debug("no dma-ranges found for node(%s)\n", np->full_name); + ret = -ENODEV; + goto out; + } + + len /= sizeof(u32); + + pna = of_n_addr_cells(node); + + /* dma-ranges format: + * DMA addr : naddr cells + * CPU addr : pna cells + * size : nsize cells + */ + dmaaddr = of_read_number(ranges, naddr); + *paddr = of_translate_dma_address(np, ranges); + if (*paddr == OF_BAD_ADDR) { + pr_err("translation of DMA address(%pad) to CPU address failed node(%s)\n", + dma_addr, np->full_name); + ret = -EINVAL; + goto out; + } + *dma_addr = dmaaddr; + + *size = of_read_number(ranges + naddr + pna, nsize); + + pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", + *dma_addr, *paddr, *size); + +out: + of_node_put(node); + + return ret; +} + +static int of_bus_isa_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + return get_dma_ranges(np, dma_addr, paddr, size); +} + +static int of_bus_default_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + return get_dma_ranges(np, dma_addr, paddr, size); +} + #endif /* CONFIG_OF_ADDRESS_PCI */ #ifdef CONFIG_PCI @@ -424,6 +565,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_pci_map, .translate = of_bus_pci_translate, .get_flags = of_bus_pci_get_flags, + .get_dma_ranges = of_bus_pci_get_dma_ranges, }, #endif /* CONFIG_OF_ADDRESS_PCI */ /* ISA */ @@ -435,6 +577,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_isa_map, .translate = of_bus_isa_translate, .get_flags = of_bus_isa_get_flags, + .get_dma_ranges = of_bus_isa_get_dma_ranges, }, /* Default */ { @@ -445,6 +588,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_default_map, .translate = of_bus_default_translate, .get_flags = of_bus_default_get_flags, + .get_dma_ranges = of_bus_default_get_dma_ranges, }, }; @@ -820,74 +964,17 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index, * size : nsize cells * * It returns -ENODEV if "dma-ranges" property was not found - * for this device in DT. + * for this device in DT, except if PCI device then, dma-ranges + * can be optional property, and in that case returns size with + * entire host memory. */ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size) { - struct device_node *node = of_node_get(np); - const __be32 *ranges = NULL; - int len, naddr, nsize, pna; - int ret = 0; - u64 dmaaddr; - - if (!node) - return -EINVAL; - - while (1) { - naddr = of_n_addr_cells(node); - nsize = of_n_size_cells(node); - node = of_get_next_parent(node); - if (!node) - break; - - ranges = of_get_property(node, "dma-ranges", &len); - - /* Ignore empty ranges, they imply no translation required */ - if (ranges && len > 0) - break; - - /* - * At least empty ranges has to be defined for parent node if - * DMA is supported - */ - if (!ranges) - break; - } - - if (!ranges) { - pr_debug("no dma-ranges found for node(%s)\n", np->full_name); - ret = -ENODEV; - goto out; - } - - len /= sizeof(u32); - - pna = of_n_addr_cells(node); - - /* dma-ranges format: - * DMA addr : naddr cells - * CPU addr : pna cells - * size : nsize cells - */ - dmaaddr = of_read_number(ranges, naddr); - *paddr = of_translate_dma_address(np, ranges); - if (*paddr == OF_BAD_ADDR) { - pr_err("translation of DMA address(%pad) to CPU address failed node(%s)\n", - dma_addr, np->full_name); - ret = -EINVAL; - goto out; - } - *dma_addr = dmaaddr; - - *size = of_read_number(ranges + naddr + pna, nsize); - - pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", - *dma_addr, *paddr, *size); - -out: - of_node_put(node); + struct of_bus *bus; - return ret; + /* get bus specific dma-ranges. */ + bus = of_match_bus(np); + return bus->get_dma_ranges(np, dma_addr, paddr, size); } EXPORT_SYMBOL_GPL(of_dma_get_range); -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges @ 2017-05-16 5:22 ` Oza Pawandeep via iommu 0 siblings, 0 replies; 38+ messages in thread From: Oza Pawandeep via iommu @ 2017-05-16 5:22 UTC (permalink / raw) To: Joerg Roedel, Robin Murphy Cc: devicetree-u79uwXL29TY76Z2rM5mHXA, Oza Pawandeep, linux-pci-u79uwXL29TY76Z2rM5mHXA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, bcm-kernel-feedback-list-dY08KVG/lbpWk0Htik3J/w, linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r current device framework and OF framework integration assumes dma-ranges in a way where memory-mapped devices define their dma-ranges. (child-bus-address, parent-bus-address, length). of_dma_configure is specifically written to take care of memory mapped devices. but no implementation exists for pci to take care of pcie based memory ranges. for e.g. iproc based SOCs and other SOCs(suc as rcar) have PCI world dma-ranges. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; this patch fixes the following problems of_dma_get_range. 1) return of wrong size as 0. 2) not handling absence of dma-ranges which is valid for PCI master. 3) not handling multipe inbound windows. 4) in order to get largest possible dma_mask. this patch also retuns the largest possible size based on dma-ranges, for e.g. dma-ranges = <0x43000000 0x00 0x00 0x00 0x00 0x80 0x00>; we should get dev->coherent_dma_mask=0x7fffffffff. based on which IOVA allocation space will honour PCI host bridge limitations. the implementation hooks bus specific callbacks for getting dma-ranges. Signed-off-by: Oza Pawandeep <oza.oza-dY08KVG/lbpWk0Htik3J/w@public.gmane.org> diff --git a/drivers/of/address.c b/drivers/of/address.c index 02b2903..800731c 100644 --- a/drivers/of/address.c +++ b/drivers/of/address.c @@ -6,6 +6,7 @@ #include <linux/ioport.h> #include <linux/module.h> #include <linux/of_address.h> +#include <linux/of_pci.h> #include <linux/pci.h> #include <linux/pci_regs.h> #include <linux/sizes.h> @@ -46,6 +47,8 @@ struct of_bus { int na, int ns, int pna); int (*translate)(__be32 *addr, u64 offset, int na); unsigned int (*get_flags)(const __be32 *addr); + int (*get_dma_ranges)(struct device_node *np, + u64 *dma_addr, u64 *paddr, u64 *size); }; /* @@ -171,6 +174,144 @@ static int of_bus_pci_translate(__be32 *addr, u64 offset, int na) { return of_bus_default_translate(addr + 1, offset, na - 1); } + +static int of_bus_pci_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + struct device_node *node = of_node_get(np); + int ret = 0; + struct resource_entry *window; + LIST_HEAD(res); + + if (!node) + return -EINVAL; + + *size = 0; + /* + * PCI dma-ranges is not mandatory property. + * many devices do no need to have it, since + * host bridge does not require inbound memory + * configuration or rather have design limitations. + * so we look for dma-ranges, if missing we + * just return the caller full size, and also + * no dma-ranges suggests that, host bridge allows + * whatever comes in, so we set dma_addr to 0. + */ + ret = of_pci_get_dma_ranges(np, &res); + if (!ret) { + resource_list_for_each_entry(window, &res) { + struct resource *res_dma = window->res; + + if (*size < resource_size(res_dma)) { + *dma_addr = res_dma->start - window->offset; + *paddr = res_dma->start; + *size = resource_size(res_dma); + } + } + } + pci_free_resource_list(&res); + + /* + * return the largest possible size, + * since PCI master allows everything. + */ + if (*size == 0) { + pr_debug("empty/zero size dma-ranges found for node(%s)\n", + np->full_name); + *size = DMA_BIT_MASK(sizeof(dma_addr_t) * 8) - 1; + *dma_addr = *paddr = 0; + ret = 0; + } + + pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", + *dma_addr, *paddr, *size); + + of_node_put(node); + + return ret; +} + +static int get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + struct device_node *node = of_node_get(np); + const __be32 *ranges = NULL; + int len, naddr, nsize, pna; + int ret = 0; + u64 dmaaddr; + + if (!node) + return -EINVAL; + + while (1) { + naddr = of_n_addr_cells(node); + nsize = of_n_size_cells(node); + node = of_get_next_parent(node); + if (!node) + break; + + ranges = of_get_property(node, "dma-ranges", &len); + + /* Ignore empty ranges, they imply no translation required */ + if (ranges && len > 0) + break; + + /* + * At least empty ranges has to be defined for parent node if + * DMA is supported + */ + if (!ranges) + break; + } + + if (!ranges) { + pr_debug("no dma-ranges found for node(%s)\n", np->full_name); + ret = -ENODEV; + goto out; + } + + len /= sizeof(u32); + + pna = of_n_addr_cells(node); + + /* dma-ranges format: + * DMA addr : naddr cells + * CPU addr : pna cells + * size : nsize cells + */ + dmaaddr = of_read_number(ranges, naddr); + *paddr = of_translate_dma_address(np, ranges); + if (*paddr == OF_BAD_ADDR) { + pr_err("translation of DMA address(%pad) to CPU address failed node(%s)\n", + dma_addr, np->full_name); + ret = -EINVAL; + goto out; + } + *dma_addr = dmaaddr; + + *size = of_read_number(ranges + naddr + pna, nsize); + + pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", + *dma_addr, *paddr, *size); + +out: + of_node_put(node); + + return ret; +} + +static int of_bus_isa_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + return get_dma_ranges(np, dma_addr, paddr, size); +} + +static int of_bus_default_get_dma_ranges(struct device_node *np, u64 *dma_addr, + u64 *paddr, u64 *size) +{ + return get_dma_ranges(np, dma_addr, paddr, size); +} + #endif /* CONFIG_OF_ADDRESS_PCI */ #ifdef CONFIG_PCI @@ -424,6 +565,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_pci_map, .translate = of_bus_pci_translate, .get_flags = of_bus_pci_get_flags, + .get_dma_ranges = of_bus_pci_get_dma_ranges, }, #endif /* CONFIG_OF_ADDRESS_PCI */ /* ISA */ @@ -435,6 +577,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_isa_map, .translate = of_bus_isa_translate, .get_flags = of_bus_isa_get_flags, + .get_dma_ranges = of_bus_isa_get_dma_ranges, }, /* Default */ { @@ -445,6 +588,7 @@ static unsigned int of_bus_isa_get_flags(const __be32 *addr) .map = of_bus_default_map, .translate = of_bus_default_translate, .get_flags = of_bus_default_get_flags, + .get_dma_ranges = of_bus_default_get_dma_ranges, }, }; @@ -820,74 +964,17 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index, * size : nsize cells * * It returns -ENODEV if "dma-ranges" property was not found - * for this device in DT. + * for this device in DT, except if PCI device then, dma-ranges + * can be optional property, and in that case returns size with + * entire host memory. */ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size) { - struct device_node *node = of_node_get(np); - const __be32 *ranges = NULL; - int len, naddr, nsize, pna; - int ret = 0; - u64 dmaaddr; - - if (!node) - return -EINVAL; - - while (1) { - naddr = of_n_addr_cells(node); - nsize = of_n_size_cells(node); - node = of_get_next_parent(node); - if (!node) - break; - - ranges = of_get_property(node, "dma-ranges", &len); - - /* Ignore empty ranges, they imply no translation required */ - if (ranges && len > 0) - break; - - /* - * At least empty ranges has to be defined for parent node if - * DMA is supported - */ - if (!ranges) - break; - } - - if (!ranges) { - pr_debug("no dma-ranges found for node(%s)\n", np->full_name); - ret = -ENODEV; - goto out; - } - - len /= sizeof(u32); - - pna = of_n_addr_cells(node); - - /* dma-ranges format: - * DMA addr : naddr cells - * CPU addr : pna cells - * size : nsize cells - */ - dmaaddr = of_read_number(ranges, naddr); - *paddr = of_translate_dma_address(np, ranges); - if (*paddr == OF_BAD_ADDR) { - pr_err("translation of DMA address(%pad) to CPU address failed node(%s)\n", - dma_addr, np->full_name); - ret = -EINVAL; - goto out; - } - *dma_addr = dmaaddr; - - *size = of_read_number(ranges + naddr + pna, nsize); - - pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n", - *dma_addr, *paddr, *size); - -out: - of_node_put(node); + struct of_bus *bus; - return ret; + /* get bus specific dma-ranges. */ + bus = of_match_bus(np); + return bus->get_dma_ranges(np, dma_addr, paddr, size); } EXPORT_SYMBOL_GPL(of_dma_get_range); -- 1.9.1 ^ permalink raw reply related [flat|nested] 38+ messages in thread
* Re: [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges 2017-05-16 5:22 ` Oza Pawandeep via iommu (?) (?) @ 2017-05-16 7:41 ` kbuild test robot -1 siblings, 0 replies; 38+ messages in thread From: kbuild test robot @ 2017-05-16 7:41 UTC (permalink / raw) To: Oza Pawandeep Cc: kbuild-all, Joerg Roedel, Robin Murphy, iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep, Oza Pawandeep [-- Attachment #1: Type: text/plain, Size: 1994 bytes --] Hi Oza, [auto build test ERROR on v4.9-rc8] [cannot apply to next-20170516] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Oza-Pawandeep/of-pci-dma-fix-DMA-configuration-for-PCI-masters/20170516-141222 config: c6x-evmc6678_defconfig (attached as .config) compiler: c6x-elf-gcc (GCC) 6.2.0 reproduce: wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=c6x All errors (new ones prefixed by >>): >> drivers//of/address.c:580:21: error: 'of_bus_isa_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_isa_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~ >> drivers//of/address.c:591:21: error: 'of_bus_default_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_default_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ vim +/of_bus_isa_get_dma_ranges +580 drivers//of/address.c 574 .addresses = "reg", 575 .match = of_bus_isa_match, 576 .count_cells = of_bus_isa_count_cells, 577 .map = of_bus_isa_map, 578 .translate = of_bus_isa_translate, 579 .get_flags = of_bus_isa_get_flags, > 580 .get_dma_ranges = of_bus_isa_get_dma_ranges, 581 }, 582 /* Default */ 583 { 584 .name = "default", 585 .addresses = "reg", 586 .match = NULL, 587 .count_cells = of_bus_default_count_cells, 588 .map = of_bus_default_map, 589 .translate = of_bus_default_translate, 590 .get_flags = of_bus_default_get_flags, > 591 .get_dma_ranges = of_bus_default_get_dma_ranges, 592 }, 593 }; 594 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 5161 bytes --] ^ permalink raw reply [flat|nested] 38+ messages in thread
* [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges @ 2017-05-16 7:41 ` kbuild test robot 0 siblings, 0 replies; 38+ messages in thread From: kbuild test robot @ 2017-05-16 7:41 UTC (permalink / raw) To: linux-arm-kernel Hi Oza, [auto build test ERROR on v4.9-rc8] [cannot apply to next-20170516] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Oza-Pawandeep/of-pci-dma-fix-DMA-configuration-for-PCI-masters/20170516-141222 config: c6x-evmc6678_defconfig (attached as .config) compiler: c6x-elf-gcc (GCC) 6.2.0 reproduce: wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=c6x All errors (new ones prefixed by >>): >> drivers//of/address.c:580:21: error: 'of_bus_isa_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_isa_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~ >> drivers//of/address.c:591:21: error: 'of_bus_default_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_default_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ vim +/of_bus_isa_get_dma_ranges +580 drivers//of/address.c 574 .addresses = "reg", 575 .match = of_bus_isa_match, 576 .count_cells = of_bus_isa_count_cells, 577 .map = of_bus_isa_map, 578 .translate = of_bus_isa_translate, 579 .get_flags = of_bus_isa_get_flags, > 580 .get_dma_ranges = of_bus_isa_get_dma_ranges, 581 }, 582 /* Default */ 583 { 584 .name = "default", 585 .addresses = "reg", 586 .match = NULL, 587 .count_cells = of_bus_default_count_cells, 588 .map = of_bus_default_map, 589 .translate = of_bus_default_translate, 590 .get_flags = of_bus_default_get_flags, > 591 .get_dma_ranges = of_bus_default_get_dma_ranges, 592 }, 593 }; 594 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation -------------- next part -------------- A non-text attachment was scrubbed... Name: .config.gz Type: application/gzip Size: 5161 bytes Desc: not available URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170516/6c24fb7d/attachment-0001.gz> ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges @ 2017-05-16 7:41 ` kbuild test robot 0 siblings, 0 replies; 38+ messages in thread From: kbuild test robot @ 2017-05-16 7:41 UTC (permalink / raw) To: Oza Pawandeep Cc: devicetree, Oza Pawandeep, linux-pci, Joerg Roedel, linux-kernel, iommu, bcm-kernel-feedback-list, kbuild-all, Oza Pawandeep, Robin Murphy, linux-arm-kernel [-- Attachment #1: Type: text/plain, Size: 1994 bytes --] Hi Oza, [auto build test ERROR on v4.9-rc8] [cannot apply to next-20170516] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Oza-Pawandeep/of-pci-dma-fix-DMA-configuration-for-PCI-masters/20170516-141222 config: c6x-evmc6678_defconfig (attached as .config) compiler: c6x-elf-gcc (GCC) 6.2.0 reproduce: wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=c6x All errors (new ones prefixed by >>): >> drivers//of/address.c:580:21: error: 'of_bus_isa_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_isa_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~ >> drivers//of/address.c:591:21: error: 'of_bus_default_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_default_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ vim +/of_bus_isa_get_dma_ranges +580 drivers//of/address.c 574 .addresses = "reg", 575 .match = of_bus_isa_match, 576 .count_cells = of_bus_isa_count_cells, 577 .map = of_bus_isa_map, 578 .translate = of_bus_isa_translate, 579 .get_flags = of_bus_isa_get_flags, > 580 .get_dma_ranges = of_bus_isa_get_dma_ranges, 581 }, 582 /* Default */ 583 { 584 .name = "default", 585 .addresses = "reg", 586 .match = NULL, 587 .count_cells = of_bus_default_count_cells, 588 .map = of_bus_default_map, 589 .translate = of_bus_default_translate, 590 .get_flags = of_bus_default_get_flags, > 591 .get_dma_ranges = of_bus_default_get_dma_ranges, 592 }, 593 }; 594 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 5161 bytes --] [-- Attachment #3: Type: text/plain, Size: 176 bytes --] _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel ^ permalink raw reply [flat|nested] 38+ messages in thread
* Re: [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges @ 2017-05-16 7:41 ` kbuild test robot 0 siblings, 0 replies; 38+ messages in thread From: kbuild test robot @ 2017-05-16 7:41 UTC (permalink / raw) Cc: kbuild-all, Joerg Roedel, Robin Murphy, iommu, linux-pci, linux-kernel, linux-arm-kernel, devicetree, bcm-kernel-feedback-list, Oza Pawandeep, Oza Pawandeep [-- Attachment #1: Type: text/plain, Size: 1994 bytes --] Hi Oza, [auto build test ERROR on v4.9-rc8] [cannot apply to next-20170516] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Oza-Pawandeep/of-pci-dma-fix-DMA-configuration-for-PCI-masters/20170516-141222 config: c6x-evmc6678_defconfig (attached as .config) compiler: c6x-elf-gcc (GCC) 6.2.0 reproduce: wget https://raw.githubusercontent.com/01org/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree make.cross ARCH=c6x All errors (new ones prefixed by >>): >> drivers//of/address.c:580:21: error: 'of_bus_isa_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_isa_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~ >> drivers//of/address.c:591:21: error: 'of_bus_default_get_dma_ranges' undeclared here (not in a function) .get_dma_ranges = of_bus_default_get_dma_ranges, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ vim +/of_bus_isa_get_dma_ranges +580 drivers//of/address.c 574 .addresses = "reg", 575 .match = of_bus_isa_match, 576 .count_cells = of_bus_isa_count_cells, 577 .map = of_bus_isa_map, 578 .translate = of_bus_isa_translate, 579 .get_flags = of_bus_isa_get_flags, > 580 .get_dma_ranges = of_bus_isa_get_dma_ranges, 581 }, 582 /* Default */ 583 { 584 .name = "default", 585 .addresses = "reg", 586 .match = NULL, 587 .count_cells = of_bus_default_count_cells, 588 .map = of_bus_default_map, 589 .translate = of_bus_default_translate, 590 .get_flags = of_bus_default_get_flags, > 591 .get_dma_ranges = of_bus_default_get_dma_ranges, 592 }, 593 }; 594 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 5161 bytes --] ^ permalink raw reply [flat|nested] 38+ messages in thread
end of thread, other threads:[~2017-05-22 16:42 UTC | newest] Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-05-16 5:22 [PATCH v6 0/3] OF/PCI address PCI inbound memory limitations Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep 2017-05-16 5:22 ` [PATCH v6 1/3] of/pci/dma: fix DMA configuration for PCI masters Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep via iommu 2017-05-17 17:10 ` Bjorn Helgaas 2017-05-17 17:10 ` Bjorn Helgaas 2017-05-17 17:10 ` Bjorn Helgaas 2017-05-17 17:10 ` Bjorn Helgaas 2017-05-19 1:37 ` Oza Oza 2017-05-19 1:37 ` Oza Oza 2017-05-19 1:37 ` Oza Oza via iommu 2017-05-17 19:13 ` Arnd Bergmann 2017-05-17 19:13 ` Arnd Bergmann 2017-05-17 19:13 ` Arnd Bergmann 2017-05-19 2:58 ` Oza Oza 2017-05-19 2:58 ` Oza Oza 2017-05-19 2:58 ` Oza Oza via iommu 2017-05-22 16:42 ` Oza Oza 2017-05-22 16:42 ` Oza Oza 2017-05-22 16:42 ` Oza Oza via iommu 2017-05-16 5:22 ` [PATCH v6 2/3] iommu/pci: reserve IOVA " Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep via iommu 2017-05-17 17:11 ` Bjorn Helgaas 2017-05-17 17:11 ` Bjorn Helgaas 2017-05-17 17:11 ` Bjorn Helgaas 2017-05-17 17:11 ` Bjorn Helgaas 2017-05-19 1:21 ` Oza Oza 2017-05-19 1:21 ` Oza Oza 2017-05-19 1:21 ` Oza Oza via iommu 2017-05-16 5:22 ` [PATCH v6 3/3] PCI/of fix of_dma_get_range; get PCI specific dma-ranges Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep 2017-05-16 5:22 ` Oza Pawandeep via iommu 2017-05-16 7:41 ` kbuild test robot 2017-05-16 7:41 ` kbuild test robot 2017-05-16 7:41 ` kbuild test robot 2017-05-16 7:41 ` kbuild test robot
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.